Tools for Any SEO Challenge: Get more traffic to your website with versatile solutions for every step of your SEO journey.
« previous next »
Pages: 1

info821

*
  • *
  • 2
How To Crawl ONLY New URLs?
« on: March 10, 2009, 01:14:50 AM »
Hello,

We have a large site. Each time the sitemap crawls it takes about 7-10 hours.

How can we crawl only new pages to add to the sitemap?

Thank you
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: How To Crawl ONLY New URLs?
« Reply #1 on: March 10, 2009, 12:31:55 PM »
Hello,

in order to find your new page sitemap generator should crawl other pages, finding the link to new pages.
In case if all your new pages are localized, for instance in news/ folder, you can specify it as starting URL and no URLs "above" that folder will be crawled by generator.
Logged
Pages: 1
« previous next »