help ! The crawler never finishes. Any idea why ?
« on: September 01, 2006, 03:52:36 PM »
I have over 100,000 pages on my website.  The crawler never finishes.  Any idea why ?
Re: help ! The crawler never finishes. Any idea why ?
« Reply #1 on: September 02, 2006, 06:22:29 PM »
Hello,

as described in Cannot complete sitemap generation - crawler stops after some time:
Quote
- increase max_execution_time setting in php.ini file at your server and restart apache
- execute sitemap generator from command line (if you have ssh access to your server)
- use "Save state" option at Sitemap Generator configuration page and execute the crawler multiple times with "Resume generation" enabled until full sitemap is created
- limit the number of pages to index in Sitemap Generator configuration
Re: help ! The crawler never finishes. Any idea why ?
« Reply #2 on: September 02, 2006, 07:11:34 PM »
Maybe U can add <java script> and test, run again to end crawling?
Re: help ! The crawler never finishes. Any idea why ?
« Reply #3 on: September 05, 2006, 03:22:10 AM »
Thank you for suggestion, we will consider this option for the next versions of sitemap generator.