Sitemap-Generator does not finish crawling.
« on: September 11, 2011, 09:49:50 AM »
Hello,

I have problems with the sitemap-generator.
It does not finish crawling the website.

It allways says something like "Auto-restart monitoring: Sun Sep 11 2011 10:47:07 GMT+0200 (26 second(s) since last update)"
and
"Sun Sep 11 2011 10:49:22 GMT+0200: resuming generator (120 seconds with no response)"

maybe you can check the settings on [ External links are visible to forum administrators only ]

Thanks in advance.
Re: Sitemap-Generator does not finish crawling.
« Reply #1 on: September 11, 2011, 03:31:59 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Re: Sitemap-Generator does not finish crawling.
« Reply #2 on: September 11, 2011, 04:33:31 PM »
ok now memory limit is 92MB and the max_execution_time ist 60s, but I still have the same problem.

can I change some settings of the sitemap generator to reduce the execution time of the script?
Re: Sitemap-Generator does not finish crawling.
« Reply #3 on: September 12, 2011, 11:27:03 AM »
Hello,

it is possible to limit the number of URLs to index or maximum depth level in generator configuration.
Re: Sitemap-Generator does not finish crawling.
« Reply #4 on: September 12, 2011, 11:31:00 AM »
ok and which configurations for the number of URLs and the maximum depth level would you recommend with the memory limit of 92 MB and the max_execution_time of 60s?
Re: Sitemap-Generator does not finish crawling.
« Reply #5 on: September 13, 2011, 08:05:10 PM »
Hello,

can you provide a temporary ftp access via private message for troubleshooting?
Re: Sitemap-Generator does not finish crawling.
« Reply #6 on: September 27, 2011, 09:31:54 PM »
I get this same problem too. Google has over 2k pages indexed but the most sitemaps has ever gotten to is 1,400. I've tried repeatedly for the last couple of days in case it was just the server being sluggish but never seems to quite get to the end and at the last run said there was only just over 1k pages. Can you point me in the right direction?
Re: Sitemap-Generator does not finish crawling.
« Reply #7 on: September 28, 2011, 11:13:28 AM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?