Sitemap Generator Forum

When I try to run the crawler it shows "Please wait. Sitemap generation in progress..."

Then.. this shows up:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

Apache Server at mydomain.com Port 80

Please help.

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.

I have already adjusted my memory limit to 128M and the max execution has already been adjusted to 3000.  Those are the settings while I'm getting that error. 

Anything else to check?

Try to increase them to 256M and 9000.

I changed it to 256M and 9000, and the internal server error is gone.  However, now the crawler is timing out after only about 5 minutes and 350 pages of 3000 crawled.

Crawler is stopping at this point:

Links depth: 2
Current page: zip-code-search/85050-phoenix-az/
Pages added to sitemap: 156
Pages scanned: 160 (8,031.2 KB)
Pages left: 350 (+ 3611 queued for the next depth level)
Time passed: 0:04:31
Time left: 0:09:54
Memory usage: 3,227.4 Kb

auto-restart monitoring: tue apr 26 2011 10:17:00 gmt 0700 (100 seconds since last update)
tue apr 26 2011 10:15:00 gmt 0700: resuming generator (120 seconds with no response)
tue apr 26 2011 10:13:00 gmt 0700: resuming generator (120 seconds with no response)

it goes 120 seconds no response and then goes through same message  continually

Hello,

please let me know your generator URL/login in private message to check this.

Not happy with this product.  I ran the scenario several times, and each time I upload the generator directory, change the execution and memory limits to what you suggest my load time for my site slows to the pace of a snail.  When I remove the generator directory it goes back to normal.

I give up.

Hello,

didn't you setup a scheduled task to run generator by any chance?