Error when I trying to crawl my site
« on: July 06, 2012, 02:28:56 AM »
Hello there,

I have a problem,  I trying to crawl my site but I receive the error evertime that I do the task:
I got this error:
============================
Links depth: 2
 Current page: order-online-en.html
 Pages added to sitemap: 820
 Pages scanned: 820 (22,643.2 KB)
Pages left: 310 (+ 3893 queued for the next depth level)
Time passed: 0:10:50
 Time left: 0:04:05
 Memory usage: 3,053.5 Kb
 Resuming the last session (last updated: 2012-07-06 00:17:39)
Server error!
 
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there was an error in a CGI script.
 
If you think this is a server error, please contact the webmaster.

Error 500
 
mywebsiteexample.com
Fri Jul 6 02:16:34 2012
 Apache/2.2.11
==========================


Any help please..!!!!!
Re: Error when I trying to crawl my site
« Reply #1 on: July 06, 2012, 06:07:42 AM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Re: Error when I trying to crawl my site
« Reply #2 on: July 07, 2012, 01:13:23 AM »
I increased everthing and I still have the same problem:

max_execution_time:       was 30 and I tried 120, 200, 500 and more :(

memory_limit:      was 64M and I tried to 256M, 512M and more  :(


any other idea?
Re: Error when I trying to crawl my site
« Reply #3 on: July 09, 2012, 02:47:17 PM »
Hello,

please let me know your generator URL/login in private message to check this.