stop at depth 3
« on: June 28, 2013, 04:03:12 PM »
a week ago i install and i crawl until depth 10 , and then i upgrade to dedicated server , i use the same setting i crawl from new server and it stop at depth 3 ? can anyone help please , total links different are 35,000 links different.

2896 queued for the next depth level but it doesn't continue , it still scanning and moving , but the number isn't moving at all , Pages added to sitemap: 2748 is remain unchange no matter how long u wait for it to scan.


Links depth: 3
Current page: auction_search.php?start=40&limit=10&item_type=buy_out &order_field=a.name&order_type=DESC
Pages added to sitemap: 2748
Pages scanned: 2958 (551,195.4 KB)
Pages left: 7917 (+ 2896 queued for the next depth level)
Time passed: 0:17:28
Time left: 0:46:47
Memory usage: 15,094.7 Kb
« Last Edit: June 28, 2013, 04:14:18 PM by tajima666 »
Re: stop at depth 3
« Reply #1 on: July 05, 2013, 08:26:20 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 
Re: stop at depth 3
« Reply #2 on: July 06, 2013, 01:14:46 AM »
i already told them to do it , but still same thing , can u give me the configuration or setting that needed to change ?
Re: stop at depth 3
« Reply #3 on: July 08, 2013, 10:11:00 PM »
Hello,

please let me know your generator URL/login in private message to check this.
Re: stop at depth 3
« Reply #4 on: July 25, 2013, 06:55:37 AM »
Dear Admin,

Need you help. I have been using the generator without much issue. Yesterday, I revamped my website, copied the generator folder and sitemap.xml and ror.xml into the new site. Rights has been given to both xml and generator/data folder.

Excution time set to 3000sec
Memory set to 256m
Running on a Window server

But it stop at:

Links depth: 5
Current page: usr/product.aspx?pgid=8&grpid=97&id=2515¤cy=MYR
Pages added to sitemap: 541
Pages scanned: 639 (94,151.9 KB)
Pages left: 2490 (+ 5837 queued for the next depth level)
Time passed: 0:12:18
Time left: 0:47:55
Memory usage: 5,526.6 Kb

Will you be able to identify the problem? I did not upload the new generator launched in May 2012.
Re: stop at depth 3
« Reply #5 on: July 25, 2013, 07:42:45 AM »
they still haven't fix my issue yet which is similar to yours  :-\
Re: stop at depth 3
« Reply #6 on: July 26, 2013, 07:48:35 AM »
Hello,

in case of Windows server it might be needed to increase script timeout in IIS configuration as described on:
[ External links are visible to logged in users only ]
[ External links are visible to logged in users only ]