Tools for Any SEO Challenge: Get more traffic to your website with versatile solutions for every step of your SEO journey.
« previous next »
Pages: 1

admin103

*
  • *
  • 4
Crawling keeps stopping at 5000 ish
« on: October 17, 2007, 09:13:01 AM »
Hiya,

I keep trying to fully XML-Sitemap my site, but the script keep stopping around the 5000 links mark.

Any ideas?  Need to see anything from my site?


Many thanks.
Logged

XML-Sitemaps Support

*
  • *****
  • 11782
Re: Crawling keeps stopping at 5000 ish
« Reply #1 on: October 17, 2007, 11:57:46 AM »
Hello,

please try to increase max_execution_time and memory_limit settings in php configuration on your host.
Logged

admin103

*
  • *
  • 4
Re: Crawling keeps stopping at 5000 ish
« Reply #2 on: October 17, 2007, 06:12:04 PM »
Hiya,

I did do. I read a lot of forum posted first to try and work it out myself. Settings are:-

max_execution_time = 9000
memory_limit = 500M

The script seems to run for about 5 - 10 mins. I then have to "Resume last session" for it continue. This general does work, but it stops when it gets to approx 5000 links. Its currently stuck on "Continue the interrupted session (2007-10-13 16:04:16, URLs added: 4783, estimated URLs left: 17363)".

Any ideas?
Logged

XML-Sitemaps Support

*
  • *****
  • 11782
Re: Crawling keeps stopping at 5000 ish
« Reply #3 on: October 18, 2007, 12:07:10 PM »
Hello,

could you please let me know your generator URL in private message?
Logged

admin103

*
  • *
  • 4
Re: Crawling keeps stopping at 5000 ish
« Reply #4 on: October 18, 2007, 12:25:53 PM »
PM sent.

Thanks.
Logged
Pages: 1
« previous next »