Tools for Any SEO Challenge: Get more traffic to your website with versatile solutions for every step of your SEO journey.
« previous next »
Pages: 1

alexghiurco

*
  • *
  • 3
Generator stop after crawling few thousand
« on: January 27, 2011, 02:33:37 PM »
Hello.
Just like the title say, generator stops after crawling about 3,5k or 4 url`s.
It stops in manual and also in cron job.
I don`t know what to do , to finish the crawling in auto mode.
Please help me .
Thank you.
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: Generator stop after crawling few thousand
« Reply #1 on: January 27, 2011, 03:17:22 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Logged

design2

*
  • *
  • 3
Re: Generator stop after crawling few thousand
« Reply #2 on: January 28, 2011, 12:22:49 PM »
I am having the same problem (it stops at 1630 files). My php.ini max_execution_time is set at 60. What should it ideally be?
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: Generator stop after crawling few thousand
« Reply #3 on: January 28, 2011, 11:11:39 PM »
That depends on the number of pages that will be found, try to increase it to 600.
Logged

design2

*
  • *
  • 3
Re: Generator stop after crawling few thousand
« Reply #4 on: January 29, 2011, 12:10:48 PM »
Unfortunately increasing the php.ini max_execution_time to 600 made no difference, it still stops at 1629 files.
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: Generator stop after crawling few thousand
« Reply #5 on: January 29, 2011, 01:24:28 PM »
Hello,

please let me know your generator URL/login in private message to check this.
Logged

design2

*
  • *
  • 3
Re: Generator stop after crawling few thousand
« Reply #6 on: January 29, 2011, 02:32:56 PM »
I just sent you a PM
Logged

piemant

*
  • *
  • 4
Re: Generator stop after crawling few thousand
« Reply #7 on: January 29, 2011, 03:15:03 PM »
Similar problem, I've sent a PM.
Logged
Pietro Mantovani

alexghiurco

*
  • *
  • 3
Re: Generator stop after crawling few thousand
« Reply #8 on: January 29, 2011, 10:05:29 PM »
I have incresead memory limit at 128M and time at 600 but generator still stops at 5k url. Any ideeas?
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: Generator stop after crawling few thousand
« Reply #9 on: January 30, 2011, 07:34:13 AM »
Hello,

please let me know your generator URL/login in private message to check this.
Logged
Pages: 1
« previous next »