Toggle navigation
PRO Sitemaps
Standalone Generator
About Sitemaps
News
SEO Tools
Sitemap Generator Forum
Guest
Login
Register
Menu
Home
Help
Search
Login
Register
XML Sitemaps Discussions Category
/
Unlimited PHP Sitemap Generator
/
crawl keeps hanging
Tools for Any SEO Challenge:
Get more traffic to your website with versatile solutions for every step of your SEO journey.
« previous
next »
Pages:
1
jojor2004
9
crawl keeps hanging
«
on:
October 24, 2014, 10:02:20 PM »
Hi, my crawl script hangs all the time, no particular number of pages can be 20,000 or 75,000 - sometimes crawls for 6hrs others 6mins,
please advise
Logged
XML-Sitemaps Support
11781
Re: crawl keeps hanging
«
Reply #1 on:
October 25, 2014, 02:49:38 PM »
Hello,
it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Logged
Oleg Ignatiuk
https://www.xml-sitemaps.com
Send me a Private Message
SEM and SEO Reports, more than 45M domains
: The world's leading Competitive Intelligence Tool for digital marketing.
jojor2004
9
Re: crawl keeps hanging
«
Reply #2 on:
October 25, 2014, 04:54:52 PM »
that would sound right if it wasn't for the fact that the script has previously created a sitemap with 150k pages.
Logged
jojor2004
9
Re: crawl keeps hanging
«
Reply #3 on:
October 25, 2014, 10:07:16 PM »
memory is set at 512 and execution time 90 seconds - hang iscompletely random - can run for 20k pages or 6 pages and then stops.
please advise
Logged
jojor2004
9
Re: crawl keeps hanging
«
Reply #4 on:
October 26, 2014, 04:45:27 PM »
any chance of some advice on this - current crawl hangs just after 148935 pages and will not go any further - just stops
Logged
jojor2004
9
Re: crawl keeps hanging
«
Reply #5 on:
October 26, 2014, 04:55:44 PM »
some further info on this is that i have checked the debug output and it hangs every time it tries to saves it's progress.
Logged
XML-Sitemaps Support
11781
Re: crawl keeps hanging
«
Reply #6 on:
October 27, 2014, 04:43:07 AM »
Hello,
looks like you would need to increase memory_limit even more then, please try 800M.
Also, in this case I would recommend to run generator in command line if you have ssh access to your server.
Logged
Oleg Ignatiuk
https://www.xml-sitemaps.com
Send me a Private Message
SEM and SEO Reports, more than 45M domains
: The world's leading Competitive Intelligence Tool for digital marketing.
Pages:
1
« previous
next »