Sitemap Generator Forum

Hi, my crawl script hangs all the time, no particular number of pages can be 20,000 or 75,000 - sometimes crawls for 6hrs others 6mins,

please advise

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 

that would sound right if it wasn't for the fact that the script has previously created a sitemap with 150k pages.

memory is set at 512 and execution time 90 seconds - hang iscompletely random - can run for 20k pages or 6 pages and then stops.

please advise

any chance of some advice on this - current crawl hangs just after 148935 pages  and will not go any further - just stops

some further info on this is that i have checked the debug output and it hangs every time it tries to saves it's progress.

Hello,

looks like you would need to increase memory_limit even more then, please try 800M.
Also,  in this case I would recommend to run generator in command line if you have ssh access to your server.