Sitemap Generator Forum

When I start the crawler, the screen seems to frozen after only few minutes... Does it mean the script is stopped? How can I know if the script is in run or not?

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.

I am having the same problem, what should these variables be set to? I have a site with almost 50 000 pages

Hello,

max_execution_time depends on the page generation time of your site mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page on your site, then 1000 pages will be crawled in about 16 minutes (and max_execution_time should be at least 1000).

Hi,

I tried to put the memory at 500MB, crawl to max level of 3 (8000 pages)  and put a delay of 1s between every request
after 1 or 2 mn it freeze

Links depth: 2
Current page: french-alps/isere
Pages added to sitemap: 278
Pages scanned: 320 (10,179.3 KB)
Pages left: 1405 (+ 513 queued for the next depth level)
Time passed: 1:15
Time left: 5:30
Memory usage: 1,498.2 Kb

I don't think the problem is coming from the memory, can you help me please?

Thanks

Mat

There should be no issues with memory limit with this number of links. Please PM me your generator URL/login so that I can check it.