Sitemap Generator Forum

Hi,

After patiently trying for weeks and reading lots of posts regarding all relevant errors and having tried all the tips posted, I am having to start this thread.

My website has over 11,000 links and the sitemap generator should be able to crawl them all. However, it stops abrubtly at 3950 and does not go any further. This is getting really frustreating and I would appreciate your help.

All the directory and file permissions are as they should be. Base URL to crawl includes "www" prefix.

My php.ini settings are as follows:

memory_limit = 3072M
upload_max_filesize = 48M
post_max_size = 360M
max_execution_time = 60000

The most recent screenshot is as follows:



Please HELP!

Hello,

from what I see, there should be no issues in crawling the site in your case. As the first step, can you please try to install the most recent version of sitemap generator (v2.9), remove all files from generator/data/ folder and start to regenerate the sitemap.

Dear Admin,

Thank you for replying.

Since submitting the above post, I have upgraded to v2.9, removing all previous files but the results are not so different.

It does not seem to go beyond the 3950 mark for some reason. Lowering the memory limit, however, decreases the number of crawled links.

Any ideas?

Possibly you have direct link to large files (like videos) which should be excluded from crawling. Please PM me your generator URL so that I can check it.

Thanks for you reply.

I do not have any links to large files; e.g. videos etc.

I am now sending you the generator URL via PM.

Thanks

The same problem for me, other time give the next error.

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 44 bytes) in /usr/home/anunciosgratis/www/generator/pages/class.utils.inc.php(2) : eval()'d code on line 6

How many URLs do you have crawled at that point?