Sitemap Generator Forum

Hi,

Im using the sitemap generator for several websites and everything is working great.
For one of my "bigger" websites something odd is happening.

Now and again (I run my sitemap once every 3 days using a cronjob) a link to an article is removed and the next time the sitemap tool crawls my site the link is added again. The links that are removed and then added are different every time but they are all located in the "archive" section of my site. To load my archive page takes a couple of seconds because of the number of articles in that section.

The articles them self don't disappear from the archive section as far as I can see, nor do I add articles to that section.

In short: Why does the sitemap generator tool remove a link to an article on 1 crawl only to add that link again the next crawl?

Thanks in advance.

ps. I'm using Standalone Sitemap Generator (PHP) v2.9

Hello,

in case if it takes long to open those pages, may be generator requests for those pages are timing out from time to time. Ideal solution would be to make those pages smaller (increase number of pages but decrease page size), since otherwise search engine crawlers will have issues when visiting your site as well.

I figured as much as well, but it's only 1 link every time it happens. And I checked to today, and today it was a link in a completely different section with less articles in it.