Sitemap Generator Forum

Possibly you had 2 sitemap generator processes running at the same time.

Negative on two at one time. It was a manual start.

I still have duplicate sitemaps in the index file. If there were 2 at one time, I would have one index file that was written over.

The example is still there if you want to look at the url.

In this case you need to manually run it in command line (instead of cron task) and check to make sure that it doesn't interrupt and completes the whole process in one step without interrupting.

I did start it as a manual crawl. That is how I know it was not a duplicate crawl.

I do not have a cron job presently set up to run.

The crawl went through without the browser closing and it was a single session.

However when it went to generate the files when it got to 100 percent crawled I took the screen shots. It shows that it got to a certain point,(like 25 percent of the file generated) then stopped generating the file and went back to a previous crawl point, crawled for a while. It then stopped crawling, and picked up at the 25 percent and got to maybe 40 percent of the file generation and then started to crawl again.

When everything was said and done, the sitemap had duplicate files in it.

URL   Last Modified
1   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
2   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
3   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
4   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
5   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
6   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
7   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
8   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
9   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
10   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
11   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
12   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
13   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
14   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
15   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
16   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00
17   [ External links are visible to forum administrators only ]   2013-06-16T06:51:57+00:00

That is what the sitemap.xml.gz looks like.

Note the duplication of Sitemap 3,5,7,9,and 11.

Google now is reporting that I have submitted more url's than I have on my website.

I think they look at that as spaming their servers.

I am waiting for a suggestion before I re crawl it.

Thank you.

> It shows that it got to a certain point,(like 25 percent of the file generated) then stopped generating

Server setup should be in the way allowing generator to create full sitemap without interruption to avoid possible issues.

There is nothing wrong with the server.

I run Cron Jobs on it all the time. Go Daddy will confirm that it is functioning the way it should.

If you have another test you would like me to run I will run it.

You can check the server information here: