I did start it as a manual crawl. That is how I know it was not a duplicate crawl.
I do not have a cron job presently set up to run.
The crawl went through without the browser closing and it was a single session.
However when it went to generate the files when it got to 100 percent crawled I took the screen shots. It shows that it got to a certain point,(like 25 percent of the file generated) then stopped generating the file and went back to a previous crawl point, crawled for a while. It then stopped crawling, and picked up at the 25 percent and got to maybe 40 percent of the file generation and then started to crawl again.
When everything was said and done, the sitemap had duplicate files in it.
URL Last Modified
1 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
2 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
3 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
4 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
5 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
6 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
7 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
8 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
9 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
10 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
11 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
12 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
13 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
14 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
15 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
16 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
17 [ External links are visible to forum administrators only ] 2013-06-16T06:51:57+00:00
That is what the sitemap.xml.gz looks like.
Note the duplication of Sitemap 3,5,7,9,and 11.
Google now is reporting that I have submitted more url's than I have on my website.
I think they look at that as spaming their servers.
I am waiting for a suggestion before I re crawl it.
Thank you.