I have used xml sitemaps for the past year with out much problem. My site has about 60,000 pages and it would run 3 or 4 hours than stop. I would start it again from where it left off and it would finish up and create a good site map.
Now, out of the blue I can't get a good site map created, it'll run for a few hours and stop at around 25,000 pages. When I tried to restart it keeps stopping after about 5 minutes so I can't get a complete site map anymore.
My site is on a dedicated managed server and I have been working with the hosting companies that manages my server. They have no idea why is would stop and are asking me I can give them any error messages. Is there a way to log errors so we can trouble shoot this? Any idea what the problem could be?
I tried again and it stopped scanning site again, here is the read out:
Links depth: 3
Current page: articles.detail.php/41208/145/Recreation_and_Leisure/24/Camping_Gear
Pages added to sitemap: 24180
Pages scanned: 45240 (587,131.5 KB)
Pages left: 9039 (+ 43369 queued for the next depth level)
Time passed: 144:56
Time left: 28:57
Memory usage: 57,071.3 Kb
Guest