Hi,

I've got [ External links are visible to forum administrators only ]. Is a large dog site in Spanish language.
When I bought Sitemap Generator I was very happy, cause my website went up to #1 in Google (I had no sitemap so far).
I made many sitemaps, with the following results (cronologically):
#    |      date/time       |  pages    |proc time|bandwidth|newURL|removedURL|BorkenLinks
 2   2007-01-08 22:16    31,926    4,197.20s      239.49    0    0    30
3    2007-03-28 23:12    25,241    14,359.00s    703.53    0    0    2
4    2007-03-31 19:02    81,999    75,373.00s 2,261.73    0    0    2
5    2007-04-02 10:32    81,986    49,752.00s 2,258.68    0    0    2
6    2007-05-12 08:57    99,308    72,661.00s 2,691.65    1000    1000    1
7    2007-05-13 19:25      122,893  82,961.00s 3,295.61    1000    1000    1
8    2007-09-23 20:32      5,201    2,432.60s    174.74    1000    1000    0
9    2007-09-23 21:17      5,208    2,528.70s    174.96    7    1000    0
10    2007-09-23 22:34     5,216      2,637.40s    175.10    8    1000    0
11    2007-09-24 00:15     5,225    1,936.50s    175.39    9    1000    0
12    2007-09-24 00:49     5,226      1,948.40s    175.32    1    1000    0
13    2007-09-24 01:30     5,233    1,954.70s    175.52    7    1000    0
14    2007-09-24 10:32    5,252    1,957.20s    176.10    19    1000    0
15    2007-09-24 12:12    5,269    2,041.80s    176.54    17    1000    0
16    2007-09-24 12:50    5,273    2,041.20s    176.67    4    1000    0
17    2007-09-24 13:59    5,283    2,054.80s    176.97    10    1000    0
18    2007-09-25 02:00    6,140    2,398.80s    186.71    858    1000    0
19    2007-09-25 16:00    6,162    2,117.30s    187.42    57    1000    0
20    2007-09-25 16:38    6,162    2,106.70s    187.42    24    1000    0

You can see that the latest several times I tried to make a new sitemap (with the same configuration, I didn't touch it at all), won't go over 6.000 pages. I've got a forum with over 800.000 posts! 6.000 pages is nothing!

Another thing a noticed, is that when it used to make the big ones (up to 122K pages) is that it used to crawl several "Link depth" (up to 5, I think I recall), but now stops at 2, and shows as it's done, I mean, it doesn't show an error message, just finishes.

Please, I need to solve this. Could you gimme a hand?

Best regards.

Pablo (Spain)
« Last Edit: September 25, 2007, 08:38:18 PM by Todo Perros »
Re: Used to index 122.000 pages, but now no more than 6.000. Why? what happens?
« Reply #2 on: September 26, 2007, 08:18:18 AM »
Hi Oleg,

No, i did not change anything, other than the normal growing of the site. That's the weird thing.
I see the site crawling some of the forum pages (only some of them), but not listing it in the sitemap.