Competitor Analysis Tool: Analyze rival websites and gain valuable SEO and PPC insights
« previous next »
Pages: 1

blogger

*
  • **
  • 10
excessive traffic .Possible to avoid?
« on: July 04, 2010, 03:01:32 AM »
Hello Oleg.
I ran an unlimited version of sitemap generator on one of my websites and it created a sitemap of about 2300 urls before i stopped it. For those 2300 urls sitemap generated 180 mb of data.
My question is.. is it possible to create sitemap without actually loading page/graphics etc to reduce bandwidth. One of my sites has near 100k links and traffic on my host is limited.

Please let me know.
P.S. I tried to post in section of this forum dedicated to unlimited version but wasn't able to.

Thanks
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: excessive traffic .Possible to avoid?
« Reply #1 on: July 04, 2010, 08:38:18 AM »
Hello,

image/css/etc files are never loaded by sitemap generator, only the html part is crawled. Also, you can use "Do not parse" option to skip certain pages from crawling, while still adding them in sitemap.
Logged

blogger

*
  • **
  • 10
Re: excessive traffic .Possible to avoid?
« Reply #2 on: July 05, 2010, 03:31:49 PM »
Thanks for reply.

Are these stats below look normal?

Links depth: 13
Current page: the-northern-face-cayenne
Pages added to sitemap: 12154
Pages scanned: 13360 (1,198,314.1 KB)
Pages left: 1596 (+ 3755 queued for the next depth level)
Time passed: 15:27:44
Time left: 1:50:49
Memory usage: 26,701.6 Kb
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: excessive traffic .Possible to avoid?
« Reply #3 on: July 06, 2010, 08:10:56 AM »
Hello,

yes, it makes just about 2Kb per page, which is rather lower than average page size I would say.
Logged

blogger

*
  • **
  • 10
Re: excessive traffic .Possible to avoid?
« Reply #4 on: July 10, 2010, 06:57:01 AM »
Oleg,Thank you for quick reply. Could you please explain how you gotten 2kb/page?
in above stats i divided 1198314 / 13360  = 89k

Am I missing something?
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: excessive traffic .Possible to avoid?
« Reply #5 on: July 10, 2010, 03:54:40 PM »
Sorry, my mistake, I used 26M memory usage for calculation. 89K is probably somewhat higher than average page. Can you check the size of html source for one of your pages?
Logged
Pages: 1
« previous next »