Sitemap Generator Forum

Our SEO guru is on vacation and I've got just enough experience to be dangerous, so I really appreciate any and all ideas in advance...
We have a client with 3 domains that all reference the same materials. My guess is soemthing is wong wiht indexing for a couple of reasons-

1. site: is only returning pages for one of the domains...the one that was set up as the dominant domain
2. site:  only returns a list of about 25% of the pages for that domain
3. Webmaster tools says there is no data available even for the one site that seems to be indexing to some degree
4. Indexs seem to have failed in webmaster tools for the other 2 domains

Webtool work was done a month and  a half ago, so it doesnt look like somethign that will work itself out over time.

Here's my guess how to fix it and I may be entirely wrong:
1. Create an 'uber sitemap' - I don't remember what this is called, but it combines multiple sitemaps
2. Make an individual sitemap for each domain
3. List the individual sitemaps in the 'ubersitemap'
4. Go back to each domain in webtools and point each domain at the ubersitemap.

Does this sound like I'm on the right track?
Are there other issues I should be considering?
Can anyone point me to a tutorial to make the ubersitemap if that's the way to go?

Thanks again for any and all ideas!

I've since learned that what I was describing as a 'ubersitemap' is called a Sitemap Index File. So let's start there...

If a company has a single set of pages, with 3 domains pointed at these same pages, would a Sitemap Index File be a good way to allow webtools to recognize it doesn't need to crawl all 3 domains? ...ie pages crawled for one are the same for the other two?

If this is the right approach, would we simply submit the same Indexfile for each domain in webtools?

Thanks again!

Hello,

in case if that is exactly the same site, it is recommended to create one sitemap for the main domain and setup 301 redirect from other domains to the main domain to avoid duplciate content issue.

Thanks Oleg! The pages for all domains are exactly the same (same folder) so maybe I've been missing the point.

To review:
1. Best I can tell they have only indexed the non-www version of the site. I thought they should have indexed the [ External links are visible to forum administrators only ] as well, so the confusion is probably on my side.

2. One of the domains is the old URL that is still in use but being phased out. So if I am understanding you correctly, we shouldn’t expect to see anything for 'site:oldwebsite.com' because we aren’t supposed to be indexing anything other than the new (primary) domain?

3. As far as 301redirects, our SEO guru said that he didn’t need redirects because he 'pointed' all of the domains to the same folder using the hosting tools. But others have told us we should use 301 redirects to clear up Google analytics reporting. Now I'm wondering if we need the 301s for indexing as well.

Assuming we do need 301's, can you point me to a tutorial? It sounds like I will get 'on the job training' here:)

Much appreciated, and I'm relieved to hear that things may not be messed up at all.
ZS

Hello,

in case if your main domain is non-www version, you should create file named ".htaccess" in your website folder with:

RewriteEngine on
RewriteCond %{HTTP_HOST} !^domain\.com
RewriteRule (.*) http://domain.com/$1 [R=301,L]

Thanks Oleg!
I'm still wondering why we don't index the 'www' version and wonder if there is any good reason not to make the change? Wouldn't 'www' be stronger under most circumstances?

Also, I've had an entirely differnt approcah recommended as follows- Can you help me understand why this isnt a good approcah?

Webmasters at Google Groups are reporting that some verified sites' sitemap URLs are not being accepted by Google Webmaster Tools. The error some webmasters are receiving is: "URL not allowed - This url is not allowed for a Sitemap at this location."

Susan Moskwa of the Google Webmaster Central team says that this is a problem that exists for those who have numerous domains in their accounts. The alternative way is to specify the Sitemap through robots.txt. She writes:

As you've noticed, our cross-submissions system may experience some problems if you have a large number of sites in your account. For now, I'd recommend submitting your Sitemaps via your robots.txt file, or submitting separate Sitemaps for each subdomain.
The robots.txt sitemap protocol is explained in this Google Webmaster Help document. Simply add the following line to your robots.txt file:

Sitemap: [path to sitemap]


Thanks again, I really appreciate your input!
Michael

Hello,

you should select either one - www domain or non-www domain (doesn't really matter which one). Then you should redirect all URLs to this main domain and make sure that all URLs in sitemap are pointing to it and submit it to google to corresponding domain entry as well.

I think I understand! Much appreciated!
ZS