Toggle navigation
PRO Sitemaps
Standalone Generator
About Sitemaps
News
SEO Tools
Sitemap Generator Forum
Guest
Login
Register
Menu
Home
Help
Search
Login
Register
XML Sitemaps Discussions Category
/
Site maps - General Discussion
/
Google have 11 000 URL restricted by robots
Competitor Analysis Tool:
Analyze rival websites and gain valuable SEO and PPC insights
« previous
next »
Pages:
1
pinkstarfruit
40
Google have 11 000 URL restricted by robots
«
on:
May 29, 2011, 08:27:56 AM »
I have 11 000 url restricted by robots is that normal?
Google webmaster is blocking this kind of threads
/tags.php?tag=investment+in+thailand URL restricted by robots.txt
/threads/2207-Obama-Visit-To-Indonesia?p=2389 URL restricted by robots.txt
Logged
XML-Sitemaps Support
11781
Re: Google have 11 000 URL restricted by robots
«
Reply #1 on:
May 29, 2011, 09:06:40 AM »
Hello,
if you don't want those pages to be indexed by search engines, you can keep the exclusion in robots.txt.
Logged
Oleg Ignatiuk
https://www.xml-sitemaps.com
Send me a Private Message
SEM and SEO Reports, more than 45M domains
: The world's leading Competitive Intelligence Tool for digital marketing.
pinkstarfruit
40
Re: Google have 11 000 URL restricted by robots
«
Reply #2 on:
May 29, 2011, 12:01:47 PM »
Yeah i know but should i?
Logged
XML-Sitemaps Support
11781
Re: Google have 11 000 URL restricted by robots
«
Reply #3 on:
May 30, 2011, 06:51:20 AM »
Perhaps you would want to index tags.php pages, as for the second example it looks like a duplicate URL for the topic.
Logged
Oleg Ignatiuk
https://www.xml-sitemaps.com
Send me a Private Message
SEM and SEO Reports, more than 45M domains
: The world's leading Competitive Intelligence Tool for digital marketing.
Pages:
1
« previous
next »