On Page SEO Checker: Get a list of improvements for your website for free in less than 5 minutes!
« previous next »
Pages: 1

pinkstarfruit

*
  • *
  • 40
Google have 11 000 URL restricted by robots
« on: May 29, 2011, 08:27:56 AM »
I have 11 000 url restricted by robots is that normal?


Google webmaster is blocking this kind of threads


/tags.php?tag=investment+in+thailand URL restricted by robots.txt

/threads/2207-Obama-Visit-To-Indonesia?p=2389 URL restricted by robots.txt
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: Google have 11 000 URL restricted by robots
« Reply #1 on: May 29, 2011, 09:06:40 AM »
Hello,

if you don't want those pages to be indexed by search engines, you can keep the exclusion in robots.txt.
Logged

pinkstarfruit

*
  • *
  • 40
Re: Google have 11 000 URL restricted by robots
« Reply #2 on: May 29, 2011, 12:01:47 PM »
Yeah i know but should i?
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: Google have 11 000 URL restricted by robots
« Reply #3 on: May 30, 2011, 06:51:20 AM »
Perhaps you would want to index tags.php pages, as for the second example it looks like a duplicate URL for the topic.
Logged
Pages: 1
« previous next »