On Page SEO Checker: Get a list of improvements for your website for free in less than 5 minutes!
« previous next »
Pages: 1

kajlan

*
  • *
  • 3
crawling problmes
« on: November 17, 2009, 10:07:12 PM »
Hi

my sitemape only index the main page

why

everything is correct i double check it


i got no errors also

[ External links are visible to forum administrators only ]


Logged

kajlan

*
  • *
  • 3
Re: crawling problmes
« Reply #1 on: November 18, 2009, 02:11:48 AM »
Solved


problem was from


No-follow

is there a way to leave no-follow and still be able to crawle the hole site


it,s a webdirectory 
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: crawling problmes
« Reply #2 on: November 18, 2009, 10:15:29 PM »
Hello,

basically, the "nofollow" attribute is designed especially to exclude pages from indexing, so it's always taken into consideration by sitemap generator.
Logged

kajlan

*
  • *
  • 3
Re: crawling problmes
« Reply #3 on: November 19, 2009, 01:19:17 AM »
the no-follow is tag in the header

<META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW">

it,s saying index the page but don,t follow external links

follow internal page


but in the script it,s  indexing only the main page

is there an answer for this >>>?

Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: crawling problmes
« Reply #4 on: November 19, 2009, 09:18:43 PM »
Hello,

nofollow meta tag tells search engines not to follow ANY links on that page, not only external links: [ External links are visible to logged in users only ]
Logged
Pages: 1
« previous next »