Toggle navigation
PRO Sitemaps
Standalone Generator
About Sitemaps
News
SEO Tools
Sitemap Generator Forum
Guest
Login
Register
Menu
Home
Help
Search
Login
Register
XML Sitemaps Discussions Category
/
Unlimited PHP Sitemap Generator
/
crawling problmes
Competitor Analysis Tool:
Analyze rival websites and gain valuable SEO and PPC insights
« previous
next »
Pages:
1
kajlan
3
crawling problmes
«
on:
November 17, 2009, 10:07:12 PM »
Hi
my sitemape only index the main page
why
everything is correct i double check it
i got no errors also
[ External links are visible to forum administrators only ]
Logged
kajlan
3
Re: crawling problmes
«
Reply #1 on:
November 18, 2009, 02:11:48 AM »
Solved
problem was from
No-follow
is there a way to leave no-follow and still be able to crawle the hole site
it,s a webdirectory
Logged
XML-Sitemaps Support
11781
Re: crawling problmes
«
Reply #2 on:
November 18, 2009, 10:15:29 PM »
Hello,
basically, the "nofollow" attribute is designed especially to exclude pages from indexing, so it's always taken into consideration by sitemap generator.
Logged
Oleg Ignatiuk
https://www.xml-sitemaps.com
Send me a Private Message
SEM and SEO Reports, more than 45M domains
: The world's leading Competitive Intelligence Tool for digital marketing.
kajlan
3
Re: crawling problmes
«
Reply #3 on:
November 19, 2009, 01:19:17 AM »
the no-follow is tag in the header
<META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW">
it,s saying index the page but don,t follow external links
follow internal page
but in the script it,s indexing only the main page
is there an answer for this >>>?
Logged
XML-Sitemaps Support
11781
Re: crawling problmes
«
Reply #4 on:
November 19, 2009, 09:18:43 PM »
Hello,
nofollow meta tag tells search engines not to follow ANY links on that page, not only external links: [ External links are visible to logged in users only ]
Logged
Oleg Ignatiuk
https://www.xml-sitemaps.com
Send me a Private Message
SEM and SEO Reports, more than 45M domains
: The world's leading Competitive Intelligence Tool for digital marketing.
Pages:
1
« previous
next »