Toggle navigation
PRO Sitemaps
Standalone Generator
About Sitemaps
News
SEO Tools
Sitemap Generator Forum
Guest
Login
Register
Menu
Home
Help
Search
Login
Register
XML Sitemaps Discussions Category
/
Bug reporting for Standalone Sitemap Generator
/
Sitemap ignores the robot.txt
Competitor Analysis Tool:
Analyze rival websites and gain valuable SEO and PPC insights
« previous
next »
Pages:
1
coolbits
2
Sitemap ignores the robot.txt
«
on:
June 19, 2019, 07:39:13 PM »
Sitemap ignores the robot.txt
In robots.txt prohibited:
Disallow: /*&v=list
Sitemap:
/catalog/teplye-kolgotki?page=2&v=list&s=hit&d=desc
Option "Use robots.txt" enabled, but do not work.
«
Last Edit: June 19, 2019, 07:48:47 PM by coolbits
»
Logged
XML-Sitemaps Support
11782
Re: Sitemap ignores the robot.txt
«
Reply #1 on:
June 20, 2019, 09:03:35 PM »
Hello,
please let me know your generator URL/login in private message to check this.
Also, you can add this in Exclude URLs setting:
v=list
Logged
Oleg Ignatiuk
https://www.xml-sitemaps.com
Send me a Private Message
SEM and SEO Reports, more than 45M domains
: The world's leading Competitive Intelligence Tool for digital marketing.
Pages:
1
« previous
next »