Competitor Analysis Tool: Analyze rival websites and gain valuable SEO and PPC insights
« previous next »
Pages: 1

mroyusa

*
  • *
  • 5
how to not crawl dynamic pages?
« on: January 28, 2011, 07:28:33 AM »
How do you not crawl dynamic pages created by website?
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: how to not crawl dynamic pages?
« Reply #1 on: January 28, 2011, 10:47:16 AM »
Hello,

you should just enter your homepage link as Starting URL and further dynamic pages will be found by generator crawler as long as they linked somewhere on your site.
Logged

mroyusa

*
  • *
  • 5
Re: how to not crawl dynamic pages?
« Reply #2 on: January 28, 2011, 11:05:20 AM »
My question was to NOT crawl dynamic pages.
[ External links are visible to forum administrators only ]]=1
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: how to not crawl dynamic pages?
« Reply #3 on: January 28, 2011, 11:18:56 AM »
Sorry, I misunderstood the request. You should add a question mark "?" in "Exclude URLs" setting, and all links with it will be excluded.
Logged

mroyusa

*
  • *
  • 5
Re: how to not crawl dynamic pages?
« Reply #4 on: January 28, 2011, 11:20:43 AM »
Thanks worked perfectly, do I have to buy the standalone version for each site i own?
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: how to not crawl dynamic pages?
« Reply #5 on: January 28, 2011, 11:09:09 PM »
You can use the same license for more than one domain as long as you are the owner of those websites: https://www.xml-sitemaps.com/license.html
Logged
Pages: 1
« previous next »