I want to crawl links with Querystring parameters. When I run the free generator, it does crawl the parameter I want, but with only three individual values. I don't know how the generator found/picked those three. I also don't know how I can manage this in the future as new values need to be added/removed. There is one main parameter I would like to crawl and I need to crawl it for every value on the site.
Any suggestions?
Thank you.