I figured this issue warranted its on thread.
I wanted to create an exception so that the generator ignores/does not index a full level on the site.
site.net/category1/something/somthing
I want to create the exception so that everything under and including the folder category1 is ignored.
furthermore, i would like to ignore all links that have the "nofollow" attribute on them.
How to do this?
Guest