Sitemap Generator Forum

Hi !

I would love to have a 'Parse but do not include' option in the config panel - i have tons of non relevant pages related to product pagination and i don't want them to be indexed but i do want the products within those pages to be included in the sitemap.

Basically it would do a kind of 'noindex,follow'.

How hard is this to do ? If i donate a bit, can it be done asap ? 

Best,
Remi

I am new to the script, forgive me if I am wrong.

Isn't what you ask for already there? I think it is the "Exclude URLs:" option.

URL's you list in that space are not included in the site map, however they are parsed and their links are included in the site map. (at least that is my understanding of it)

Hello,

you can actually add <meta robots> with "noindex,follow" on those pages, isn't that ok for you?


wayne3,

"Exclude URLs" option also tells generator NOT to parse those pages, while Remi wants to parse them (i.e. find all links located on those pages), but do not include them in sitemap.

Ah, thanks for clearing it up!  As I said I am new to the script, but am liking it very much!

Since I was mistaken about the "exclude URL's" feature, I would also like to see a "parse but do not include" option.

It would be impossible for me to use the "no index,no follow" meta tags in the dynamically generated pages on my web site since the criteria I want to use to "parse but do not include" is based on the URL sub string and not the templates which set the meta tags. In other words I want to include some pages that use that template, but not others that use the same template so the URL sub string is the only way to do it.

So here's my vote for a "parase but do not include" option!

hi admin,

The problem is that i would have 'noindex' urls indexed on the sitemap but they would blocked on the site...
basicaly, without this option i can't sync entries from the sitemap with followed pages on the site.

Can you do something ? (How much time would that take to implement?)

Many thanks

Remi

We will check this as a possible addition to the next version of Sitemap Generator that should be released soon (no ETA available yet).