Hello,

I have a phpbb forum with 10K+ topics that I would like to index (urls are static format not dynamic)

I have created a script that lists links for those topics at 1000 per page

Assuming that url format for the script is [ External links are visible to forum administrators only ]
and at the bottom of the first listall.php page is a link to the next liastall.php page (such as [ External links are visible to forum administrators only ]) to get to the next 1000 topics links, and so on

My question is: is it possible to configure the generator to only traverse and index links shown on this script pages (i.e. visit and index: listall.php, listall.php?page=2, listall.php?page=3, etc)?
If yes, what configuration to set for that?

Thanks for any tips
Hello,

assuming that you have "viewtopic.php?t=X" links on those pages, you can add "viewtopic.php" into "Do not parse" option. In this case the URLs will still be included in sitemap, but generator won't crawl them.