Hello,
I have a phpbb forum with 10K+ topics that I would like to index (urls are static format not dynamic)
I have created a script that lists links for those topics at 1000 per page
Assuming that url format for the script is [ External links are visible to forum administrators only ]
and at the bottom of the first listall.php page is a link to the next liastall.php page (such as [ External links are visible to forum administrators only ]) to get to the next 1000 topics links, and so on
My question is: is it possible to configure the generator to only traverse and index links shown on this script pages (i.e. visit and index: listall.php, listall.php?page=2, listall.php?page=3, etc)?
If yes, what configuration to set for that?
Thanks for any tips