I've installed the sitemap generator on my domain [ External links are visible to forum administrators only ], however when crawling the data, i got 3000+ pages to crawl and somehow the pages crawled seems redundant. How do i removed the redundancy in sitemap and how do i configure the sitemap generator to only crawl the important pages and ignore the unnecessary pages?
The weird redundant link i saw in my sitemap looks like this...
<url>
<loc>[ External links are visible to forum administrators only ]</loc>
Thanks in advance.
Guest