Limited crawling
« on: December 11, 2025, 06:23:49 PM »
Hello,

I've run the script several times, but it always stops at the same number of links: 667, even though I've set it to 100,000.

More than three-quarters of the pages/links aren't being crawled.

What could be causing this?

How can I find out?
Re: Limited crawling
« Reply #1 on: December 11, 2025, 07:00:19 PM »
Hello,

could you please provide an example URL that is not included in a sitemap and how it can be reached starting from homepage?
 
Re: Limited crawling
« Reply #2 on: December 12, 2025, 06:27:39 PM »
Hi,

problem solved, thanks.

Regards Uli
« Last Edit: December 12, 2025, 06:29:34 PM by ulilaechelt »