I have a page that has potentially 50,000 listings on it, and I want to be able to sitemap and index them all. I had some sub-pages with a lot of links set on the home page, and it was crawling those and I would have a sitemap of about 1500 pages. So to help get a sitemap with all of the pages, I experimented with a footer link with links of links to all the pages. Since doing this, I have only gotten less than 300 pages on a crawl.
I checked the configuration and it says "maximum depth level", and it is set to 0, which says "unlimited". However, it is not doing this.
How do I get it to crawl every single link on every single page?