Sitemap Generator Forum

Hello,

Based on my sitemap details at configuration at my generator;
1. Pages crawled totally 23680 pages and pages indexed 17049. Why pages indexed are less than pages crawed?
2. I adjusted all options at configuration from what I have read from this forum. Crawling pages on my website is 23680 pages and it took 10.36 hours. Is this typical period it takes on crawling?

Note: I was trying to upload a cropped picture but it said cannot uplod.

Thank you

Hello,

some pages are redirected to other URL, some are blocked with robots meta tag, some have canonical meta tag pointing to another URL - all these pages are crawled, but not added in sitemap.

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.