Crawler creates too few pages
« on: November 01, 2012, 02:44:18 PM »
Hi, After the crawler is ready with the sitemap.xml it has not added all pages to the sitemap, why is that? It is only adding 124 pages but when I use your free online sitemap generator it generates all pages... That means the free version is better... ???

Please I need help solving this.
-------------------------------------------------

Game Fishing in Gagnef Sweden
Re: Crawler creates too few pages
« Reply #1 on: November 01, 2012, 06:42:52 PM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?
 
Re: Crawler creates too few pages
« Reply #2 on: November 02, 2012, 08:09:42 AM »
I think it's solved the only changes i made was entering a /index.php after the site url in the settings Starting url like this: [ External links are visible to forum administrators only ] and that seem to have done the trick.. On another site i have left the url without the index.ph and it still works so it's kinda strange to me I had to change it for this site.
Maybe you could explain it better?
-------------------------------------------------

Game Fishing in Gagnef Sweden
Re: Crawler creates too few pages
« Reply #3 on: November 05, 2012, 03:47:12 PM »
What is the website URL that did not work without adding "index.php"?
Re: Crawler creates too few pages
« Reply #4 on: November 05, 2012, 07:23:27 PM »
This is the url: [ External links are visible to forum administrators only ]
-------------------------------------------------

Game Fishing in Gagnef Sweden
Re: Crawler creates too few pages
« Reply #5 on: November 07, 2012, 08:41:13 AM »
Perhaps you have a redirect depending on the language setting so it didn't work.