Sitemap Generator Forum

Hi,

I am having real trouble with getting the site map generator to actualy generate a site map.

I am testing this for a rollout to a number of websites that we host. I am using iis6 with php installed from the web platform installer (PHP 5.3.5, fastCGI for iis6 and the windows cache extentiopn for php 5.3).

It is running on a dev site, that auto generates the robots.txt that excludes it from beig crawled ( because it is a development environment) but i have added an exception for the site crawler user agesnt, an I have forced the user agent name by adding the line <option name="xs_crawl_ident"> to the generator.conf file and set the user agent to that of my robots.txt

Also in the generator.conf file I have set <option name="xs_robotstxt"> to 0 to try to force it not to check the robots.txt

However I try to get my the generator to crawl, I just get back a blank xml file.

Any help would be very apreciated.

Regards

Hello,

as the first step I would try to temporarily unblock your dev instance to all user agents and remove robots.txt just to check if crawling is working correctly without those limitations.

I am testing this for a rollout to a number of websites that we host

Please also note that license agreement allows to use it only for website that you own and can't be used for 3rd party sites: https://www.xml-sitemaps.com/license.html