How can i stop the sitemap generator from crawling any further
« on: February 11, 2010, 03:49:21 PM »
I am currently in the process of changing my price comparison site from .htm to .php so the search engines can pick up or see all the content. Just realised that the Sitemap Generator might be trying to index everything and if it is it will probably still be trying to do that in 6 months time given the size of the database. I would therefore like to stop it and specify URLs that should scanned say to a depth of 2/3 which I presume can be done. The first problem is that I cant log in, presumably because scanning in is progress and i was wondering is there is any file on the server that I could access to stop the thing.
Re: How can i stop the sitemap generator from crawling any further
« Reply #2 on: February 12, 2010, 01:04:33 AM »
Hi Oleg,

Thank you for getting back to me. I'm fairly certain the log-in details I am using are correct but as I changed them earlier in the day and only logged in once since perhaps there was a glitch. Anyway to check I followed the link you provided but unfortunately there is no  generator/config.inc.php. There are two php files generator/index.php and generator/runcrawl.php but neither have the log-in info. There are also two folders, a data folder and a pages folder. The latter contained a folder page-config.inc.php but that didn't have the info either.

The version I have is "Standalone Sitemap Generator (PHP) v3.0, 2009-05-19" and I was wondering if there is some other folder I should be looking at.

Ian
Re: How can i stop the sitemap generator from crawling any further
« Reply #4 on: February 15, 2010, 12:25:26 PM »
Hi Oleg, thank you for that. Login and password had disapeared. I added them and it opened up fine.

Ian