Standalone Sitemap Generator and htacesss Rewrite Rules
« on: August 11, 2005, 07:27:55 AM »
Is there anyway to get the standalone sitemap generator to work with RewriteRules within an htaccess file.

I have a site set up that uses the Rewrite Rule to rewrite meaningful urls for improved search engine friendliness into the true urls. So for example

[ External links are visible to forum administrators only ]
will get rewritten as [ External links are visible to forum administrators only ]
but visitors and search engines see the first (more friendly URL)

When the sitemap generator comes across links within these pages (which are links to individual products) it seems to stumble. It doesn't seem to find the friendly URL which should look something like:
[ External links are visible to forum administrators only ]
and get rewritten to
[ External links are visible to forum administrators only ]

Instead it seems to go on 'burrowing' down into 'made up' sub-directories and I end up with a sitemap that contains urls such as:

[ External links are visible to forum administrators only ]
and on it goes adding yet more 'Books' sub-directories and it never seems to stop.

I must admit to not knowing too much about Rewrite Rules so if there is anyone here who could help me (maybe I need to  add something else to the htaccess file?) I'd very much appreciate it. If the sitemap generator doesn't work with Rewrite Rules then could you let me know also, before I waste too much time trying to sort this out.

Thanks.
Al.

« Last Edit: August 11, 2005, 07:29:32 AM by info4 »
Re: Standalone Sitemap Generator and htacesss Rewrite Rules
« Reply #1 on: August 11, 2005, 10:04:28 AM »
Hi Al,

there is no problem with crawling a site with mod_rewrite rules with sitemap generator. It acts exactly as SE bot does and just retrieves the URLs found on your pages.
I believe the problem is that you have the following tag:
<base href='[ External links are visible to logged in users only ]

Generator supports the <base> tag, but the attributes should be enclosed in double quotes ("). For the moment you can replace this line with:
<base href="[ External links are visible to logged in users only ]">

The next version of generator will work with both quotes types.
Re: Standalone Sitemap Generator and htacesss Rewrite Rules
« Reply #2 on: August 11, 2005, 02:44:15 PM »
Brilliant answer, that seems to have fixed the problem, Well spotted and thanks!!

Al.