Sitemap Generator Forum

I've just launched an opencart site and have used the standalone generator to produce a sitemap for the site.  The generator is showing 8328 pages on the site, when I only have 240 products and maybe 20 other pages in total.  When I look at the site map it is showing hundreds of search pages which can't be right surely.  Can anyone tell me how to stop it doing this?
Plus I've just realised that even though the sitemap.xml file is being written to by the generator, as I can see the content when I view it with notepad+, it can't be accessed by the search engines and I get a 404 when trying to access it myself. 
You can view the problem at [ External links are visible to forum administrators only ].

Hello,

do you have .htaccess file in your site root folder? If yes, what is its content?

Yes I do.  Here are the contents:

# 1.To use URL Alias you need to be running apache with mod_rewrite enabled.

# 2. In your opencart directory rename htaccess.txt to .htaccess.

# For any support issues please visit: [ External links are visible to forum administrators only ]

Options +FollowSymlinks

# Prevent Directoy listing
Options -Indexes

# Prevent Direct Access to files
<FilesMatch "\.(tpl|ini|log)">
 Order deny,allow
 Deny from all
</FilesMatch>

# SEO URL Settings
RewriteEngine On
# If your opencart installation does not run on the main web folder make sure you folder it does run in ie. / becomes /shop/

RewriteBase /
RewriteRule sitemap.xml /index.php?route=feed/google_sitemap
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^?]*) index.php?_route_=$1 [L,QSA]

### Additional Settings that may need to be enabled for some servers
### Uncomment the commands by removing the # sign in front of it.
### If you get an "Internal Server Error 500" after enabling any of the following settings, restore the # as this means your host doesn't allow that.

# 1. If your cart only allows you to add one item at a time, it is possible register_globals is on. This may work to disable it:
# php_flag register_globals off

# 2. If your cart has magic quotes enabled, This may work to disable it:
#php_flag magic_quotes_gpc Off

# 3. Set max upload file size. Most hosts will limit this and not allow it to be overridden but you can try
# php_value upload_max_filesize 999M

# 4. set max post size. uncomment this line if you have a lot of product options or are getting errors where forms are not saving all fields
# php_value post_max_size 999M

# 5. set max time script can take. uncomment this line if you have a lot of product options or are getting errors where forms are not saving all fields
# php_value max_execution_time 200

# 6. set max time for input to be recieved. Uncomment this line if you have a lot of product options or are getting errors where forms are not saving all fields
# php_value max_input_time 200

You would need o remove this line:
RewriteRule sitemap.xml /index.php?route=feed/google_sitemap

Hi, sorry for the delay in replying but I got distracted with other stuff.  I changed the .htaccess as instruted but it didn't seem to make much difference.  The crawl went on for hours and to date when I interrupted it it reckoned that it had found 21743 pages, 21729 of which it has added to the sitemap.  That can't be right surely?  Would it be because I only commented out the line instead of removing it all together?

Hello,

I've updated your crawler settings, please check now.

That worked perfectly.  Thank you.

Thank you ...