Too long and too many urls!
« on: December 20, 2008, 12:40:36 PM »
Hi

I have been looking for some time for an xml sitemap solution for my joomla site with about 10000 products.

Installation was easy and set it working.  These are the current figures from my site crawl and wonder how it can be creating so many urls - I only have 10000 products and very little other content at the moment.  It also looks as though it's going to take about a week to complete. Could somebody help/advise - I have got something wrong in the set up?

Links depth: 4
Current page: draper/133-air-tools-spares/52905-spring.html
Pages added to sitemap: 12399
Pages scanned: 12680 (379,310.4 KB)
Pages left: 19025 (+ 10229 queued for the next depth level)
Time passed: 921:05
Time left: 1381:59
Memory usage: 45,726.7 Kb

In addition, it is not possible to see images on my site whilst the crawling is taking place.

Thanks
Sharon
Re: Too long and too many urls!
« Reply #1 on: December 21, 2008, 08:07:23 PM »
Hello,

make sure that you applied sitemap exclusion rules for Joomla:
Add this in "Exclude URLs" option:
Code: [Select]
style=
style,
print=
do_pdf=
pop=1
task=emailform
task=trackback
task=rss
recommend
checklinkstatus
sorting
task=clk
action,search
rand,
=rss
the restart generator from the scratch (do not resume existing session)