xs_crawl_ident not working
« on: June 19, 2007, 11:20:57 AM »
Our web is made in flash, but when Googlebot (and others) index our website we show them the same content but in html. To make a correct sitemap for google and others I need the Sitemap Generator to visit the site as user-agent Googlebot. It should work with 'xs_crawl_ident' , but it isn't functioning for me (either with 'xs_usecurl' on or of).

Please HELP!

Geert
Re: xs_crawl_ident not working
« Reply #1 on: June 19, 2007, 10:20:19 PM »
Hello,

how does your script determine googlebot? Sitemap Generator appends the following to the string defined in 'xs_crawl_ident' option:
Code: [Select]
(powered by www.xml-sitemaps.com)
Re: xs_crawl_ident not working
« Reply #2 on: June 19, 2007, 10:30:49 PM »
Just resolved it myself... first I only put googlebot, but now I have put Googlebot/2.X (+http://www.googlebot.com/bot.html) and it works...

BTW does it append that string also with an original license??

Geert