Sitemap Generator Forum

I have the script installed on two sites. The first, MYSITE.com crawls properly, writes properly, pings properly, and Google likes the map (kudos!).

The second, a subdomain, MYSITE2.com has a separate install. I can reach it properly from MYSITE2.COM/THEDIRECTORY, It writes properly (to .../ MYSITE/MYSITE2/MAP) but refuses to crawl beyond the second page, even though it lists the 4 pages it finds, which in turn branch to further pages. This site is _tiny_, only 46 pages, and only to a depth of 3 (index - mainpages (3 of them) - in turn to the rest). The config. looks identical to the main site (except directories, of course). Both are on a dedicated server, no complaints about resources, even on the large MYSITE, and I have tried removing all limitations to crawling depth, memory use, etc.

Any thoughts?

Please check with our SE bot simulator tool that your subdomain site is crawlable: https://www.xml-sitemaps.com/se-bot-simulator.html

Oleg:

Under the ahha! category - mixed http:// www. mysite and http:mysite on the index page. Saw the error by looking at the simulator, and now the script runs fine.

Thanks!

Great! Glad that helped

Same problem - crawls just 4 pages

simulator result
[ External links are visible to forum administrators only ]
There was an error while retrieving the URL specified

I have checked robots.txt file, the pages have <meta name="robots" content="index, follow">
 in each - the cart is os commerce set up by crecommerce.com - php.ini allows 98megs memory

pulling my hair out - It was working ok until the hoster siteground started mucking around with it (the shop) and put it a different directory

Same thing happens in pub version of xml sitemaps

thanks for any ideas
peter

Hello,

I cannot open that URL, looks like your server is currently down.

[ External links are visible to forum administrators only ] is working ok right now

Peter