Sitemap Generator Forum

Hello,

I have uploaded all the files and folders as documented and started the indexing.  It is working well, but when I stopped the script to check if there is something coming in sitemap.xml file, I found the sitemap.xml file empty.

Also I found there is not sitemap.html file in data folder.

Would you please let me know why it is doing so and how can I avoid this as soon as possible?

Thanks.

Hello,

sitemap files are created only after the whole crawling process is finished.

Hello,

sitemap files are created only after the whole crawling process is finished.

I have a large number of files in my website to index, I therefore tried to index only 5 files to check if it works good or not.  Please click on the following link to see the snapshot of the error I am getting.

[ External links are visible to forum administrators only ]

Thanks.

Hello,

you have defined paths for the sitemaps incorrectly. Make sure that files you specified are existing on your server.

Hello,

you have defined paths for the sitemaps incorrectly. Make sure that files you specified are existing on your server.

Would you please check the path and let me know what is wrong with the paths?

[ External links are visible to forum administrators only ]

I have public_html folder on root and sitemap.xml file and generator folder in public_html folder.

Thanks.

Hello,

you should not change any path settings unless you are sure it is required. You should reupload fresh config file from Sitemap Generator package and open your config screen - the paths should be filled automatically.

After a fresh sitemap upload, I checked it for 6-7 pages and it worked and created the sitemap.  Then I started crawling the whole site.  Although the site is large, but should it take more than 48 hours to complete?  The crawling process is going on and on and is not completed.

Thanks.

Hello,

yes, the crawling process may take some time, depending on the number of pages.

Its been 4-5 days and crawling is still going on day and night.  Should it take that much time?  Still around 3000 pages are done and 4000 pages are left?  Moreover, when I checked the status today, its showing the resume date of 10th April, however I did resume it today after stopping it.

What should I do?

Thanks.

Hello,

if you have SSH access to your host, you should try to execute it via command line for better performance.

I am on windows 2003 server.  It is still going on today.  I stopped it and resumed it and it says "Resuming the last session (last updated: 2007-04-10 08:31:54)".  Why it is giving me the old date?

Thanks.

Hello,

possibly the script cannot save the sessions state, make sure that generator/data/crawl_dump.log file is writable (has corresponding file permissions).

I can see lots of data in generator/data/crawl_dump.log  file it means the file is writable.  Moreover, I have already told you that when I created the sitemap with 6 pages, it worked and created the sitemap successfully.  But now when I ran it for the whole site, its taking days and days to complete.

Please resolve this issue as soon as possible, its been many days and the work is on pending.

Thanks.

Hello,

please let me know your generator URL in private message.

After setting the variables in php.ini file, I resumed the crawl.  Should it work now?

Thanks.