Processing time: 40:21:58s ... too long
« on: April 13, 2014, 06:41:43 PM »
this is my Standalone Sitemap Generator Report

Quote
Hello,
sitemap for your website has been updated.
[ External links are visible to forum administrators only ]
Sitemap details
-------------------------
Sitemap created on: 13 April 2014, 16:32
Processing time: 40:21:58s
Pages indexed: 23286
Crawled pages size: 2,521.59Mb

perhaps too committed to this server, and it might make it slower?

Quote
XML Sitemap
[ External links are visible to forum administrators only ]
Text Sitemap
[ External links are visible to forum administrators only ]
ROR Sitemap
[ External links are visible to forum administrators only ]
HTML Sitemap
[ External links are visible to forum administrators only ]
Images sitemap (20454 images)
[ External links are visible to forum administrators only ]
Broken Links
-------------------------
1000 broken links found!

DISABLE 
Create sitemap in Text format
Create sitemap in ROR format

only ENABLED
Create HTML Sitemap
Create Images Sitemap

because it continues to create those disabled and what makes the task so demanding (duration) ?

willing to leave password (via PM) to access the control panel

Thank for support !!!



Re: Processing time: 40:21:58s ... too long
« Reply #1 on: April 14, 2014, 05:52:34 AM »
Hello,

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.

Sitemap files are created only after the crawling is completed, so disabling some sitemap types will not affect crawling time.
Re: Processing time: 40:21:58s ... too long
« Reply #2 on: April 14, 2014, 09:56:45 AM »
I apologize but I did not understand.

From control panel I have disabled:
text Sitemap
ROR Sitemap

but reports continue to appear in the link: why?
how can I disable these additional sitemap to make it work faster?

thanks
Re: Processing time: 40:21:58s ... too long
« Reply #3 on: April 14, 2014, 02:56:16 PM »
Hello,

sitemaps are not created in this case, but links in reports still remain.
Re: Processing time: 40:21:58s ... too long
« Reply #4 on: April 14, 2014, 03:06:56 PM »
ok I got it!

then there is the problem that an operation of sitemaps, can not last 40 hours: I'm wrong?

Sitemap created on: April 13, 2014, 16:32
Processing time: 40:21:58 s
Indexed Pages: 23286
Crawled pages Size: 2,521.59 Mb

How can I reduce this activity to be lighter on the server?
or do you think it is not a problem?
Re: Processing time: 40:21:58s ... too long
« Reply #5 on: April 16, 2014, 06:16:00 AM »
Hello,

as mentioned above, the crawling time depend on how fast the pages are opened on your website.