Crawl not completing on one site but no error message
« on: May 31, 2022, 12:31:02 PM »
I have sitemaps setup for 3 sites on my server that have been run as cron jobs for many years.
I notice one of the sites has not been completing for a couple of months.
I've upgaded to the latest version but no joy. There is no error message as such but it doesn't finish.
Server host can't find anything in the logs either.

Thanks,

Steve
Re: Crawl not completing on one site but no error message
« Reply #2 on: May 31, 2022, 06:21:06 PM »
Thanks Oleg - I have sent you the details
Re: Crawl not completing on one site but no error message
« Reply #3 on: June 01, 2022, 04:51:40 PM »
Ok I think I have worked out what's causing this:

I uploaded a brand new copy, created new xml files and set the correct permissions.

The crawl runs ok UNLESS 'Create a separate sitemap with embedded videos information' is selected (see screenshot).

I have a sitemap_video.xml but it won't allow the crawl to complete so neither sitemap gets written.

Steve
Re: Crawl not completing on one site but no error message
« Reply #4 on: June 16, 2022, 09:04:47 PM »
Note that only videos from these sources can be automatically detected by sitemap generator bot:
YouTube, Google Video, Vimeo, Dailymotion, MTV
(since they provide API that allow to get video details like play duration/thumbnail etc)

In other case, VideoObject Schema markup is supported that you can use for any videos. 
[ External links are visible to logged in users only ]