Dear Forum,
Does anyone know if the generator resumes crawling on depth-level 1 after a script-abortion, or does it continue on the last depth level for example on link depth 4? Sothat the generator does not have to crawl all pages again?
I mean if I safe script status every 30 seconds, and resume via the generators continue function.
Just asking, cause it consumes a lot of time on my site to get to link depth 5, where most of the sites are located.
Thanks!