Automatic Refresh On Stalled Crawl Does Not Work.
« on: July 28, 2012, 12:43:37 PM »
Comment:

The instruction to automatically refresh the crawl - on stalled crawl simply does not work.

Having tested it very which to date - it simply does not refresh. It also has nothing to do with the server performance criteria either - since the automatic on line crawl has no such issues on the same crawl.

This aspect needs fixing on the Unlimited script!


Re: Automatic Refresh On Stalled Crawl Does Not Work.
« Reply #1 on: July 31, 2012, 07:12:31 PM »
Ah I have found a way of making the long crawls work:

1 - Interrupt the current crawl by clicking the "configuration" tab
2 - Save the same configuration details
3 - Go back to Crawling and Resume the last Crawl

All this has to be done manually though - which takes the automated aspect out of what is supposed to be an "automated" process.

Clearly the settings should do this job without the need for manual intervention - but they dont.

Yet the On Line system does its job in minutes for the same sites.

Waiting two days for a 10k pages crawl may or may not be worth it!

It is very time consuming and disappointing given how fast the On Line system works.

Can I request that until this is fixed and working as effectively as the On Line System that as a paid User access is made available to the On Line crawler with extended page crawl limits?
Re: Automatic Refresh On Stalled Crawl Does Not Work.
« Reply #3 on: August 07, 2012, 08:09:12 PM »
That is a kick in the teeth given that I have already paid for the Standalone product which is not yet efficiently operating and is very buggy.

This does not work anything like as efficiently as your on line product.

That you now want me to pay for your remakably better operating on line product smacks of taking the micky.

Not happy at all with this "Offer" at all.

Please help me fix the product I have bought?

Script NOT Running in The Background as indicated
« Reply #4 on: August 07, 2012, 08:19:25 PM »
Have set the script to run in the "background" even if the browser is closed.

Although that message appears - the crawl has actually stopped when the browser window is closed.

I therefore have to refresh the window to get the script re-started - at which point it has gone back to the start of the crawl again!

Perhaps worse - is that if the Standalone Script worked as fast as the on line script [which you charge more for] - this would not even be an issue.

Also when  it does work the Standalone Script takes about two days to crawl 10000 pages. The relative performance of the On line Sitemap Generator and the Standalone Script is vastly different.

Using exactly the same source web sites - so it cant be anything to do with the speed of the web server.

Can you make my Standalone Script work as efficiently as the On Line Sitemap Generator or provide us with access to the unlimited On Line Generator WITHOUT paying anything for the priviledge.

Had I known about how slow the Standalone would perform I would not have purchased it.

This is not a pleasant experience and has wasted a great deal of our time.
Re: Automatic Refresh On Stalled Crawl Does Not Work.
« Reply #5 on: August 08, 2012, 08:43:00 AM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Re: Automatic Refresh On Stalled Crawl Does Not Work.
« Reply #6 on: August 08, 2012, 12:35:20 PM »
now trying this