Odd Behevior with "Crawling" page.
« on: July 10, 2008, 04:17:57 PM »
Background:

I started a crawl last night.  Ran out of memory.  Altered some settingings and started again.  Still ran out of memory.

This morning I re-started after reading here, and tweaking more settings.  This time I choose to run in the background.  Two issues:

First:  For about the 1st 45 minutes the Generator seemed to "hang". By that I mean it I couldn't access any pages under the generator.  It would give me a 'loading' message in the broswer, but never loaded.  Other pages on my site were working fine. Eventually it gave me the regular 'crawling' screen that gave all the information.  I would from time to time hit 'Refresh' to see where it was.  Which leads to issue two:

Eventually, after around 7,000 pages or so, and an hour+ of crawling I ended up with a "crawling" page that ONLY has the heading "Crawling", the menu bar, the 'No Sitemaps Found' message, and the "Run in Background" option.  NO other information is on the page.  No stop button, and not even a button to start a new crawl.

Any help you can provide would be much appreciated.

The generator can be found here: [ External links are visible to forum administrators only ]
Re: Odd Behevior with "Crawling" page.
« Reply #1 on: July 12, 2008, 05:01:30 AM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.