Crawling Error
« on: June 25, 2011, 12:47:54 AM »
Hello

I keep getting the error
error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 7091487 bytes) in /home/d4info/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 27

I have read the forums and I have already increased my

max_execution_time
and
memory_limit

You can see my PHP settings
[ External links are visible to forum administrators only ]

It seems like the script just keeps eating more and more Memory till it errors out...

Please help !!!
Re: Crawling Error
« Reply #1 on: June 26, 2011, 03:25:48 PM »
Hello,

looks like you have different PHP settings for main domain folder, please try to copy php.ini  file from domain root to generator folder.
Re: Crawling Error
« Reply #2 on: June 28, 2011, 01:57:10 PM »
Hi Oleg, Thanks that seemed to help for a while until the new max  memory was hit now i get this line

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 53752815 bytes) in /home/d4info/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 27

You can see my php settings here:  [ External links are visible to forum administrators only ]

Please let me know if there is any way to stop the script from eating so much memory.

Thanks
Re: Crawling Error
« Reply #3 on: June 28, 2011, 06:23:11 PM »
Hello,

looks like you have lots of nearly duplicate/empty pages on your site, current generator stats are:
Pages scanned: 71900 (1,980,947.6 KB)
Pages left: 208754 (+ 11621 )

I've updated your crawler settings, please check now.
Re: Crawling Error
« Reply #4 on: June 28, 2011, 06:31:58 PM »
Thanks...should I continue to crawl or create a new crawl ?

Thanks for all your help :)