Sitemap Generator Forum

I get the following fata error message when trying to run a crawl.  Any ideas?

Links depth: 4
Current page: 2100_Lab_%20Meter_Specs.pdf
Pages added to sitemap: 80
Pages scanned: 180 (807.7 Kb)
Pages left: 278 (+ 52 queued for the next depth level)
Time passed: 0:00
Time left: 0:00
Memory usage: 670.7 Kb

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 13527364 bytes) in /home/httpd/vhosts/astisensor.com/httpdocs/Sitemap_Generator/pages/class.http.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 59

Hello!

As discussed here:
https://www.xml-sitemaps.com/forum/index.php/topic,130.html
https://www.xml-sitemaps.com/forum/index.php/topic,122.html

You should either:
- increase the memory_limit setting in php.ini
- limit the number of pages included in sitemap at Configuration page
- exclude a part of you pages from indexing with "Do not parse URLs" option

Also disabling the "HTML sitemap generation" and "Changelog calculation" reduces the memory usage a bit.

I forgot to remove the pdf files from getting parsed.  That solved the problem.  I only have not further question.  I have an extensive graphical set of imagemaps that are used to allow users to easy and graphically navigate to find their location with the terminal link ending in an html file with the appropriate contact information.  The starting page is located at [ External links are visible to forum administrators only ] if you are interested to view it in action.  code from this starting page is posted below.  Is there any way that the href linkages contained within the image map could be indexed in the sitemap?  A sample of the code is given below.  This would be a terrific addition!!

<table cellpadding="0" cellspacing="0" align="center">
<tr>
<TD VALIGN=TOP>

<img SRC="worldaa.gif" BORDER="0" width="377" height="244" usemap="#worldaa">
<map name="worldaa">
<area shape="polygon" alt="North America " coords="159,32,117,36,48,49,11,65,58,53,50,68,38,87,45,105,48,107,27,116,34,129,67,129,73,121,85,109,94,114,129,103,128,100,83,97,81,86,101,74,114,68,115,53,129,53,147,44,156,37,157,32" href="na.htm">
<area shape="polygon" alt="South  America" coords="96,119,75,120,71,138,71,143,79,156,85,159,90,185,97,199,102,206,111,210,111,210,115,205,115,205,104,200,106,188,114,170,118,167,121,155,124,143,122,139,111,136,107,127,98,125,96,119" href="sa.htm">
<area shape="polygon" alt="Europe" coords="205,39,196,38,177,44,147,47,154,63,155,81,159,83,169,79,174,79,177,83,178,83,187,83,187,83,190,82,190,76,193,70,210,74,207,65,207,62,216,62,216,53,213,46,206,38,206,38" href="eu.htm">
<area shape="polygon" alt="Africa" coords="158,83,147,96,144,113,144,113,153,126,175,137,180,150,177,158,186,185,195,182,217,170,227,152,215,137,224,128,225,118,216,118,213,112,209,106,201,91,189,88,185,90,177,86,174,83,167,83" href="af.htm">
<area shape="polygon" alt="Middle East" coords="241,74,213,72,212,75,209,77,195,73,191,83,204,83,201,87,217,116,244,100,244,85,240,74" href="me.htm">
<area shape="polygon" alt="Asia"  coords="210,35,210,35,219,56,220,62,210,65,213,68,244,72,246,98,246,98,245,104,259,128,279,131,279,131,288,145,289,145,306,150,322,145,330,147,331,137,331,137,312,124,311,84,303,59,293,50,264,44,231,37,231,37,209,35" href="as.htm">
<area shape="polygon" alt="Australia/Oceania" coords="329,151,315,149,315,149,306,156,293,165,282,186,304,195,311,206,348,190,373,158,376,127,333,129,332,146,329,150,329,150" href="au.htm">
<area shape="rect" coords="3,95,12,108" href="na.htm">
<area shape="default" nohref></map></td>
  </tr>
</table>

Hi,

that's great and thank you for the map area point.
Please try to redownload the generator (it's slightly modified) and let me know if it works fine for you now.

PS: however, it still better to include the "normal" links at your pages additinally to image maps to ensure they're crawled successfully by search engine bots.

New version worked perfect with the image maps href links.  Thank you for the feature addition!

sh-2.05$
1 | 66 | 61.9 | 0:03 | 3:32 | 1 | 616.1 Kb | 1 | 0 | 616
20 | 47 | 1,116.9 | 0:12 | 0:29 | 1 | 678.8 Kb | 16 | 446 | 62
60 | 7 | 2,114.1 | 0:26 | 0:03 | 1 | 720.4 Kb | 44 | 563 | 42
80 | 514 | 2,787.4 | 0:36 | 3:53 | 2 | 783.9 Kb | 63 | 169 | 63

...... ..... ......

3100 | 454 | 71,124.2 | 17:50 | 2:36 | 3 | 3,656.6 Kb | 1956 | 3951 | 18
3120 | 434 | 71,675.9 | 17:58 | 2:30 | 3 | 3,681.7 Kb | 1971 | 4021 | 25
3140 | 414 | 72,058.1 | 18:03 | 2:22 | 3 | 3,706.1 Kb | 1981 | 4044 | 25
3180 | 374 | 72,706.7 | 18:10 | 2:08 | 3 | 3,689.5 Kb | 1998 | 4070 | -17
3240 | 314 | 73,940.6 | 18:35 | 1:48 | 3 | 3,791.3 Kb | 2033 | 4125 | 102
Allowed memory size of 8388608 bytes exhausted (tried to allocate 40 bytes)

I've deselected html generation and also calculate changelog is off.  I also changed the 'xs_sm_size' => '2000' from 50000.

Shouldn't runcrawl do the sitemap in chunks?  Seems like it is trying to do the whole crawl in memory at once?

=======================
Update:

I increased memory from 8MB to 32MB in php.ini.  Hopefully that does it!

memory_limit = 32M      ; Maximum amount of memory a script may consume

Hello,

yes, increasing memory_limit should help.

Hey,

I just installed the script today and it gives me the following error after let's say 10 min of activity

Allowed memory size of 67108864 bytes exhausted (tried to allocate 15993278 bytes) in /home/fileclus/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 8

momory_limit of 64M here, no ror, html and text sitemaps enabled...still after a while it gives me the out of memory message....php5, apache 2.2....quad core 4 gig system runnign redhat ent 5

update...raised memory_limit to 128M from 64...script runs longer but still the same error...also I have disabled html, text, logging and ror sitemaps...nothing

played with var_export and serialize options...same thing...so..basically i disabled everything, tried everything...doesn't work..now what?

update...raised memory_limit to 128M from 64...script runs longer but still the same error...also I have disabled html, text, logging and ror sitemaps...nothing

played with var_export and serialize options...same thing...so..basically i disabled everything, tried everything...doesn't work..now what?

another update...i have raised it to 256Mb...the script runs for about 1h without erros...but in the end..the same error...

Hello,

how many URLs do you have crawled at that point?

Hello Sir/Madam

I have a problem about my website. It's stop generating sitemap and have a error occured.

How to solve it?
***********************************************************
Links depth: 5
Current page: 9-16310211-B0030VBQOK-HAPPYBABY_Organic_Baby_Food_Stage_2_Apricot_Sweet_Potato_35_Ounce_Pouchs_Pack_of_16.html
Pages added to sitemap: 30670
Pages scanned: 31020 (1,170,912.2 KB)
Pages left: 20 (+ 25276 queued for the next depth level)
Time passed: 12:45:44
Time left: 0:00:29
Memory usage: 136,343.5 Kb
Auto-restart monitoring: Sun Oct 02 2011 13:04:37 GMT+0800 (China Standard Time) (35 second(s) since last update)
Resuming the last session (last updated: 2011-10-02 03:53:09)
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 32511466 bytes) in /home2/gbtcheap/public_html/my_webapps/seo/sitemap/pages/class.utils.inc.php on line 102

******************************************************
My site is: [ External links are visible to forum administrators only ]



Please help me ASAP. Thank you

Achiles Matas

Hello,

please let me know your generator URL/login in private message to check this.