Sitemap Generator Forum

I've installed the program and the configuration file saves, but when I click the crawl tab and try to run the crawl I get this error in a box on the page:

Internet Explorer cannot display the webpage
   
   Most likely causes:
You are not connected to the Internet.
The website is encountering problems.
There might be a typing error in the address.
 
   What you can try:
     Diagnose Connection Problems 
 
     More information
 
Any thoughts???

Thanks,

Lawrence

It does seem to be crawling some of the pages. When I return to the crawl page after I get this message, there's a checkbox that says that at least some of the pages have been crawled asking if I want to continue with that session. However, it continues to time out.

I see this message: Please wait. Sitemap generation in progress...

but no progress indicators, just a blank white box, then the error in the previous post appears in the box.

Here are my current php settings:

;;;;;;;;;;;;;;;;;;;
; Resource Limits ;
;;;;;;;;;;;;;;;;;;;

max_execution_time = 900
max_input_time = 500
memory_limit = 32M

I find this in the log file:

26-Aug-2007 07:42:28] PHP Warning:  fopen(/home/ihsrea5/public_html/generator/data/crawl_state.log) [<a href='function.fopen'>function.fopen</a>]: failed to open stream: Permission denied in /home/ihsrea5/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 24
[26-Aug-2007 07:42:28] PHP Warning:  fwrite(): supplied argument is not a valid stream resource in /home/ihsrea5/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 25
[26-Aug-2007 07:42:28] PHP Warning:  fclose(): supplied argument is not a valid stream resource in /home/ihsrea5/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 26
[26-Aug-2007 07:42:29] PHP Warning:  fopen(/home/ihsrea5/public_html/generator/data/crawl_dump.log) [<a href='function.fopen'>function.fopen</a>]: failed to open stream: Permission denied in /home/ihsrea5/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 24
[26-Aug-2007 07:42:29] PHP Warning:  fwrite(): supplied argument is not a valid stream resource in /home/ihsrea5/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 25
[26-Aug-2007 07:42:29] PHP Warning:  fclose(): supplied argument is not a valid stream resource in /home/ihsrea5/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 26

Hello,

make sure that you have set 0777 permissions for generator/data/ folder.

Yes that was previously done. Anything else?

Hello,

replied to your email.

Yes, I've changed the max_execution_time setting to 10000 with the same result as my original post.

Replied via email again earlier today.

XADMIN:

Hello,
I have the same problem. Can you tell me how to solve...

Hello,

replied to your private message.

Got pretty far along with the indexing of pages by running from the SSH command line, then the script stopped and now when I try to run it I get:

FATAL:  erealloc():  Unable to allocate 79691776 bytes

What should I do?

Also, Is there a setting from the command line that allows the script to continue running if I close my SSH connection?

Hello,

please also increase the memory_limit setting in php configuration.

Also, Is there a setting from the command line that allows the script to continue running if I close my SSH connection?

Usually you can append "&" to the command line for that, although that may not work in some cases:
/usr/bin/php /path/generator/runcrawl.php &

I had it up to 1024MB.

It looks like when I limited the number of pages it was able to create the html sitemap pages, but now I'm getting this error when it tries to create the xml sitemap file:

Warning: fwrite(): supplied argument is not a valid stream resource in /home/ihsrea5/publ
ic_html/generator/pages/class.xml-creator.inc.php(2) : eval()'d code on line 154

Warning: fwrite(): supplied argument is not a valid stream resource in /home/ihsrea5/publ
ic_html/generator/pages/class.xml-creator.inc.php(2) : eval()'d code on line 165

Warning: fwrite(): supplied argument is not a valid stream resource in /home/ihsrea5/publ
ic_html/generator/pages/class.xml-creator.inc.php(2) : eval()'d code on line 175

Warning: fclose(): supplied argument is not a valid stream resource in /home/ihsrea5/publ
ic_html/generator/pages/class.xml-creator.inc.php(2) : eval()'d code on line 176
<br>Creating HTML sitemap...<script>
top.location = 'index.php?op=view'
</script>

I've noticed that the configuration says that the sitemap.xml file is not writeable even though its currently set at 777

should the "Current path to Sitemap generator is:" be set to include the 'sitemap.xml' file name with the path or just contain the path. I've tried both ways but still get the error message on the configuration page.