Sitemap Generation takes at least 5 attempts
« on: November 09, 2011, 03:47:29 PM »
Hi

I've been using this tool for ages now and suddenly I am struggling with it, for various reasons a while back I closed my site down. Having brought it back, suddenly sitemap generation is really poor. At the moment my site has about 3,500 pages including normal website htm pages and blog pages from wordpress. Nothing really has changed from when the site was live before, in fact then there was a php Forum as well which has gone now.

Basically it times out at certain points giving the 121 seconds with no response, it starts counting again before giving a http 500 error. I then have to go back to the site from my favourites and click the "continue" option. It starts up again and the same thing happens four or five times before finally creating the map.

My site is still being built and will end up a lot bigger than it is now so I am concerned. I am tried to edit my php.ini file as I saw suggested on here but no luck. After a while with a blank screen the screen gives the update as to where it is but then the counting starts before the http 500 error.

My phpini file is below the changes I made were adding the memory_limit line and the max_execution line.

Any help will be appreciated, as I say it always worked before and apart from now not having a Forum nothing has changed, same host, same conditions etc

register_globals = off
allow_url_fopen = off
memory_limit = 150
max_execution_time settings = 270
expose_php = Off
max_input_time = 60
variables_order = "EGPCS"
extension_dir = ./
upload_tmp_dir = /tmp
precision = 12
SMTP = relay-hosting.secureserver.net
url_rewriter.tags = "a=href,area=href,frame=src,input=src,form=,fieldset="


; Only uncomment zend optimizer lines if your application requires Zend Optimizer support

;[Zend]
;zend_optimizer.optimization_level=15
;zend_extension_manager.optimizer=/usr/local/Zend/lib/Optimizer-3.3.3
;zend_extension_manager.optimizer_ts=/usr/local/Zend/lib/Optimizer_TS-3.3.3
;zend_extension=/usr/local/Zend/lib/Optimizer-3.3.3/ZendExtensionManager.so
;zend_extension_ts=/usr/local/Zend/lib/Optimizer_TS-3.3.3/ZendExtensionManager_TS.so


; -- Be very careful to not to disable a function which might be needed!
; -- Uncomment the following lines to increase the security of your PHP site.

;disable_functions = "highlight_file,ini_alter,ini_restore,openlog,passthru,
;            phpinfo, exec, system, dl, fsockopen, set_time_limit,
;                     popen, proc_open, proc_nice,shell_exec,show_source,symlink"
Re: Sitemap Generation takes at least 5 attempts
« Reply #2 on: November 11, 2011, 03:44:06 PM »
Hi

Thanks for the tip but no luck I'm afraid, doesn't seem to matter what figures I use in the php.ini file it simply won't create a sitemap in less than 5 or 6 attempts. It's very odd as before my site was at least 30,000 pages and it worked ok.
Re: Sitemap Generation takes at least 5 attempts
« Reply #3 on: November 12, 2011, 08:06:25 AM »
In this case I would recommend to run generator in command line if you have ssh access to server (instead of running it in browser).
Re: Sitemap Generation takes at least 5 attempts
« Reply #4 on: November 12, 2011, 01:04:53 PM »
Hi

I've never tried but I'm pretty sure godaddy give ssh access, assuming I can get ssh access what command do I use to run generator? Thanks
Re: Sitemap Generation takes at least 5 attempts
« Reply #5 on: November 14, 2011, 04:31:40 PM »
The command line is displayed on Crawling page of sitemap generator and looks like:
/usr/bin/php /path/to/generator/runcrawl.php