Had this problem too - setting IP address to 127.0.0.1 made runcrawl work but it is crawling the wrong website
On shared (apache) server with PHP 5
config:-
[ ] brackets are not real data - my edits
$grab_parameters = array(
'xs_inc_skip' => '\\.(pdf|doc|txt|rtf)',
'xs_exc_skip' => '\\.(zip|m4a|m4v|rar|tar|bz2|tgz|exe|gif|jpg|png|class|jar|mpeg|mpg|mp3|wav|mp4|avi|wmv|gz|mov|mid|ra|ram)',
'xs_proto_skip' => '(\\#|mms:|mailto:|https:|javascript:|ftp:|news:|aim:)',
'xs_exec_time' => '900',
'xs_cache' => '',
'xs_max_pages' => '0',
'xs_initurl' => '[ External links are visible to forum administrators only ].[mydomain].co.uk',
'xs_freq' => 'daily',
'xs_lastmod' => '0',
'xs_lastmodtime' => '2005-06-30 19:35:14',
'xs_priority' => '0.5',
'xs_descpriority' => '0.8',
'xs_autopriority' => '1',
'xs_smname' => '/var/www/[homedir...]/sitemap.xml',
'xs_smurl' => '[ External links are visible to forum administrators only ].[mydomain].co.uk/sitemap.xml',
'xs_gping' => '0',
'xs_yping' => '',
'xs_makehtml' => '1',
'xs_maketxt' => '1',
'xs_makeror' => '1',
'xs_makebase' => '0',
'xs_compress' => '0',
'xs_excl_urls' => '',
'xs_notconfigured' => '0',
'xs_login' => '',
'xs_password' => '',
'xs_savestate_time' => '180',
'xs_ipconnection' => '127.0.0.1',
'xs_portconnection' => '80',
'xs_sm_size' => '40000',
'xs_sm_text_filename' => '',
'xs_sm_text_url' => '',
'xs_cleanurls' => '',
'xs_usecurl' => '1',
'xs_robotstxt' => '1',
'xs_dumptype' => 'serialize',
'xs_cleanpar' => 'PHPSESSID|sid|osCsid',
'xs_chlogorder' => 'asc',
'xs_exclude_check' => '1',
'xs_dateformat' => 'Y, F j',
'xs_utf8' => '0',
'xs_allow_httpcode' => '200',
'xs_chlog' => '0',
'xs_htmlname' => '/var/www/[homedir...]/xml-sitemap/data/sitemap.html',
'xs_htmlpart' => '1000',
'htmlurl' => '[ External links are visible to forum administrators only ].[mydomain].co.uk/xml-sitemap/data/sitemap.html',
'xs_max_depth' => '0',
'xs_memlimit' => '0',
'xs_delay_req' => '3',
'xs_delay_ms' => '3',
'xs_incl_urls' => '',
'xs_incl_only' => '',
'xs_ind_attr' => '',
'xs_email' => '',
'xs_metadesc' => '0',);
BUT I AM GETTING DATA FOR:-
/var/www/[FQDN_public_webdir]/
is there anyway of telling this software where to look in the server dir tree ?
this is a dedicated server and I have root access - i want to use this on more than one virtual host directory for each website eg:
/var/www/[THIS WEBSITE/homedir...]/xml-sitemap
/var/www/[ANOTHER WEBSITE/homedir...]/xml-sitemap
/var/www/[ANOTHER WEBSITE/homedir...]/xml-sitemap
and get the right data for each site
thanks