Competitor Analysis Tool: Analyze rival websites and gain valuable SEO and PPC insights
« previous next »
Pages: 1

jnmunsey

*
  • *
  • 15
Migrating to new faster server and crawls now take four times as long
« on: September 17, 2010, 03:17:27 PM »
I migrated to a new, faster server - 8 cores instead of 4  and 12GB RAM up from 4GB.  Crawls now are taking 6.5 hours, up from under 2 hours previously.  Page count remains the same.

What should I be looking for?
« Last Edit: September 17, 2010, 03:24:04 PM by jnmunsey »
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: Migrating to new faster server and crawls now take four times as long
« Reply #1 on: September 18, 2010, 08:37:10 AM »
Hello,

please let me know your generator URL/login in private message to check this.
Logged

jnmunsey

*
  • *
  • 15
Re: Migrating to new faster server and crawls now take four times as long
« Reply #2 on: October 03, 2010, 05:55:05 PM »
FYI the issue was related to my PHP caching app not running due to PHP running in CGI mode.  The caching apps require mod_php and do not run under CGO/Fast CGI as they run each PHP script in a separate process.

Once fixed the speed returned.
Logged

XML-Sitemaps Support

*
  • *****
  • 11781
Re: Migrating to new faster server and crawls now take four times as long
« Reply #3 on: October 04, 2010, 08:10:24 AM »
Thank you for the follow-up!
Logged
Pages: 1
« previous next »