acnz

*
  • *
  • 14
robots.txt network error by Google
« on: July 23, 2007, 04:09:56 PM »
Finally ! A reason for a problem that seems to be plaguing a lot of people - this might be handy information.

You run sitemap generator perfectly.
It creates sitemap.xml and you can see it in your browser.
Submit to Google and the reply from Google is robots.txt unreachable.
You check in your browser and you can see it ok.

IS YOUR WEB HOST BLOCKING GOOGLE ??

This is a reply from my Host Providers:
"It is possible that their ip is blocked.
We have blocked some ips in different servers due to different reasons. Like
some spiders cause excessive access. For example,your website uses a lot of
sql and the spider visits your website frequently, it can cause a lot of
load to mysql and the server, then it will be blocked."

Solution ? Suggest that they UNBLOCK GOOGLE !!