Sitemap.xml error URL timeout:robots.txt timeout

by Sai on January 4, 2009

I recently came across this error when I logged onto my webmaster tools.I searched the web to find out the solution and finally I did solve the problem.

But how?

  • Some say this is a Google bot’s bug
  • Some say there is a problem from your webhost who are blocking Googlebot using a firewall.

While you can’t do much in the first case,lets see whether the second one is the real problem.

Just contact you host or submit a ticket and ask them if they could to give the list of blocked IP’s by their firewall.

See whether there is any Googlebot’s IP in there.Just for your reference some Google IP’s are 

216.239.32.0/19 

64.233.160.0/19 

66.249.80.0/20 

72.14.192.0/18 

209.85.128.0/17 

66.102.0.0/20 

74.125.0.0/16 

64.18.0.0/20 

207.126.144.0/20

(Remember Google IP’s are not limited to these and will be changing from time to time).

Ask them to immediately remove the IP from the blocked list should you find a Googlebot IP.

Resubmit your sitemap to see if the error still exists.If it does then may the first reason be the problem which even I think is.

Because I made no changes in my blog except to publish posts.The only change I made was to upgrade wordpress from 2.5.1 to 2.7.

Could that be the problem?

No.

Even after the upgrade my sitemap showed no errors.But suddenly one bad day,the errors began to show.

It continued for three days and in these days I resubmit the same sitemap 6 times but to no avail.

The seventh time I submitted the same sitemap and logged onto webmaster tools after 1hr and after publishing a new post,no  ERRORS was shown.

Sitemap.xml is perfectly alright and accepted by Google.

Another error which is got in those three days was “your sitemap was not processed.Was it made according to our guidelines?” or something of that sort.

What could be the real problem?

Is this not a Googlebot bug?

Visitors who read this article also read:

  1. xml-sitemap error, make sure the file exists and is writableIf you got the above message when trying to implement the XML sitemap, then you have the solution here. I’m...
  2. error : /tmp directory fullWarning: Unknown: Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/tmp) in...
  3. www or no-www in a website URL?www or no www? Enforce www preference plugin for WordPress If you are in a dilemma whether to use www...
  4. Ovi Store App login errorYesterday was a strange day for me with my Nokia device. There’s not a single day till yesterday that I...
  5. Blogger Vs blog.co.in,which is better?Like any other starter, i used blogger in my early days of blogging.It was a nice,neatly built,easy to use interface...
  6. Fatal error:call to undefined function wp_get_current_user(); in line 920You probably might have got the above error and hence are here.Ok lets see what to do now. I uploaded...

Leave a Comment

Please tell us what YOU think. Copied comments will not be published. Any link to third party websites inside the comment box will either be edited or removed.

{ 1 trackback }

Copyright © 2008-11.Sai Digital Magazine.