Catching Googlebot

It is always nice to see unique visitors to your site. For a blog that doesn’t get too many hits, it’s very nice.

Google Analytics is a great, robust way to analyze your visitors based on various parameters such as date visited, location, etc. It produces readable charts and graphs and datasets that can be exported via the web interface.

Anaytics is part of Google’s Webmaster Tools which in itself is very useful. You provide a site map and Googlebot will crawl your site and index it. Today, I caught Googlebot in the act. Recently I implemented a script to track my visitors using Perl and MySQL. Basically, each time a page is loaded, the time stamp is stored with the IP address to a table in a MySQL database.

As I am still working on the code I happened to be looking at this log and noticed a new IP address. Not expecting any visits to the site, I wanted to see where this visitor was coming from. After a quick Google search I determined this IP to be none other than our friendly neighborhood spider-I mean Googlebot. It’s name was crawl-66-249-74-117.googlebot.com residing at 66.249.74.117

http://whatismyipaddress.com/ip/66.249.74.117

Now, the question is, what did it do? Unfortunately, my script isn’t powerful enough (yet?) to give any more information.

Interstingly enough, there are A LOT of Googlebots out there. More specifically, Google has a large chuck of IPv4 address space allocated to the Gooblebot software.

FROM TO WHOIS
64.233.160.0 64.233.191.255 WHOIS
66.102.0.0 66.102.15.255 WHOIS
66.249.64.0 66.249.95.255 (My site was hit from this range) WHOIS
72.14.192.0 72.14.255.255 WHOIS
74.125.0.0 74.125.255.255 WHOIS
209.85.128.0 209.85.255.255 WHOIS
216.239.32.0 216.239.63.255 WHOIS

Google’s infrastructure really is mind-blowing…

 

One thought on “Catching Googlebot”

Leave a Reply