Maine Munchies Ad

Wednesday, September 06, 2006

See Googlebot Visits with trackSpiders.cgi

Since Googlebot visits will now be more in sync with the Google cache, according to the Official Google Webmaster Central Blog, more webmasters will want to keep a close eye on Googlebot crawls. Note that the cache timestamp will now be updated even if Googlebot receives a 304 HTTP status code. I wonder what kind of time lag there will be between the time Googlebot visits a page and that timestamp is reflected in the actual Google cache.

Our free, open source spider tracking tool will track each Googlebot visit, displaying timestamp, HTTP status code and page retrieved. The tool tracks other search engine spiders as well. See the following demos:

trackSpiders.cgi - tracks Googlebot, Yahoo! Slurp, MSNbot
trackSpiders.cgi?s=gy - tracks Googlebot and Yahoo! Slurp

This free search engine tool can be downloaded here.

Technorati tags: , , , , , , , ,


Post a Comment

<< Home