Video SEO

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Tuesday, 5 September 2006

Better details about when Googlebot last visited a page

Posted on 07:34 by Unknown
Most people know that Googlebot downloads pages from web servers to crawl the web. Not as many people know that if Googlebot accesses a page and gets a 304 (Not-Modified) response to a If-Modified-Since qualified request, Googlebot doesn't download the contents of that page. This reduces the bandwidth consumed on your web server.

When you look at Google's cache of a page (for instance, by using the cache: operator or clicking the Cached link under a URL in the search results), you can see the date that Googlebot retrieved that page. Previously, the date we listed for the page's cache was the date that we last successfully fetched the content of the page. This meant that even if we visited a page very recently, the cache date might be quite a bit older if the page hadn't changed since the previous visit. This made it difficult for webmasters to use the cache date we display to determine Googlebot's most recent visit. Consider the following example:
  1. Googlebot crawls a page on April 12, 2006.
  2. Our cached version of that page notes that "This is G o o g l e's cache of http://www.example.com/ as retrieved on April 12, 2006 20:02:06 GMT."
  3. Periodically, Googlebot checks to see if that page has changed, and each time, receives a Not-Modified response. For instance, on August 27, 2006, Googlebot checks the page, receives a Not-Modified response, and therefore, doesn't download the contents of the page.
  4. On August 28, 2006, our cached version of the page still shows the April 12, 2006 date -- the date we last downloaded the page's contents, even though Googlebot last visited the day before.
We've recently changed the date we show for the cached page to reflect when Googlebot last accessed it (whether the page had changed or not). This should make it easier for you to determine the most recent date Googlebot visited the page. For instance, in the above example, the cached version of the page would now say "This is G o o g l e's cache of http://www.example.com/ as retrieved on August 27, 2006 13:13:37 GMT."

Note that this change will be reflected for individual pages as we update those pages in our index.
Email ThisBlogThis!Share to XShare to Facebook
Posted in crawling and indexing, search results | No comments
Newer Post Older Post Home

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Popular Posts

  • Come see us at SES London and hear tips on successful site architecture
    If you're planning to be at Search Engine Strategies London February 13-15, stop by and say hi to one of the many Googlers who will be ...
  • All About Googlebot
    I've seen a lot of questions lately about robots.txt files and Googlebot's behavior. Last week at SES, I spoke on a new panel called...
  • Come and see us at Search Engine Strategies Chicago
    If you're planning to be at SES Chicago this week, be sure to stop by and say hi to the many Googlers who are coming out to brave the c...
  • Discover your links
    Update on October 15, 2008 : For more recent news on links, visit Links Week on our Webmaster Central Blog. We're discussing internal l...
  • Useful information you may have missed
    When we launched this blog in early August, we said goodbye to the Inside Google Sitemaps blog and started redirecting it here. The redirec...
  • Listen in - Matt Cutts and Vanessa Fox talk search
    Tune into Webmaster Radio Thursday, August 31 at 1 pm Pacific to hear Matt Cutts and me take over GoodKarma while GoodROI (Greg Nil...
  • Debugging blocked URLs
    Vanessa's been posting a lot lately, and I'm starting to feel left out. So here my tidbit of wisdom for you: I've noticed a cou...
  • About badware warnings
    Some of you have asked about the warnings we show searchers when they click on search results leading to sites that distribute malicious sof...
  • How to verify Googlebot
    Lately I've heard a couple smart people ask that search engines provide a way know that a bot is authentic. After all, any spammer cou...
  • Badware alerts for your sites
    As part of our efforts to protect users, we have been warning people using Google search before they visit sites that have been determined t...

Categories

  • crawling and indexing
  • events
  • feedback and communication
  • general tips
  • localization
  • products and services
  • search results
  • sitemaps
  • webmaster guidelines
  • webmaster tools

Blog Archive

  • ►  2007 (12)
    • ►  March (2)
    • ►  February (7)
    • ►  January (3)
  • ▼  2006 (34)
    • ►  December (5)
    • ►  November (7)
    • ►  October (7)
    • ▼  September (8)
      • Fresher query stats
      • Introducing Google Checkout
      • How to verify Googlebot
      • Debugging blocked URLs
      • For Those Wondering About Public Service Search
      • Setting the preferred domain
      • Information about Sitelinks
      • Better details about when Googlebot last visited a...
    • ►  August (7)
Powered by Blogger.

About Me

Unknown
View my complete profile