Google clarifies how Google's crawlers handle cache control headers
May 13, 2025 am 09:12 AM
Google has added a new section to its crawler and fetcher documentation for HTTP caching, which clarifies how Google’s crawlers handle cache control headers. With that, Gary Illyes from Google also wrote a blog post named HTTP caching asking site owners to allow Google to cache its pages.
What is new. The new documentation now says:
“Google’s crawling infrastructure supports heuristic HTTP caching as defined by the HTTP caching standard, specifically through the ETag response- and If-None-Match request header, and the Last-Modified response- and If-Modified-Since request header.”
Google does not support other HTTP caching directives and if both ETag and Last-Modified response header fields are present in the HTTP response, Google’s crawlers use the ETag value.
Caching decreased. Gary Illyes from Google said in the blog post that the “number of requests that can be returned from local caches has decreased.” He said, “10 years ago about 0.026% of the total fetches were cacheable, which is already not that impressive; today that number is 0.017%.”
“If you’re in the business of making your users happy and perhaps also want to potentially save a few bucks on your hosting bill, talk to your hosting or CMS provider, or your developers about how to enable HTTP caching for your site. If nothing else, your users will like you a bit more,” Gary Illyes added.
Why we care. Caching may help Google crawl your site more efficiently, which may make for a happier Googlebot. There is no mention of any SEO or ranking benefit to using caching, there is also no mention of crawl budget benefit.
The above is the detailed content of Google clarifies how Google's crawlers handle cache control headers. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
How to protect deep work time and retain focus as an SEO
Jun 19, 2025 am 10:07 AM
For any SEO professional, staying focused and productive can be a challenge.With constant algorithm updates, changing trends and a barrage of emails and notifications, it can feel like you’re always playing catch-up.That’s where deep work sessions co
Google AI Overviews, clicks and traffic impact: Unraveling the mystery
Jun 22, 2025 am 09:42 AM
Google started including AI Overviews (AIO) in U.S. search results on May 14. While Google has made vague references to the fact that links within AIO may experience higher click-through rates (CTRs), it remains unclear when directly questioned about
WordPress 6.5 gains lastmod date for sitemaps files
Jun 23, 2025 am 09:42 AM
WordPress version 6.5 now includes support for the lastmod element in sitemap files, which can help search engines identify new or updated content. This enhancement may improve crawl efficiency and reduce server load.Lastmod. The lastmod element can
Rethinking your keyword strategy: Why optimizing for search intent matters
Jun 20, 2025 am 10:20 AM
Search engines continue to evolve, but SEO strategies have failed to keep up. For years, we have relied on keyword research to choose specific searches to target. However, keyword research often prioritizes the wrong goals. Executed well, keyw
Reminder: New Google Search reputation abuse policy starts soon
Jun 28, 2025 am 10:22 AM
Google’s new Search spam policy surrounding reputation abuse – a tactic often called “parasite SEO” by SEO professionals – will go into effect “after May 5,” as confirmed by Google. May 5 falls on this Sunday.This wasn’t unexpected. Back in March, Go
SCAMPER your way to better SEO and content marketing ideas
Jun 18, 2025 am 09:47 AM
There is a lot of content out there. And guess what? 99% of it is terrible. Then, there is that 1% of content – the really good to absolutely phenomenal stuff. Sometimes, that is hot news, like the recent Google Search leak, but there are als
SEO pros and cons of HubSpot vs. WordPress vs. Webflow
Jun 17, 2025 am 11:58 AM
I get asked all the time which web platform someone should use. The main options are HubSpot, WordPress and Webflow. Answer the same question a few times, and it’s probably worth spelling out for the masses.Before I explain my approach to answering t
Google is not about blue links, says ex-CEO Eric Schmidt
Jun 27, 2025 am 09:25 AM
“Google is not about blue links. It’s about organizing the world’s information,” said former executive chairman and CEO of Google Eric Schmidt during a recent appearance on CNBC.When asked about the “blue link economy” and all the brands and business
