What Does Google-Extended Actually Control for Website Publishers?
Google-Extended is a robots.txt token, not a crawler. There is no Google-Extended user agent in your logs: Google crawls with its usual agents and reads this token as a permission setting. Disallowing it tells Google not to use your content to train future Gemini models, or to ground answers in the Gemini app and in Grounding with Google Search on Vertex AI. Google says it does not affect a site's inclusion in Google Search and is not a ranking signal. Because AI Overviews and AI Mode are part of Search, blocking Google-Extended does not remove you from them. To limit those features, use Search Console's Search generative AI control, or page controls such as nosnippet and max-snippet.
What does each Google control affect?
| Control | Google Search results | AI Overviews and AI Mode | Gemini app grounding | Vertex AI grounding | Gemini model training |
|---|---|---|---|---|---|
| Disallow Google-Extended in robots.txt | No effect | No effect | Excluded | Excluded | Excluded from future training |
| Search generative AI control set to exclude | No effect on ranking elsewhere | Excluded | No stated effect | No stated effect | No stated effect |
| nosnippet or max-snippet | Snippet removed or shortened | Not used, or used less, as direct input | Not documented | Not documented | Not documented |
| noindex | Page removed | Page cannot appear | Not documented | Not documented | Not documented |
| Disallow Googlebot | Pages not crawled | Pages cannot be used | Not documented | Not documented | Not documented |
The first row comes from Google's crawler documentation, the second from Search Console help, and the snippet rows from Google's robots meta tag specification. Where Google has not documented an effect, we have left it as not documented rather than guess.
Can blocking AI crawlers stop Google AI Overviews from using your content?
Not by blocking Google-Extended, and not by blocking other companies' AI crawlers. AI Overviews and AI Mode are built on Google Search's index, which Googlebot fills. Google's AI features documentation says the controls for what appears from your pages are nosnippet, data-nosnippet, max-snippet and noindex, and that Google-Extended applies to other Google systems.
Since August 31, 2026, every site can also use the Search generative AI control in Search Console. Set to exclude, it removes your links and content from AI Overviews, AI Mode and Discover's AI features, and Google says it is not used as a ranking or inclusion signal elsewhere in Search. How to apply it per subfolder, and how the snippet controls differ, is covered in how to limit what Google AI features use from your pages.
Blocking Googlebot itself would also keep your content out of AI Overviews, but it stops Google crawling your pages at all, so they lose their snippets and nearly all Search visibility; blocked URLs can still be listed without a description if other sites link to them.
Does blocking Google-Extended reduce my chances of appearing in AI Mode?
According to Google, no. Its documentation says Google-Extended does not affect inclusion in Google Search and is not used as a ranking signal there, and AI Mode is a Search feature. You can check the effect yourself in Search Console's Generative AI performance report, which shows AI Overview and AI Mode impressions by page, as covered in how AI Overviews and AI Mode show up in Search Console.
The Gemini app is different. Google says Google-Extended controls grounding in Gemini Apps, where the model is given content from the Search index at prompt time. Blocking it can therefore reduce how often the Gemini app uses and cites your pages, even while AI Mode is unaffected. It also removes your content from developers' Gemini answers grounded through Vertex AI; Google's Vertex documentation says grounding does not use pages that disallow Google-Extended, as covered in why Gemini API answers show redirect links.
Why can't a CDN or firewall block Google-Extended?
Because there is no Google-Extended traffic to block. Google says the token has no separate user agent string; crawling happens with existing Google user agents. A firewall rule matching “Google-Extended” catches nothing, and a rule blocking Google's crawlers broadly would hit Googlebot and remove you from Search. The only way to use the token is a robots.txt group such as:
User-agent: Google-Extended
Disallow: /
This also means a CDN setting labelled as blocking Google's AI use may work by editing robots.txt rather than blocking requests. Check what your CDN or host changes, as covered in how hosting providers block AI bots.
How do you check your current settings?
- Read your robots.txt at your domain and look for a Google-Extended group. If there is none, Google-Extended follows your
User-agent: *rules, so a file that allows Googlebot but disallows everything else also blocks it. If you use a CDN, check whether it adds rules: Cloudflare's managed robots.txt, for example, prepends a block that disallows Google-Extended alongside other AI crawlers. - Check the Search generative AI control in Search Console under Settings, and whether each property inherits it from a parent.
- Inspect key pages with Search Console's URL Inspection to see any nosnippet, max-snippet or noindex rules Googlebot received, including X-Robots-Tag headers.
- Write down the intent behind each setting, so a future team does not remove a deliberate block or keep an accidental one.
Should you block Google-Extended?
It depends on what you want to protect:
- Block it if you do not want your content used to train future Gemini models, and you accept less use of your content in Gemini app answers and Vertex AI grounding.
- Allow it if appearing in Gemini app answers matters to you, for example because your audience uses the Gemini app, or developers build tools on Vertex AI grounding in your field.
- Either way, your Google Search, AI Overviews and AI Mode visibility is governed by other controls.
Rules for other companies' crawlers are separate decisions, covered in how to allow search AI bots while blocking training crawlers. If your brand has vanished from AI Overviews and you are unsure why, see whether you could have opted out of AI Overviews.
We have not measured how blocking Google-Extended changes Gemini app citations. This page is based on Google's crawler documentation, its AI features and robots meta tag documentation, Search Console help and Vertex AI grounding documentation.
What might this look like in practice?
Consider an illustrative cookery publisher, Tarncombe Kitchen, that blocked Google-Extended in 2024 to keep recipes out of AI training, then worried when AI Overviews summarised its recipes anyway. Reading Google's documentation, the editor saw that Google-Extended never covered AI Overviews. The team kept the Google-Extended block, used data-nosnippet on the method steps of premium recipes so AI features quoted ingredients and timings but not full methods, and tracked AI feature impressions in Search Console's generative AI report. Gemini app citations stayed low, which the team accepted as the cost of opting out of training.
What do people ask about Google-Extended?
Is Google-Extended a crawler?
No. It is a robots.txt token. Google crawls with its usual user agents.
Does blocking Google-Extended remove me from AI Overviews?
No. Google says it does not affect inclusion in Google Search, which includes AI Overviews and AI Mode.
What does blocking Google-Extended remove me from?
Training of future Gemini models, and grounding in the Gemini app and Grounding with Google Search on Vertex AI.
How do I opt out of AI Overviews then?
Use the Search generative AI control in Search Console, or page controls such as nosnippet.
Published by The Kovmere Team · updated Sep 29, 2026 · More resources