What is Google-Extended?
Google-Extended is a token you name in robots.txt to tell Google whether content it crawls from your site may be used for its Gemini models. It is not a crawler. No bot called Google-Extended ever visits your site. Google's documentation says Google-Extended "doesn't have a separate HTTP request user agent string" and that crawling is done with existing Google user agents, with the token "used in a control capacity" [2].
A site that does not want its pages used to train Gemini adds this to robots.txt:
User-agent: Google-Extended
Disallow: /Googlebot still crawls the site as before, the pages stay in Google Search, and rankings do not change. What changes is what Google may do with the crawled content afterward.
What does Google-Extended control?
Google-Extended controls two uses of your content: training future Gemini models, and grounding answers in Gemini Apps and in Grounding with Google Search on Vertex AI [2]. Google's own wording describes the token as managing whether crawled content "may be used for training future generations of Gemini models that power Gemini Apps and Vertex AI API for Gemini and for grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy)" in those two products [2].
That second use is the one most guides leave out. Google-Extended is not only a training opt-out. Blocking it also asks Google not to feed your pages to Gemini Apps at the moment it answers a prompt, which is when a page can be cited.
| Use of your content | Covered by Google-Extended? |
|---|---|
| Training future Gemini models | Yes [2] |
| Grounding answers in Gemini Apps | Yes [2] |
| Grounding with Google Search on Vertex AI (developer API) | Yes [2] |
| Appearing in Google Search results | No [2] |
| Google Search ranking | No [2] |
| AI Overviews and AI Mode in Google Search | No [5] |
Is Google-Extended a crawler?
No. Google-Extended is a robots.txt product token, not a crawler, and it never appears in your server logs. Google lists it on its page of common crawlers, which is one reason people assume it is a bot, but the same entry says Google-Extended has no user agent string of its own [2].
This matters for anyone checking whether a block "worked". With GPTBot or ClaudeBot, a site owner can look for the bot's user agent in access logs and see whether it keeps visiting. With Google-Extended there is nothing to look for. Googlebot keeps crawling either way, and the robots.txt rule only changes what Google is permitted to do with the content. There is no published way to verify from the outside that Google honors the rule; site owners rely on Google's documentation.
Does blocking Google-Extended remove you from AI Overviews?
No. Blocking Google-Extended does not remove a site from AI Overviews or AI Mode. Those features are part of Google Search, and Google states that Google-Extended "does not impact a site's inclusion in Google Search" [2]. Google's guidance on AI features points site owners to the ordinary Search controls instead: nosnippet, data-nosnippet, max-snippet and noindex [5]. The same page mentions Google-Extended only as a way to limit "AI training and grounding in some of Google's other systems" [5].
This is the most widely repeated mistake about Google-Extended. A publisher that blocks it expecting to disappear from AI Overviews will see no change, because AI Overviews draw on the Search index that Googlebot builds.
The Search controls are blunt. nosnippet and noindex also affect normal search results, which is why publishers pushed for a dedicated AI Search opt-out (covered below).
When did Google introduce Google-Extended?
Google introduced Google-Extended on September 28, 2023, as a way to manage whether sites help improve Bard and Vertex AI generative APIs [1]. Since then Google has renamed the products it covers and clarified what it does not touch, but the token itself has not changed.
2023: launched for Bard and Vertex AI
Google announced Google-Extended in a post by Danielle Romain, VP of Trust, on September 28, 2023 [1]. The post described it as "a new control that web publishers can use to manage whether their sites help improve Bard and Vertex AI generative APIs, including future generations of models that power those products" [1]. Google's documentation changelog records the new user agent token on the same date [3].
2024: Gemini Apps, and a Search clarification
On February 8, 2024, Google updated the documentation to say Gemini Apps (the renamed Bard) is affected by Google-Extended and that Google-Extended does not affect Google Search [3]. Search Engine Journal reported the added sentence that blocking it does not affect inclusion or ranking in Google Search [4].
2026: a separate opt-out for AI Search
On June 2, 2026, Google announced a Search Console toggle that lets sites opt out of AI Overviews, AI Mode and AI Overviews in Discover while staying in regular results [6]. Google said the setting would not be used as a ranking signal [6]. The toggle is separate from robots.txt and from Google-Extended [6] [7]. It rolled out first to a subset of UK site owners, following a conduct requirement from the UK's Competition and Markets Authority, with a global rollout planned [6] [7]. It does not cover the Gemini app [6] [8].
What is the difference between Google-Extended and the AI Search opt-out?
Google-Extended governs Gemini training and Gemini grounding, while the Search Console AI opt-out governs AI Overviews and AI Mode inside Google Search. They cover different products and neither one replaces the other.
| Google-Extended | Search Console AI opt-out | |
|---|---|---|
| How you set it | robots.txt token [2] | Toggle in Search Console [6] |
| Introduced | September 28, 2023 [1] | Announced June 2, 2026 [6] |
| Gemini model training | Covered [2] | Not covered |
| Gemini app answers | Covered (grounding) [2] | Not covered [6] [8] |
| AI Overviews and AI Mode | Not covered [5] | Covered [6] |
| Regular Google Search | No effect [2] | No effect [6] |
| Availability | Global | UK subset first [6] |
A site that wants out of every Google generative feature needs both. A site that wants citations in AI Overviews but no part in model training can block Google-Extended and leave the toggle off.
What is the difference between Google-Extended and GPTBot?
GPTBot is a real crawler with its own user agent that fetches pages for OpenAI, while Google-Extended is only a permission flag applied to pages Googlebot has already fetched. Blocking GPTBot stops a bot from visiting. Blocking Google-Extended stops nothing from visiting; it limits reuse. The same logic separates Google-Extended from ClaudeBot and from Common Crawl's CCBot, which are all crawlers you can see in logs.
This is also why Google could launch Google-Extended without a new bot. Google does not need a second crawl to train or ground Gemini: it already holds the content from Googlebot. The token simply tells Google which of those uses a site permits.
How do you set up Google-Extended in robots.txt?
To block Google-Extended, add a User-agent: Google-Extended group to the robots.txt file at the root of your domain with the paths you want excluded. Google's documentation gives an example that blocks an archive except for one section [2]:
User-agent: Google-Extended
Allow: /archive/1Q84
Disallow: /archive/Three checks are worth making:
- Spelling and hyphen. The token is
Google-Extended. A typo means the group matches nothing and the default rules apply. - Don't touch Googlebot. Never block Googlebot to keep content out of Gemini. That removes the site from Google Search, which Google-Extended exists to avoid.
- One robots.txt per host. A subdomain such as
blog.example.comneeds its own rule.
Should you block Google-Extended?
Block Google-Extended if keeping your content out of Gemini model training matters more to you than being cited in Gemini app answers. Leave it open if Gemini citations are part of your visibility plan, because blocking it also blocks grounding in Gemini Apps [2].
For most businesses that want to be found in AI answers, the trade-off runs like this:
- Brands chasing AI visibility have good reason to leave Google-Extended open. Blocking it costs grounding in the Gemini app and gains nothing in Search.
- Publishers protecting licensed or paywalled content have good reason to block it, since training use is the concern and Search traffic is unaffected.
- Anyone trying to leave AI Overviews should not use Google-Extended at all. The tools for that are the Search Console toggle where it is available [6] or the snippet and indexing controls [5].
Google has not published how many sites block Google-Extended or what share of Gemini answers come from grounding on blocked versus open sites. Any figure presented as Google-Extended's traffic impact is an estimate.