robots.txt permission token
Google-Extended: what it is.
Google-Extended is a robots.txt product token that lets a site manage whether content Google crawls may be used for Gemini. It is not a crawler.
Identity
- Operator: Google
- Kind: a robots.txt permission token, not a crawler. It makes no requests of its own.
- robots.txt token:
Google-Extended - In Geordy's crawler directory: A permission token, not a fetcher. It says whether Google may use what Googlebot already fetched for Gemini and Vertex AI grounding. Nothing requests a page under this name. See the full list.
What it controls, per Google
Google calls Google-Extended a standalone product token that web publishers can use to manage whether content Google crawls from their sites may be used for Gemini.
Google says it covers training future Gemini models, which power Gemini Apps and the Vertex AI API for Gemini, and grounding, where Search index content is supplied to Gemini Apps and to Grounding with Google Search on Vertex AI.
Google says Google-Extended does not impact a site's inclusion in Google Search and is not a ranking signal.
robots.txt
Allow Google-Extended
User-agent: Google-Extended
Allow: /Block Google-Extended
User-agent: Google-Extended
Disallow: /Google names Google-Extended as the robots.txt token. The lines below are standard robots.txt syntax around that token.
Google says Google-Extended does not have a separate HTTP request user agent string. Crawling is done with Google's existing user agents, so nothing ever requests a page as Google-Extended and it cannot appear in a request log.
Check your site
See whether your robots.txt currently allows or blocks Google-Extended, and what else AI crawlers find on your site.
Sources
Last reviewed: 19 October 2026. Every statement about what Google-Extended does comes from the operator's own documentation: