---
title: "Google-Extended: block or allow this AI crawler · DotsAgent"
description: "Google-Extended is run by Google and listed under training. Its robots.txt token, whether it follows the rules, its IP list and how to control it."
url: https://dotsagent.io/agent-ready/bots/google-extended
---

Google · Training

# What is Google-Extended?

A control token, not a crawler: Googlebot does the fetching, and Google-Extended decides whether that content may be used for Gemini training and grounding.

- **robots.txt token**: `Google-Extended`
- **Operator**: Google
- **Purpose**: Training
- **Obeys robots.txt**: The operator says it follows robots.txt.
- **Documented by operator**: Yes, on the operator's own pages.
- **Documentation**: [developers.google.com](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers)

It has no user agent of its own, so it never appears in your access logs.

## Block Google-Extended from your whole site

`robots.txt`

```
User-agent: Google-Extended
Disallow: /
```

## Allow it, but keep private paths out

`robots.txt`

```
User-agent: Google-Extended
Allow: /
Disallow: /admin/
```

## Other bots from Google

[Googlebot](https://dotsagent.io/agent-ready/bots/googlebot)

[Google-CloudVertexBot](https://dotsagent.io/agent-ready/bots/google-cloudvertexbot)

[GoogleOther](https://dotsagent.io/agent-ready/bots/googleother)

[Google-Agent](https://dotsagent.io/agent-ready/bots/google-agent)

[Google-GeminiNotebook](https://dotsagent.io/agent-ready/bots/google-gemininotebook)

[Gemini Spark](https://dotsagent.io/agent-ready/bots/gemini-spark)

[Gemini-Deep-Research](https://dotsagent.io/agent-ready/bots/gemini-deep-research)

[GoogleAgent-Mariner](https://dotsagent.io/agent-ready/bots/googleagent-mariner)

[Add it to a robots.txt in the generator →](https://dotsagent.io/agent-ready/robots-txt)

## Sources

1. [developers.google.com](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers)/search/docs/crawling-indexing/google-common-crawlers

Independent reference for people who build AI agents. Not affiliated with any vendor named here.

© 2026 DotsAgent · Facts checked October 1, 2026