---
title: "Googlebot: block or allow this AI crawler · DotsAgent"
description: "Googlebot is run by Google and listed under search. Its robots.txt token, whether it follows the rules, its IP list and how to control it."
url: https://dotsagent.io/agent-ready/bots/googlebot
---

Google · Search

# What is Googlebot?

Google's main crawler for Google Search. It follows robots.txt, and Google publishes its IP ranges.

- **robots.txt token**: `Googlebot`
- **Operator**: Google
- **Purpose**: Search
- **Obeys robots.txt**: The operator says it follows robots.txt.
- **Documented by operator**: Yes, on the operator's own pages.
- **Published IP list**: [developers.google.com/static/search/apis/ipranges/googlebot.json](https://developers.google.com/static/search/apis/ipranges/googlebot.json)
- **Documentation**: [developers.google.com](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers)

To refuse Gemini training without leaving Search, disallow Google-Extended instead of Googlebot.

## Block Googlebot from your whole site

`robots.txt`

```
User-agent: Googlebot
Disallow: /
```

## Allow it, but keep private paths out

`robots.txt`

```
User-agent: Googlebot
Allow: /
Disallow: /admin/
```

## Other bots from Google

[Google-Extended](https://dotsagent.io/agent-ready/bots/google-extended)

[Google-CloudVertexBot](https://dotsagent.io/agent-ready/bots/google-cloudvertexbot)

[GoogleOther](https://dotsagent.io/agent-ready/bots/googleother)

[Google-Agent](https://dotsagent.io/agent-ready/bots/google-agent)

[Google-GeminiNotebook](https://dotsagent.io/agent-ready/bots/google-gemininotebook)

[Gemini Spark](https://dotsagent.io/agent-ready/bots/gemini-spark)

[Gemini-Deep-Research](https://dotsagent.io/agent-ready/bots/gemini-deep-research)

[GoogleAgent-Mariner](https://dotsagent.io/agent-ready/bots/googleagent-mariner)

[Add it to a robots.txt in the generator →](https://dotsagent.io/agent-ready/robots-txt)

## Sources

1. [developers.google.com](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers)/search/docs/crawling-indexing/google-common-crawlers

Independent reference for people who build AI agents. Not affiliated with any vendor named here.

© 2026 DotsAgent · Facts checked October 1, 2026