---
title: "Claude-SearchBot: block or allow this AI crawler · DotsAgent"
description: "Claude-SearchBot is run by Anthropic and listed under search. Its robots.txt token, whether it follows the rules, its IP list and how to control it."
url: https://dotsagent.io/agent-ready/bots/claude-searchbot
---

Anthropic · Search

# What is Claude-SearchBot?

Anthropic's crawler for search in Claude. It follows robots.txt and has its own token, separate from ClaudeBot.

- **robots.txt token**: `Claude-SearchBot`
- **Operator**: Anthropic
- **Purpose**: Search
- **Obeys robots.txt**: The operator says it follows robots.txt.
- **Documented by operator**: Yes, on the operator's own pages.
- **Published IP list**: [claude.com/crawling/bots.json](https://claude.com/crawling/bots.json)
- **Documentation**: [support.claude.com](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)

## Block Claude-SearchBot from your whole site

`robots.txt`

```
User-agent: Claude-SearchBot
Disallow: /
```

## Allow it, but keep private paths out

`robots.txt`

```
User-agent: Claude-SearchBot
Allow: /
Disallow: /admin/
```

## Other bots from Anthropic

[ClaudeBot](https://dotsagent.io/agent-ready/bots/claudebot)

[Claude-User](https://dotsagent.io/agent-ready/bots/claude-user)

[Add it to a robots.txt in the generator →](https://dotsagent.io/agent-ready/robots-txt)

## Sources

1. [support.claude.com](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-ca

Independent reference for people who build AI agents. Not affiliated with any vendor named here.

© 2026 DotsAgent · Facts checked October 1, 2026