Anthropic · User fetches
What is Claude-User?
Fetches a page when a Claude user asks for it. Unlike most user-triggered fetchers, it follows robots.txt; Anthropic says all its bots do.
- robots.txt token
Claude-User- Operator
- Anthropic
- Purpose
- User fetches
- Obeys robots.txt
- Yes The operator says it follows robots.txt.
- Documented by operator
- Yes, on the operator's own pages.
- Published IP list
- claude.com/crawling/bots.json
- Documentation
- support.claude.com
Block Claude-User from your whole site
User-agent: Claude-User
Disallow: /
Allow it, but keep private paths out
User-agent: Claude-User
Allow: /
Disallow: /admin/
This bot fetches pages for a person, or for an agent acting on one's behalf. Several operators say robots.txt may not apply to such requests, so check the line above and enforce at your firewall if it matters.
Other bots from Anthropic
Add it to a robots.txt in the generator →