---
title: "meta-externalfetcher: block or allow this AI crawler · DotsAgent"
description: "meta-externalfetcher is run by Meta and listed under user fetches. Its robots.txt token, whether it follows the rules, its IP list and how to control it."
url: https://dotsagent.io/agent-ready/bots/meta-externalfetcher
---

Meta · User fetches

# What is meta-externalfetcher?

Fetches pages on a user's request, including for Meta's agent features.

- **robots.txt token**: `meta-externalfetcher`
- **Operator**: Meta
- **Purpose**: User fetches
- **Obeys robots.txt**: The operator says robots.txt may not apply, or community reports say it ignores it.
- **Documented by operator**: Yes, on the operator's own pages.
- **Documentation**: [developers.facebook.com](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/)

Meta says it may bypass robots.txt and publishes no IP ranges, which leaves little to block it on.

## Block meta-externalfetcher from your whole site

`robots.txt`

```
User-agent: meta-externalfetcher
Disallow: /
```

## Allow it, but keep private paths out

`robots.txt`

```
User-agent: meta-externalfetcher
Allow: /
Disallow: /admin/
```

This bot fetches pages for a person, or for an agent acting on one's behalf. Several operators say robots.txt may not apply to such requests, so check the line above and enforce at your firewall if it matters.

## Other bots from Meta

[meta-externalagent](https://dotsagent.io/agent-ready/bots/meta-externalagent)

[meta-webindexer](https://dotsagent.io/agent-ready/bots/meta-webindexer)

[meta-externalads](https://dotsagent.io/agent-ready/bots/meta-externalads)

[facebookexternalhit](https://dotsagent.io/agent-ready/bots/facebookexternalhit)

[Muse](https://dotsagent.io/agent-ready/bots/muse)

[Add it to a robots.txt in the generator →](https://dotsagent.io/agent-ready/robots-txt)

## Sources

1. [developers.facebook.com](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/)/docs/sharing/webmasters/web-crawlers/

Independent reference for people who build AI agents. Not affiliated with any vendor named here.

© 2026 DotsAgent · Facts checked October 1, 2026