What is meta-externalagent?

Meta's crawler for training AI models and for indexing. It follows robots.txt.

robots.txt token

meta-externalagent

Operator

Meta

Purpose

Training

Obeys robots.txt

●Yes The operator says it follows robots.txt.

Documented by operator

Yes, on the operator's own pages.

Documentation

developers.facebook.com

One token covers both uses, so blocking it for training also stops the indexing, and Meta publishes no IP ranges.

Block meta-externalagent from your whole site

User-agent: meta-externalagent
Disallow: /

Allow it, but keep private paths out

User-agent: meta-externalagent
Allow: /
Disallow: /admin/

Other bots from Meta

Add it to a robots.txt in the generator →