Meta · User fetches
What is meta-externalfetcher?
Fetches pages on a user's request, including for Meta's agent features.
- robots.txt token
meta-externalfetcher- Operator
- Meta
- Purpose
- User fetches
- Obeys robots.txt
- No The operator says robots.txt may not apply, or community reports say it ignores it.
- Documented by operator
- Yes, on the operator's own pages.
- Documentation
- developers.facebook.com
Meta says it may bypass robots.txt and publishes no IP ranges, which leaves little to block it on.
Block meta-externalfetcher from your whole site
User-agent: meta-externalfetcher
Disallow: /
Allow it, but keep private paths out
User-agent: meta-externalfetcher
Allow: /
Disallow: /admin/
This bot fetches pages for a person, or for an agent acting on one's behalf. Several operators say robots.txt may not apply to such requests, so check the line above and enforce at your firewall if it matters.
Other bots from Meta
Add it to a robots.txt in the generator →