Meta · Search
What is meta-webindexer?
Meta's crawler for search in Meta AI. It follows robots.txt; Meta publishes no IP ranges for it.
- robots.txt token
meta-webindexer- Operator
- Meta
- Purpose
- Search
- Obeys robots.txt
- Yes The operator says it follows robots.txt.
- Documented by operator
- Yes, on the operator's own pages.
- Documentation
- developers.facebook.com
Block meta-webindexer from your whole site
User-agent: meta-webindexer
Disallow: /
Allow it, but keep private paths out
User-agent: meta-webindexer
Allow: /
Disallow: /admin/
Other bots from Meta
Add it to a robots.txt in the generator →