Google · Agents

What is Google-Agent?

Fetches pages for agents running on Google infrastructure when a user asks them to act. Google says user-triggered fetchers generally ignore robots.txt.

robots.txt token: Google-Agent
Operator: Google
Purpose: Agents
Obeys robots.txt: No The operator says robots.txt may not apply, or community reports say it ignores it.
Documented by operator: Yes, on the operator's own pages.
Published IP list: developers.google.com/static/crawling/ipranges/user-triggered-agents.json
Documentation: developers.google.com

Google is experimenting with a Web Bot Auth identity, https://agent.bot.goog, for these requests.

Block Google-Agent from your whole site

robots.txt

User-agent: Google-Agent
Disallow: /

Allow it, but keep private paths out

robots.txt

User-agent: Google-Agent
Allow: /
Disallow: /admin/

This bot fetches pages for a person, or for an agent acting on one's behalf. Several operators say robots.txt may not apply to such requests, so check the line above and enforce at your firewall if it matters.

Other bots from Google

Googlebot Google-Extended Google-CloudVertexBot GoogleOther Google-GeminiNotebook Gemini Spark Gemini-Deep-Research GoogleAgent-Mariner

Add it to a robots.txt in the generator →

Sources

developers.google.com/crawling/docs/crawlers-fetchers/google-agent
developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers