Agents act: they call tools, touch data, execute actions. That demands controls a passive chatbot never needed. A practical guide grounded in Google's Secure AI Framework.
Yohan Consani · Published
For years, the security risk of an AI assistant fit into one simple question: what can it say? A traditional chatbot only returns text, and the worst case was usually a wrong, biased, or leaked answer. With the arrival of agents, the question changes in kind, because an agent acts. It calls APIs, writes to knowledge bases, triggers workflows, reads sensitive data, and drives other systems. The risk shifts from what it says to what it executes.
This is exactly the shift that Google's Secure AI Framework (SAIF) helps organize. SAIF is an open conceptual framework for securing AI systems end to end: from data to model, from infrastructure to application. Its latest version, SAIF 2.0, takes an explicit step into the new terrain: securing AI agents. This article is a practical synthesis of that material, translated for anyone who needs to put agents into production responsibly.
Rogue Actions are unintended actions executed by a model-based agent, whether accidental or malicious.Google · Secure AI Framework (SAIF)
Why an agent is different from a chatbot
The difference is simple to state and deep in its consequences: a chatbot predicts text; an agent takes actions in the world. That ability to act transforms the attack surface. SAIF names this agent-specific risk Rogue Actions (RA), unintended actions executed by the agent, accidental or malicious. What's now at stake is a transaction sent, a record deleted, an email fired off, an access granted.
The side effect is that classic AI risks, once manageable, gain a much sharper edge once an agent can execute:
- Prompt Injection (PIJ): before, it could at most force an unwanted answer. With an agent, an instruction hidden in a document or web page can hijack real actions: making the agent call the wrong tool with the attacker's arguments.
- Insecure Model Output: the model's output is no longer just text on a screen; it feeds systems that execute it (a query, a command, a function call). Unvalidated content becomes action.
- Sensitive Data Disclosure: it now includes data leaked through querying of the model or agent itself, extracted via clever prompting, not only through a conventional access flaw.
The SAIF map: four areas
SAIF organizes the AI system into four areas that must be secured together. A chain is only as strong as its weakest link. Thinking through each one avoids the common mistake of hardening the model and forgetting everything around it.
- Data: the sources that feed and train the system, their provenance and handling.
- Infrastructure: the training, serving, and storage platform the system runs on.
- Model: the model itself, its usage, its inputs and outputs.
- Application: the layer that exposes the system, including, in SAIF 2.0, the agent itself.
The spine: the three agent controls
If there's one idea to take away, it's this: safe autonomy is bounded autonomy. SAIF 2.0 distills it into three controls that, together, gate every consequential action an agent takes. They are complementary: none is enough on its own.
1. Agent User Control
The principle is human-in-the-loop: ensure user approval for any actions performed by agents or plugins that alter user data or act on the user's behalf. Instead of the agent writing directly, it proposes, and the person confirms. Low-risk actions can flow; consequential actions pass through an explicit approval gate.
2. Agent Permissions
Here the principle of least privilege applies, but with an agentic nuance: tool access should be contextual and dynamic. The agent should see only the tools and data strictly necessary for the task at hand, not a fixed, broad set of powers held “just in case.” The smaller the blast radius granted, the smaller the damage a rogue action can do.
3. Agent Observability
Limiting is useless if no one can see what happened. An agent's actions, tool use, and reasoning need to be transparent and auditable through logging. Observability is what turns an incident into a possible investigation, and what lets you tell an honest mistake apart from an attack.
Governance: what holds the controls up
Technical controls only hold up inside governance. SAIF points to Risk Governance (to inventory, measure, and monitor residual risk, what's left after the mitigations) and Product Governance, which embeds security decisions into the lifecycle. At the edges, input and output validation and sanitization remain basic hygiene: nothing entering or leaving the agent should be treated as trusted by default.
Where Charla fits in
Charla claims no certification here: what we describe is engineering alignment with an industry framework. Still, it's worth mapping the platform's real mechanisms to SAIF's three agent controls, because the architecture was designed in the same direction: autonomy with brakes.
- Agent User Control → Charla takes a propose-then-approve posture: changes to knowledge, wiki, or branding land as reviewable proposals, never as silent writes; human approval acts as a gate on retrieval.
- Agent Permissions → Per-assistant isolation (each Charla has its own knowledge base and access list, with retrieval filtered per assistant before ranking), hierarchical RBAC with scoped permissions, deny-by-default tool allowlists, sealed MCP credentials (AES-256-GCM under a Secret Manager key), and egress guarding that blocks requests to internal networks (SSRF protection).
- Agent Observability → An append-only audit log (the database role cannot UPDATE or DELETE it) recording every privileged action and every tool call, plus structured logs, traces, and metrics correlated by request id. Streaming never exposes raw chain-of-thought.
This design is neither exhaustive nor a compliance seal: it's a deliberate architectural choice. Each of these pieces exists so that a consequential agent action only happens within bounds: with permission, with the proper human sign-off, and leaving an auditable trail.
Securing an agentic AI, in the end, is about giving it bounded autonomy. Every action that matters passes through three questions: does it have permission? Was it approved by whoever should approve it? Was it logged? SAIF supplies the vocabulary; engineering supplies the brakes. And it's at that meeting of framework and practice that an agent stops being a risk and becomes a tool you can trust.