Guardrails for LLM agents: levels, limits and an off switch
Prompt-level guardrails ask the model to behave. Runtime guardrails make it impossible to misbehave past a point. Agents need the second kind.
Two kinds of guardrail
Inside the model: system prompts, output filters, classifiers. Useful, and bypassable, because they depend on the model following text.
Outside the model: controls on the tools the agent can reach. These work whatever the model was told, because the tool call is checked by something that does not read the conversation.
The runtime guardrails worth having
- Permission levels: a read-only level, a default level that asks before anything risky, and a level for agents you have watched for a while.
- Per-tool allow-lists: an agent sees only the tools it needs.
- A daily limit: past it, what would have run on its own waits for a person.
- A bulk limit: an action over a set number of items is held.
- A flood guard: an agent that queues many held requests in minutes is paused, not obeyed.
- Secret detection: content that looks like a key, a password or a card number does not leave on its own.
- An off switch per agent and per account.
Test your guardrails
Try to break them. Ask the agent to send a secret, to act for a stranger, to repeat an action a hundred times, to do something irreversible. Record what was held, what was refused, and what got through. Before changing an agent's level, look at what the change would have meant for its recent requests.
In Operon OS Agenaut
Agenaut has three levels: Observer reads only; Assistant reads and writes and asks before anything else; Trusted may also send again to people you approved before. You can set a daily limit per agent, and Agenaut pauses an agent that asks for too much too quickly. Stop is one tap: every call from that agent is refused from then on. Before you change a level, Agenaut shows what the agent's recent requests would have met under each one.
Try it. Operon OS Agenaut is in beta. Request access, or read the docs to see how an agent is connected.
Related guides
- Keeping autonomous AI agents under control
- Enterprise AI agents: approvals, roles and the two-person rule
- Add an approval step to Claude, ChatGPT and Cursor
Terms used here are explained in the glossary.