Designing Agents With Bounded Authority

An agent is a loop. A model is given a goal and a set of tools, it decides which tool to call, the result is fed back, and it decides again, until it concludes or something stops it. That is the entire idea, and its simplicity is why the first working version takes an afternoon and why the conversation about it goes wrong so consistently.

The wrong conversation is about the model. Which one reasons best, which handles long chains, which follows tool schemas most reliably. These matter, they change every few months, and they are not the design. The design is the boundary: the precise specification of what this system may do without a human in the loop, what it must escalate, what happens when it is halfway through a sequence and something fails, and what record exists afterwards.

Put it in familiar terms. You are granting privileges to a process whose behaviour you cannot fully predict, which will be handed inputs chosen by other people, and which is optimising for appearing to complete its instructions. That is not a novel security posture — it is the one you already take towards a service account, and it has an established answer: least privilege, explicit authorisation, bounded effects, a complete audit trail. What is new is that the process can be argued with by anything it reads.

Classify actions by what they cost to undo

The first artefact is not code. It is a list of every action the agent can take, classified by reversibility and blast radius. Write it down, get it agreed by whoever owns the risk, and treat it as the specification.

Class	Characteristics	Control
Read	No state change, non-sensitive data	Autonomous, logged
Read sensitive	No state change, personal or regulated data	Autonomous within scope, logged, access-controlled
Reversible write	Internal state, cheap to undo, no external visibility	Autonomous with limits, logged, undo path tested
Externally visible	Sends a message, posts, notifies a third party	Human approval, or a delay window with a cancel
Irreversible	Payment, deletion, legal commitment, physical action	Human approval, always

The line that matters is not risky versus safe. It is undoable versus not. An action you can reverse in one command has a recovery path. An action visible to a customer cannot be recovered even when the underlying record is corrected, because the customer has already read the email.

Two properties get underestimated here. Volume converts a safe action into a dangerous one: one refund is reversible, four hundred refunds in a loop is an incident. And aggregation converts safe reads into a breach: an agent permitted to read any single customer record is, unbounded, permitted to read all of them.

Permission the tools, not the prompt

The most common architectural error is to express authority in the prompt. The instructions say the agent must not delete anything, must only act on the current user's records, must always ask before sending. Those instructions will hold most of the time, which is exactly what makes them dangerous, and they will fail under adversarial input, unusual phrasing, or a long context in which the constraint has drifted far from the current turn.

Prompt text is a hint about desired behaviour. It is not a control. Every boundary that matters must be enforced outside the model, in code that the model cannot influence.

Scope credentials to the request. The agent acts with the permissions of the user it is acting for, not with a service account that can see everything. Pass a short-lived, narrowly scoped token through to the tool layer and let the underlying system enforce authorisation as it would for any other caller. If the agent asks for a record the user cannot see, the answer is a denial from the data layer, not a refusal from the model.

Validate arguments at the tool boundary. Every tool call is untrusted input to your system. Validate against a schema, check ranges, verify the referenced entities exist and belong to the user in scope, and reject anything outside the declared contract. Treat a malformed call as a bug report, log it, and count it.

Enforce rate and volume limits per session. Maximum tool calls per run, maximum records touched, maximum monetary value, maximum identical calls. These are the blast-radius limits and they are cheap. The loop that calls the same tool two hundred times is a real and frequent failure, and it is bounded by a counter, not by better instructions.

Give each tool the narrowest possible surface. A tool that executes arbitrary queries is not a tool, it is a shell. Prefer a small set of specific operations with typed parameters over one general capability, because you can reason about the former and you cannot about the latter.

Make the tool list dynamic. An agent handling a read-only enquiry does not need the refund tool in its schema at all. Removing a capability from the available set is a stronger control than declining to use it.

Prompt injection is not hypothetical

The moment an agent reads content it did not author — a web page, an email, a document, a support ticket, a code comment, the output of another system — it is processing instructions from whoever wrote that content. Models do not reliably distinguish data from instruction, and the mitigations available are partial.

Be clear about the threat model. An attacker who can get text in front of your agent can attempt to make it call tools with the agent's authority. If the agent can read the user's mailbox and can also send messages, injected text in an incoming email can attempt to exfiltrate the mailbox. The dangerous combination is always the same: access to untrusted content, access to sensitive data, and an outbound channel. Any two are manageable; all three together is where the serious failures live.

There is no prompt that reliably solves this. What helps is architecture.

Separate trust levels structurally. Mark untrusted content explicitly in the context and never grant it the standing of instruction. This raises the bar; it does not close the hole.

Break the exfiltration path. If an agent processes untrusted content, restrict its outbound capabilities in that same run. An agent that reads the web should not, in that run, also be able to send arbitrary messages or make arbitrary requests to attacker-chosen addresses. Allowlist outbound destinations. Strip or refuse rendering of markup that fetches remote resources, because an image URL is an exfiltration channel.

Require human confirmation on the crossing. Any action that moves data from a sensitive context to an external destination gets a human in the loop, with the actual content and destination shown. This is a small amount of friction on a small number of actions.

Label provenance per source. Track which tool results came from where, and carry that into decisions. When an incident happens you will want to know which document the instruction came from.

Monitor for the shape of an attack. Sudden changes in tool-call patterns, calls to tools irrelevant to the stated task, or repeated attempts at a denied action are detectable. Alert on them.

Multi-step actions fail halfway

Distributed systems engineering did not stop being relevant because a model is choosing the steps. It became more relevant, because the sequence is now decided at runtime by a component that does not know what already succeeded.

Idempotency is mandatory, not a nicety. Every mutating tool call carries a client-generated key, and the same key replays the original result rather than acting twice. Without it, any retry — and agents retry constantly, on timeouts, on parse failures, on their own initiative when a result looks wrong — risks duplicating the effect. The duplicated refund is the canonical example and it is a real one.

Decide the compensation strategy before you need it. An agent three steps into five when step four fails has left partial state. Either the sequence is a saga with a defined compensating action per step, or partial completion is explicitly acceptable and visible, or the whole thing is staged and committed at the end. The default — no strategy — resolves to inconsistent state and no record of intent.

Persist the loop state externally. The run's plan, completed steps, and pending steps live in a durable store, not only in the conversation context. This makes the run resumable, inspectable while it is happening, and killable. An agent whose state exists only as accumulated context cannot be recovered when the process dies.

Bound the loop absolutely. Maximum iterations, maximum wall-clock time, maximum spend. When a limit is reached the run stops and escalates. It does not retry with a larger budget. This is also the main defence against the cost failure described in inference cost as an architectural constraint.

Make cancellation real. A human must be able to stop a run in flight, and the stop must actually prevent pending side effects rather than merely closing the window.

The audit trail is a design requirement

When something goes wrong — and something will — you need to reconstruct exactly what happened, why the agent believed it was the right action, and who was accountable. Retrofitting this is painful and usually incomplete.

Record, for every run: a run identifier, the initiating user and the authority under which the agent acted, the goal as stated, and then per step the model and configuration used, the prompt as actually sent, the tool called with its arguments, the result returned, the source and trust level of any content that entered the context, and the timestamp. Record the outcome, the stop reason, and every approval with the identity of the approver and what they were shown at the moment they approved.

That last detail is the one that gets missed and the one that matters in a dispute. "The user approved the action" is weak. "The user was shown this exact summary and approved at this timestamp" is a record. In regulated environments that distinction separates a defensible process from a finding.

Make the trail queryable and readable by non-engineers, and retain it under the same policy as the data it touches.

Widen the boundary with evidence

The pragmatic path is to start with authority narrower than you think necessary and widen it on evidence. That evidence is the agent's own record.

Stage one: the agent proposes and a human executes. Slow, and it generates the dataset you need — what the agent wanted to do, and whether a human agreed.

Stage two: the agent executes, a human approves first. Track the approval rate per action type. Types approved almost every time are candidates for promotion; types frequently rejected are telling you something specific about where the design is wrong.

Stage three: autonomous within limits, with sampled review, anomaly alerting and a kill switch. Promote one action class at a time, and keep the ability to demote a class back to approval without a deployment.

This is progressive delivery applied to authority rather than traffic, with the same advantage: the decision to widen is made against observed behaviour rather than a belief about how well the model reasons.

What to do on Monday

List every tool your agent can call and classify each by reversibility and blast radius using the table above. Do it with the person who owns the risk in the room. Most teams discover at least one tool in the irreversible column that is currently running unsupervised.

Then check whether every boundary you believe you have is enforced in code. Search the system prompt for the words never, always, only and must. Each one is a control you think you have. For each, find the code that enforces it. The ones with no corresponding code are the gaps, and there will be more than you expect.

Add three things this week if they are absent: an idempotency key on every mutating tool call, a hard iteration and spend cap per run, and a durable audit record of prompt, tool call, arguments and result for every step. None is difficult; all three are unpleasant to add after an incident.

Finally, map the dangerous combination. Write down whether any single run can read untrusted content, access sensitive data, and reach an external destination. If one can, break that triangle before you do anything else on the roadmap.