Bounded autonomy: loops with stop criteria

A "run until the tests pass" loop sounds safe because it has a stop condition. It only has a safe one if there's also a limit on how long it's allowed to try before you decide it isn't converging. Without that, an agent stuck near a solution will keep iterating — burning time and money — until something external cuts it off, and if nothing does, it just keeps going.

One exit is not a stop criterion

Anthropic's own guidance for building agents is direct about this: agentic systems should include "stopping conditions (such as a maximum number of iterations) to maintain control," because autonomy without a cap risks "higher costs, and the potential for compounding errors" (Building Effective Agents). "Done" is the exit everyone designs for. The other two in the figure — running out of budget and running out of iterations — are the ones that actually protect you, because they're the ones that fire when the task isn't converging at all.

What the three gates actually check

Each exit in the loop answers a different question. Done checks the task itself: do the tests pass, does the output match the spec. Budget checks the cost of getting there: token spend, dollar spend, wall-clock time — a task that would eventually succeed but at ten times the reasonable cost should still stop. Max iterations checks for a loop that isn't making progress at all — an agent oscillating between two wrong fixes will never trip the done condition and never obviously blow the budget in one step, but it will burn through iteration after iteration going nowhere.

Setting the caps in practice

The Claude Agent SDK exposes iteration limits directly as a run option — max_turns caps the number of agentic turns a run can take, and the SDK's own guidance on unattended agents is blunt about why: "without max_turns and a fail-closed permission posture, an unattended agent can loop or take an action you didn't intend" (Agent SDK reference).

from claude_agent_sdk import ClaudeAgentOptions

options = ClaudeAgentOptions(
    system_prompt="Fix the failing test. Do not delete or skip it.",
    max_turns=8,
)

Budget caps are a separate mechanism — tracking cumulative token or dollar spend across turns and stopping the loop when it crosses a threshold you set, independent of how many turns that took.

Decide what "done" means before the loop starts

The done gate is only trustworthy if it was defined honestly up front — this is the same problem this series covered under definitions of done: "tests pass" can be satisfied by deleting the test. A loop with a strict done condition (a specific test suite, unmodified, must pass) and a generous iteration cap is safer than a loose done condition with no cap at all, because the loose one will find the shortcut long before it finds the fix.

Try it today

1. Find one place in your setup where an agent currently loops on "try again until it works" with no turn limit, and add one.
2. Set a token or cost budget for a single task and have the loop stop and report, not silently keep trying, when it's hit.
3. Write the done condition as something an agent can't satisfy by editing the check itself — protect the test file, not just the loop.
4. When a loop hits max iterations, log what state it stopped in, not just that it failed — that's your debugging signal for next time.
5. Review one recent long-running agent task and estimate what it would have cost if it had run twice as many iterations before you noticed.

Where this breaks

A max-iterations cap that's too low kills tasks that were genuinely about to converge, which trains people to set it too high "just in case," quietly reintroducing the runaway-cost problem. Budget caps require you to actually track spend per task, which many setups don't do by default. And none of the three gates catches an agent that reports "done" while having gamed the done condition — the gates bound the loop, they don't replace verifying what came out of it.

Next: reading the bill — what a single agent task actually costs, and where the money goes.