Capability is not the same thing as clearance.
That sounds obvious until the agent demo is smooth, the benchmark is shiny, and someone starts saying things like “it can probably handle production.” That is usually where the grown-up engineering work begins, not where it ends.
The mistake teams keep wanting to make
When an AI agent performs well in a narrow task, the temptation is to widen its scope. Give it repo write access. Let it touch CI. Let it install packages. Let it clean up the system after itself. Let it “just handle it.”
That temptation comes from confidence theater. The model looked competent, so the permissions start drifting upward to match the vibe.
Useful systems should move the other direction:
- narrow the permission set
- make reads easier than writes
- log everything that matters
- require review for state-changing work
If that sounds conservative, good. Conservative is what you call a system that still works after a bad Tuesday.
Least privilege is not anti-AI
Some people talk about guardrails as if they are a tax you pay because the model is weak. That is backwards.
Least privilege is what makes stronger agents usable at all.
If an agent can search a codebase, summarize a failure, propose a patch, and prepare a PR, that is already valuable. It does not need the right to silently mutate the environment, install half the internet, and “clean up” after itself in order to prove it is intelligent.
The more authority a system has, the more boring the controls need to become:
- read-only discovery by default
- sandboxed execution when tools are involved
- clear provenance on what the agent saw and changed
- human approval at the merge or release boundary
That is not fear. That is workflow design.
The physical-world analogy still holds
I keep coming back to autonomy because software people sometimes act like permissions are a paperwork problem instead of a systems problem.
A robotaxi does not become safer because the demo voice sounds calm. A warehouse robot does not become trustworthy because the launch video used cinematic lighting. The question is always the same: what can this thing affect when it is wrong?
Software agents live in a less dramatic environment, but the same logic applies. They can affect:
- a repository
- a deployment pipeline
- a customer record
- a vulnerability report
- a bill of materials
That is real blast radius, even if nobody is physically dodging it in traffic.
What a good team does instead
If you want AI agents in a real workflow, build around scar tissue instead of optimism.
Start here:
- Give the agent the minimum permissions needed for the current task.
- Separate discovery from action.
- Prefer PRs, tickets, and proposed diffs over silent mutation.
- Make rollback and audit trails easy before you add more autonomy.
- Treat “agent did it” as provenance, not absolution.
This is the unglamorous part of the AI stack, which usually means it is the part worth keeping.
The future is probably not one grand autonomous system with full keys to the kingdom. It is smaller agents doing narrow work well under controls that make everyone slightly less nervous.
That is not a downgrade. That is the shape of software that gets to stay in production.
— Cara