An AI coding agent can write a patch, run a command, read the failure, and try again. That is useful. But the important engineering decision happens between attempts: what evidence tells us the change is actually getting better?
A fluent explanation is easy to mistake for a working result. A green check can also be misleading when the check exercises the wrong behavior. The person developing with AI still needs a clear definition of success and a way to challenge it.
This is a practical workflow proposal, not a benchmark result or a claim that a particular agent has been tested against it.
What makes the loop agentic?
In ordinary chat, the model suggests an answer and waits. In an agentic development loop, it can inspect the project, choose an action, observe the result, and decide what to do next.
Anthropic distinguishes predefined workflows from agents that dynamically choose their process and tools. Its guidance also emphasizes feedback from the environment and explicit stopping conditions. Those ideas are useful even for a small coding task: the next action should respond to something observed, rather than repeat the original guess. Source: Building effective agents.
For development, I would organize the loop like this:
- Define: write the required behavior and its boundaries.
- Inspect: find the relevant code, existing checks, and current failure.
- Change: implement one coherent correction.
- Observe: run the checks that can demonstrate the behavior.
- Judge: compare the evidence with the acceptance criteria.
- Decide: repair, gather missing information, request a decision, or finish.
The loop needs an exit. More iterations do not automatically produce a better product.
Define acceptance before asking for implementation
Consider a fictional bug report: “The CSV export button does not work.”
That leaves several questions unanswered. Does clicking do nothing? Is the file empty? Does it contain the wrong rows? Has the download succeeded but the application still shown an error?
A more useful task contract would say:
Export the currently filtered rows as a CSV. Preserve the visible column order. Escape commas, quotes, and line breaks. When there are no matching rows, export the header row. Show an actionable error if export fails.
Now the agent and reviewer have something concrete to judge. Before editing, reproduce one failing example and record the current behavior. That gives the final check a baseline.
Also name the boundaries: whether the change may add a dependency, alter the server API, change permissions, or modify the data format. An agent should not silently resolve a product decision by expanding the patch.
Use three kinds of judgment
Different checks answer different questions. I would keep these three layers visible in a review:
| Layer | Question | Example evidence |
|---|---|---|
| Executable checks | Does a precise rule hold? | A parser reads the exported CSV and recovers the expected values |
| AI review | What plausible defect or omission deserves investigation? | A reviewer identifies an unhandled export failure and a way to reproduce it |
| Human judgment | Does the result meet the actual need? | The owner confirms the exported fields are useful and appropriate |
A compiler can check types without proving that the correct customer records were exported. An AI reviewer can propose an edge case without proving that it exists. An owner can like the interface while missing a serialization defect. Each layer contributes different evidence.
Anthropic’s evaluation guide describes code-based, model-based, and human graders, including their tradeoffs. It also distinguishes the agent’s recorded activity from the resulting state of the environment. For coding work, that distinction is especially valuable: inspect the artifact the agent produced as well as the story of how it got there. Source: Demystifying evals for AI agents.
Give an AI judge a rubric and evidence
“Review this and give it a score” produces an opinion that is difficult to act on. A better review request supplies the original requirement, the actual diff, relevant project constraints, and the outputs of checks that really ran.
Here is a reusable review prompt:
Review this change against the acceptance criteria below.
Inspect the diff and relevant surrounding code.
For each actionable finding, provide:
- the requirement or invariant at risk;
- the file and location;
- a concrete failure scenario;
- supporting evidence or a proposed reproduction;
- severity and the smallest useful next check.
Distinguish confirmed defects from hypotheses.
State what you could not verify.
Do not treat the author's completion summary as proof.A fresh review context can reduce reliance on the implementer’s narrative, but it does not guarantee independence or correctness. A different model can still share the same mistaken assumption. Keep the original requirements and checks outside the reviewer’s power to quietly redefine success.
Treat a proposed defect as something to investigate. If the reviewer says commas break the CSV, try a value containing a comma and parse the downloaded file. If the case passes, ask what evidence supports the finding before adding unnecessary complexity.
Repair the cause, then recheck the result
For the CSV example, a useful iteration would have this shape:
- Observe that a quoted value comes back incorrectly when the downloaded file is parsed.
- Trace the failure to the serialization code.
- Correct escaping without changing the filtering contract.
- Recheck that value, plus an ordinary row, an empty result, and the active filter.
- Review the final diff for accidental changes.
That is an illustrative sequence, not a report of a test performed for this article.
The same principle matters in BIM automation. A script that parses successfully has passed a syntax check. To judge a Revit change, you may also need to open the interface inside Revit, run it against disposable project files, inspect the resulting model, and verify the saved artifact. The evidence must match the claim.
Related reading: Where Ahmed is more confident than Claude.
Decide when to stop
Use explicit outcomes instead of an endless “improve it” instruction:
| Outcome | When to choose it |
|---|---|
| Accept | Required behavior is demonstrated, relevant checks pass, and material findings are resolved |
| Repair | A reproducible defect has a sufficiently clear cause and a bounded fix |
| Investigate | A finding is plausible, but evidence is missing or contradictory |
| Escalate | The next step needs access, a product decision, or authority the agent lacks |
| Stop and report | The agreed time or attempt budget is exhausted without enough evidence |
For a small bug, a team might initially allow two repair attempts before requiring a diagnosis update. That is an example operating rule, not a universal optimum. Choose a budget that fits the risk and complexity.
Do not let the agent make a failing check green by weakening the requirement. A test can be wrong, but changing it needs a reason grounded in intended behavior. Also stop adding speculative improvements once the requested outcome is satisfied.
Make the handoff inspectable
The final development report should make the decision easy to audit:
- What behavior changed, and why?
- What checks actually ran, against which revision or artifact?
- What passed, failed, or remains unverified?
- What limitations or unresolved findings affect release?
- Is the change local, pushed, merged, deployed, or verified live?
Those last states matter. A successful local build does not prove a production deployment succeeded. A preview does not prove the production branch contains the change. A published page should be checked at its live URL before calling the publishing task complete.
The useful habit is to make each iteration answer one question with evidence. Let the agent do the work, let checks challenge the work, and keep the acceptance decision tied to the result a person actually needs.
References and online resources
- Anthropic: Building effective agents — agent and workflow patterns, environmental feedback, and stopping conditions.
- Anthropic: Demystifying evals for AI agents — grader choices and evaluating outcomes as well as execution traces.
— Ahmed
