Explained · Engineering
AI agents
A chatbot answers you; an agent acts for you. Give it a goal and it runs a loop — pick a step, use a tool like search or code, check the result, adjust, repeat until the job is done. The bet is delegation: you review finished work instead of doing the work yourself.
Where it breaksErrors compound. A step that is right nine times in ten is right barely a third of the time over ten steps, and an agent that took a wrong turn keeps going confidently rather than stopping. The practical failures are loops that never terminate, tool calls made on stale assumptions, and a bill that quietly multiplies because every step is a fresh model call carrying the whole history.
500 tasks size of SWE-bench Verified, the human-validated set of real GitHub issues agents are graded onOpenAI, "Introducing SWE-bench Verified" · 2024-08-13
The loop is the whole idea
An agent is not a smarter model. It is the same model placed inside a loop that a piece of ordinary software runs. The loop has four beats: the model is shown the goal and everything that has happened so far, it names one next action, the surrounding program actually performs that action — runs a search, executes code, calls an API — and the result is pasted back into the conversation. Then the loop runs again with a slightly longer history. That is it. The model never touches the world directly; it emits a request, and the program around it, called the harness, decides whether and how to carry that request out. Most of what people call "agent quality" is the harness: which tools exist, what the results look like when they come back, and when the loop is allowed to stop.
Why anyone bothers
A single answer has to be right in one shot. A loop gets to be wrong and then find out. When an agent runs the test suite and sees it fail, that failure is new information the model did not have a moment earlier — so the second attempt is made by a better-informed system, even though the model itself has not changed at all. This is why agents beat plain prompting on tasks with a checkable result: writing code, filling a spreadsheet, reconciling records. Anything where the world will tell you plainly that you got it wrong is a good fit. Anything graded on taste, where nothing pushes back, is a bad fit — there the loop simply produces more confident drafts of the same misjudgement.
Where it breaks
Reliability multiplies rather than averages. Ten steps at ninety per cent each land you around thirty-five per cent overall, so the honest question about an agent is never "is the model good" but "how many steps, and what checks the work". The second failure is direction: nothing in the loop naturally notices that the goal has been misread, so the agent pursues the wrong objective diligently for twenty steps. The third is cost and blast radius. Every step resends a growing history, so a long run can cost many times a single answer; and a loop with real permissions — sending mail, changing records, spending money — can act on a wrong conclusion faster than a person can read it. The fix is boring engineering: fewer steps, a hard step limit, a verifiable stopping condition, and a human on anything irreversible.
How to judge one
Ignore the demo and ask three questions. What is the checkable finish line — how does the system itself know it is done, without a person deciding? What does one completed job cost end to end, counting the failed attempts, not the happy path? And what can it do that you would not let a new hire do unsupervised in week one? Agents earn their keep on repetitive work with a machine-readable verdict and a cheap cost of being wrong. They are a poor bet on one-off judgement calls with expensive mistakes, however impressive the transcript looks.