Skip to content
Sign in
The path

Build 12 · Open

Build an agent harness that fixes real code

CMU 11-768 Assignment 1: write the ReAct loop, get the CodeAgent to repair the chess-app, add context compaction, and give the ChessAgent its tools. Watch the public lectures first in one fast pass, writing down every gap, then let the gap list drive the build.

You are here
State
1 / 6
Route step
0
Notes

Time — every public lecture in one fast pass, then about a weekCost — Parts 1–3 are billable: a Modal sandbox plus LLM API credits on your own keys. `make test` is free and offline.

Done when

A state a stranger could check

The public test suite passes on your code (`make test`), the CodeAgent fixes at least one chess-app failure in the sandbox, and you can draw the loop and the compaction step from memory.

Why it is harder — First build where the thing you made acts on its own. The bugs stop being in your code and start being in what the model does with it.

What it reads

Just enough to unblock the next step
  • AI Agents (11-768) L1 · L2 · L3 · L4Every public lecture, once, at speed. Its purpose is the gap list, not comprehension. Then only the lecture a stuck test points at.
  • The Modern Software Developer (CS146S) W1b · A1The 9/24 transcript and Assignment 1 only — dissect a production harness’s system prompt and tool schemas from a captured trace before writing your own loop. The capture is the design reference, not a second course.
  • Modern AI Systems: Agents and System Optimizations (CS2680) A2Mads-Loop is the same build from another angle: a harness on SWE-bench Pro, with a spec that tells you not to trust its own starter design. Use its malformed-output and broken-tool cases as extra tests for your loop, not as a second harness.

What it ships

  • BuildAn agent harness that passes the public suite and repairs code
  • NotebookWhat the harness does that the model cannot, in plain words
  • BottleThe ReAct loop and compaction, spaced
HarnessesEvals

Teach it back

If it cannot be explained plainly, it was not learned
  1. 1Explain itExplain what you built to a friend who has never seen it. Plain words, no jargon, five sentences at most.
  2. 2Predicted vs realWhat you expected to happen, what actually happened, and why the two differ.
  3. 3Where it breaksThe question about it you still cannot answer, and the point where your explanation stops working.
  4. 4KeepThe one number, rule or picture worth keeping. It becomes a bottle and gets re-tested.

Notes

From the Learning LogThe notebook →

No notes on this build yet.

How a note lands here

A note is a Learning Log entry with the tag study-agent-harness. It is written in Connor's words, filed as a draft, and appears here once it is published from the admin queue. One note can carry several build tags.