Build 03 · Blocked
What a marginal dollar of test-time compute actually buys
Run one benchmark at increasing inference-time compute. Plot accuracy against cost. Find the knee and explain why it is where it is.
Time — about a week
Done when
A state a stranger could checkThe curve exists with your own measurements, the knee is identified, and you can state what changes its position.
Why it is harder — First build that produces a number with commercial consequences — this is the token-price to task-price bridge.
Where it sits
After Build 02 · An evaluation you would bet on — The curve is an accuracy plot. Without a trusted eval it is a plot of noise.
Shipping it opens 05 A refinement loop, and the point where it starts drifting
What it reads
Just enough to unblock the next step- Build a Reasoning Model From Scratch ch4Inference-time scaling.
What it ships
- NotebookWhat test-time compute costs, measured rather than cited
- BuildThe cost curve
Teach it back
If it cannot be explained plainly, it was not learned- 1Explain itExplain what you built to a friend who has never seen it. Plain words, no jargon, five sentences at most.
- 2Predicted vs realWhat you expected to happen, what actually happened, and why the two differ.
- 3Where it breaksThe question about it you still cannot answer, and the point where your explanation stops working.
- 4KeepThe one number, rule or picture worth keeping. It becomes a bottle and gets re-tested.
No notes on this build yet.
A note is a Learning Log entry with the tag study-test-time-curve. It is written in Connor's words, filed as a draft, and appears here once it is published from the admin queue. One note can carry several build tags.