Skip to content
Rung 04 Evaluation & MeasurementSwitch rung
Rung 04 / How you know it works

Evaluation & Measurement

How you know it works — and whether the measurement can be trusted.

7
Sources
3
Concepts
2
Entities
A brushed steel dial gauge with a copper needle, on paper.

In scope: benchmark design and critique, matched-budget controls, contamination and overfitting to the eval set, agent benchmarks, the eval-to-deployment gap, measurement methodology. Out: the findings themselves, which belong to whichever rung they move.

Three stacked sieves of tightening aperture above the small sample that survives.
BenchmarksMatched budgetContamination
Download Report

Can This Score Be Trusted?

Grader noise, evaluator failure types, compute budgets, harnesses and construct validity: how a benchmark number goes wrong before anyone reads it

QuizIntermediate7m

Checking a Leaderboard Claim

A 6-point win on a tool-calling benchmark, one run, no budget, no harness: work out what it is allowed to mean

Case StudyAdvanced7m