Build 04 · Blocked
Predict the VRAM before renting the GPU
Given model, context length and batch size, predict memory use. Verify against a real run. The residual is the interesting part.
Blocked
State
Off route
Route step
0
Notes
Done when
A state a stranger could checkIt predicts a real run within 10%, and you can name what the remaining error is made of.
Why it is harder — First build shipped as a tool a stranger uses — an L4 apparatus, not an explanation.
Where it sits
After Build 01 · Why does this laptop generate at 20.9 tokens per second? — KV-cache arithmetic IS roofline arithmetic. Taken first it is a formula to copy.
What it reads
Just enough to unblock the next step- Inference Engineering ch3 · ch6The KV-cache arithmetic only. Not the book.
- Efficiency in LLMs The KV-cache compression section, for what the levers cost.
What it ships
- BuildVRAM / KV-cache calculator, verified against a real run
- Bottleops:byte and KV-cache formulas, spaced
Teach it back
If it cannot be explained plainly, it was not learned- 1Explain itExplain what you built to a friend who has never seen it. Plain words, no jargon, five sentences at most.
- 2Predicted vs realWhat you expected to happen, what actually happened, and why the two differ.
- 3Where it breaksThe question about it you still cannot answer, and the point where your explanation stops working.
- 4KeepThe one number, rule or picture worth keeping. It becomes a bottle and gets re-tested.
No notes on this build yet.
How a note lands here
A note is a Learning Log entry with the tag study-kv-calculator. It is written in Connor's words, filed as a draft, and appears here once it is published from the admin queue. One note can carry several build tags.