Build 07 · Open
Attention, implemented, then made fast
Write attention from nothing. Get it correct, then profile it and find where the time actually goes.
Open
State
Off route
Route step
0
Notes
Done when
A state a stranger could checkYour implementation matches a reference output, and you can state which line dominates the runtime and why.
Why it is harder — First build where correctness is not enough — it also has to be measured.
Where it sits
Shipping it opens 08 FlashAttention, by hand · 09 What each collective operation actually costs
What it reads
Just enough to unblock the next step- GPU Kernels / MLSys Part I only — the execution model, enough to read a profile.
What it ships
- BuildAttention from scratch, profiled
Teach it back
If it cannot be explained plainly, it was not learned- 1Explain itExplain what you built to a friend who has never seen it. Plain words, no jargon, five sentences at most.
- 2Predicted vs realWhat you expected to happen, what actually happened, and why the two differ.
- 3Where it breaksThe question about it you still cannot answer, and the point where your explanation stops working.
- 4KeepThe one number, rule or picture worth keeping. It becomes a bottle and gets re-tested.
No notes on this build yet.
How a note lands here
A note is a Learning Log entry with the tag study-attention-from-scratch. It is written in Connor's words, filed as a draft, and appears here once it is published from the admin queue. One note can carry several build tags.