Every term on this page is explained the way you'd tell a sharp friend — the library grows with every WTF-is-X episode.
Fine-tuning takes a trained model and gives it a short second education on your own examples — your tone, your formats, your field — so the behaviour sticks without teaching anything from scratch. Think of hiring a brilliant generalist, then spending a week training them on your house style.
Added 2026-07-30
The context window is the model's working memory — how much text it can hold in mind at once, counting your question, the conversation so far, and anything you pasted in. When it fills up, the oldest material falls out, which is why very long chats start to drift. Bigger windows exist; they cost more to run.
Added 2026-07-30
A model's knowledge is stored as billions of numbers. Quantization rounds those numbers to a coarser precision — like compressing a photo — so the model takes far less memory and runs faster and cheaper. Done carefully, the quality loss is barely measurable, which is why almost every model you use is quantized.
Added 2026-07-30
Models do not read letters or words. Text is first cut into tokens — common word-pieces from a fixed vocabulary — and every token is billed, counted against the context window, and produced one at a time. A rough rule for English: a token is about four characters, or three-quarters of a word.
Added 2026-09-08
The transformer is the design nearly every modern AI model uses. Its key move is attention: while processing each word, the model looks at every other word at once and weighs which ones matter. That parallel look is what made models trainable on enormous amounts of text.
Added 2026-09-08
Instead of one dense model where every part works on every word, a mixture of experts holds many specialised sub-networks and uses a small router to wake only a couple of them per token. The model is enormous on paper and cheap per answer, because most of it stays asleep.
Added 2026-09-08
Distillation trains a small model to copy a large one. The big model answers a pile of questions, and the small model learns to reproduce those answers, ending up far cheaper to run while keeping most of the ability on the work you care about.
Added 2026-09-08
A model that does not know something does not stop — it produces the most plausible continuation, which can be a fluent, well-formatted, entirely invented fact. It is not lying or malfunctioning; generating likely text is the only thing it does, and truth is not a separate check inside it.
Added 2026-09-08
A raw model predicts likely text, which is not the same as being helpful. RLHF shows people pairs of answers, records which they prefer, trains a scoring model on those judgements, and then tunes the model to score well. It is the step that turns a text predictor into an assistant.
Added 2026-09-08
Training builds the model; inference is using it. Every time you ask a chatbot a question, a data centre runs your words through the model and generates the reply, one word-piece at a time. Training happens once — inference happens billions of times a day, and every run costs money.
Added 2026-07-30
While writing a reply, the model keeps a running set of intermediate numbers for every token so far, so each new token does not require re-reading the whole conversation. That store is the KV cache. It is what makes generation fast, and it is also what fills up the GPU.
Added 2026-09-08
Latency is how long one person waits; throughput is how many people the machine serves per second. They pull against each other — the main way to raise throughput is to make each request wait a little so it can be processed alongside others.
Added 2026-09-08
Running one request through a GPU wastes most of the chip, because the weights have to be fetched either way. Batching processes many requests together against the same fetched weights, so serving twenty people costs barely more than serving one — which is where the margin in inference comes from.
Added 2026-09-08
A small fast model guesses the next few tokens, and the big model checks them all in one pass. Guesses it agrees with are kept, the first disagreement is corrected. The output is identical to running the big model alone — it just arrives sooner.
Added 2026-09-08
A GPU is a general-purpose parallel processor that happens to be excellent at AI; a TPU is a chip built for one job and nothing else. Custom silicon can be cheaper per unit of work, and it locks you to one vendor's software and one company's supply.
Added 2026-09-08
HBM is memory stacked vertically and mounted right next to the processor, giving it a far wider road to the chip than ordinary memory. It is what lets an accelerator feed itself fast enough to be useful, and it is one of the scarcest, most expensive parts of an AI machine.
Added 2026-09-08
Processors got much faster at arithmetic than memory got at delivering numbers to work on, so modern accelerators spend most of their time waiting rather than computing. Almost every optimisation in AI serving is an attempt to move fewer bytes, not to do less maths.
Added 2026-09-08
AI capacity is limited less by chips than by electricity and the ability to connect to a grid. A modern AI hall draws as much power as a small town, and the wait for a grid connection is now measured in years, which is why builders chase places with spare power.
Added 2026-09-08
PUE is the ratio of everything a data centre draws to the part that reaches the computers; 1.5 means half again is spent on cooling and losses. AI racks run so hot that air cooling is running out of room, which is pushing the industry to plumb liquid directly to the chips.
Added 2026-09-08
Instead of hoping the model memorised the right facts, you hand it the right documents at question time. The system searches your files, pastes the best passages into the prompt, and the model answers from those. It is why a chatbot can cite your company wiki without being retrained on it.
Added 2026-07-30
A chatbot answers you; an agent acts for you. Give it a goal and it runs a loop — pick a step, use a tool like search or code, check the result, adjust, repeat until the job is done. The bet is delegation: you review finished work instead of doing the work yourself.
Added 2026-07-30
Evals are the test suite for an AI system: a fixed set of questions with known good answers, scored automatically every time something changes. Without them, you are guessing whether a new model or prompt actually made things better. Teams with sharp evals ship improvements; teams without them ship vibes.
Added 2026-07-30
An embedding turns a piece of text into a long list of numbers that stands for its meaning, positioned so that passages about similar things end up near each other. Search then becomes geometry: find the nearest points. It is the machinery behind semantic search, recommendations and the retrieval half of RAG.
Added 2026-09-08
Guardrails are the checks placed around a model rather than inside it: filters on what goes in, validators on what comes out, and limits on what the system is allowed to do. They exist because you cannot make a model refuse reliably, but you can refuse on its behalf.
Added 2026-09-08
A model cannot tell your instructions apart from the text it is reading — both arrive as the same stream of words. So a web page, email or document can carry instructions of its own, and a system that reads it may follow them. It is the defining security problem of AI that touches the outside world.
Added 2026-09-08
MCP is a common plug format between AI assistants and the tools and data they use. Write one server for your system and any assistant that speaks the protocol can use it, instead of every product building a bespoke connector for every service.
Added 2026-09-08
Benchmarks are the standard exams models are graded on, and they are how launches get compared. They are useful for tracking the field over years and close to useless for choosing a model for your particular job, which your own test cases decide.
Added 2026-09-08
Not every question needs the biggest, most expensive model. A router reads each request and sends easy ones to a small, cheap model and hard ones to the heavyweight. Users rarely notice the difference — the bill notices, which is why most serious AI products quietly do this.
Added 2026-07-30
Most requests to an AI system start with the same long preamble — instructions, examples, a document. Prompt caching stores the model's half-finished reading of that fixed part so the next request skips re-reading it, cutting both the price and the wait before the first word appears.
Added 2026-09-08
An open-weights model is one whose trained numbers you can download and run on your own hardware. That is not the same as open source — the training data and code usually stay private, and the licence often carries conditions — but it does mean nobody can take the model away from you.
Added 2026-09-08
The living decision framework — updated as the desk tests.
DRAFT · v0 · reviewed 2026-07-30
No questions answered yet — be the first.
0 answered
Want the whole library on one page? Print the cheat sheet →