Skip to content
Sign in
Explained / 30 terms

AI, explained simply. No jargon.

Explained the way you'd tell a sharp friend — one number each, and where it breaks.

The whole library, one page
A machined aluminium loupe with a copper ring, lying on its side.
The loupeWTF-is-X

Models

10,000× fewer trainable parametersreduction LoRA reports when adapting GPT-3 175B instead of retraining every weightHu et al., "LoRA: Low-Rank Adaptation of Large Language Models" · 2021-06-17

Fine-tuning

Fine-tuning takes a trained model and gives it a short second education on your own examples — your tone, your formats, your field — so the behaviour sticks without teaching anything from scratch. Think of hiring a brilliant generalist, then spending a week training them on your house style.

Added 2026-07-30

128,000 tokenscontext window of GPT-4o, roughly a 300-page bookOpenAI API model documentation · 2024-05-13

Context windows

The context window is the model's working memory — how much text it can hold in mind at once, counting your question, the conversation so far, and anything you pasted in. When it fills up, the oldest material falls out, which is why very long chats start to drift. Bigger windows exist; they cost more to run.

Added 2026-07-30

4× smallermemory cut from storing weights at 4 bits instead of 16Frantar et al., "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers" (ICLR 2023) · 2022-10-31

Quantization

A model's knowledge is stored as billions of numbers. Quantization rounds those numbers to a coarser precision — like compressing a photo — so the model takes far less memory and runs faster and cheaper. Done carefully, the quality loss is barely measurable, which is why almost every model you use is quantized.

Added 2026-07-30

~4 charactersper token, for ordinary English textOpenAI help centre, "What are tokens and how to count them" · 2024-01-01

Tokens

Models do not read letters or words. Text is first cut into tokens — common word-pieces from a fixed vocabulary — and every token is billed, counted against the context window, and produced one at a time. A rough rule for English: a token is about four characters, or three-quarters of a word.

Added 2026-09-08

8 attention headsparallel attention comparisons per layer in the original transformerVaswani et al., "Attention Is All You Need" (NeurIPS 2017) · 2017-06-12

Transformers and attention

The transformer is the design nearly every modern AI model uses. Its key move is attention: while processing each word, the model looks at every other word at once and weighs which ones matter. That parallel look is what made models trainable on enormous amounts of text.

Added 2026-09-08

46.7B total, 12.9B active per tokenparameter split in Mixtral 8x7BMistral AI, "Mixtral of Experts" · 2024-01-08

Mixture of experts (MoE)

Instead of one dense model where every part works on every word, a mixture of experts holds many specialised sub-networks and uses a small router to wake only a couple of them per token. The model is enormous on paper and cheap per answer, because most of it stays asleep.

Added 2026-09-08

40% smaller, 97% of performanceDistilBERT's size cut and retained GLUE score against BERTSanh et al., "DistilBERT, a distilled version of BERT" · 2019-10-02

Distillation

Distillation trains a small model to copy a large one. The big model answers a pile of questions, and the small model learns to reproduce those answers, ending up far cheaper to run while keeping most of the ability on the work you care about.

Added 2026-09-08

58% truthfulbest-performing model's score on TruthfulQA against 94% for humansLin et al., "TruthfulQA: Measuring How Models Mimic Human Falsehoods" (ACL 2022) · 2021-09-08

Hallucination

A model that does not know something does not stop — it produces the most plausible continuation, which can be a fluent, well-formatted, entirely invented fact. It is not lying or malfunctioning; generating likely text is the only thing it does, and truth is not a separate check inside it.

Added 2026-09-08

1.3B beat 175Blabelers preferred the small RLHF-tuned InstructGPT over the 100× larger base GPT-3Ouyang et al., "Training language models to follow instructions with human feedback" · 2022-03-04

RLHF (learning from human feedback)

A raw model predicts likely text, which is not the same as being helpful. RLHF shows people pairs of answers, records which they prefer, trains a scoring model on those judgements, and then tunes the model to score well. It is the step that turns a text predictor into an assistant.

Added 2026-09-08

Infrastructure

$0.15 / $0.60 per million tokenslisted input / output price of GPT-4o mini at launchOpenAI, "GPT-4o mini: advancing cost-efficient intelligence" · 2024-07-18

Inference

Training builds the model; inference is using it. Every time you ask a chatbot a question, a data centre runs your words through the model and generates the reply, one word-piece at a time. Training happens once — inference happens billions of times a day, and every run costs money.

Added 2026-07-30

up to 24× throughputgain vLLM reports over HuggingFace Transformers by managing KV cache memory in pagesKwon et al., "Efficient Memory Management for LLM Serving with PagedAttention" (SOSP 2023) · 2023-09-12

KV cache

While writing a reply, the model keeps a running set of intermediate numbers for every token so far, so each new token does not require re-reading the whole conversation. That store is the KV cache. It is what makes generation fast, and it is also what fills up the GPU.

Added 2026-09-08

TTFT 2,000 ms, per-token 200 mslatency constraints MLPerf Inference imposes on the Llama 2 70B server scenarioMLCommons, MLPerf Inference v4.0 results and rules · 2024-03-27

Latency vs throughput

Latency is how long one person waits; throughput is how many people the machine serves per second. They pull against each other — the main way to raise throughput is to make each request wait a little so it can be processed alongside others.

Added 2026-09-08

up to 36.9× throughputgain from continuous batching at the same latency, versus fixed-batch servingYu et al., "Orca: A Distributed Serving System for Transformer-Based Generative Models" (OSDI 2022) · 2022-07-11

Batching

Running one request through a GPU wastes most of the chip, because the weights have to be fetched either way. Batching processes many requests together against the same fetched weights, so serving twenty people costs barely more than serving one — which is where the margin in inference comes from.

Added 2026-09-08

Hardware & Power

80 GB at 3.35 TB/smemory capacity and bandwidth of the NVIDIA H100 SXMNVIDIA H100 Tensor Core GPU datasheet · 2023-03-21

GPUs vs TPUs

A GPU is a general-purpose parallel processor that happens to be excellent at AI; a TPU is a chip built for one job and nothing else. Custom silicon can be cheaper per unit of work, and it locks you to one vendor's software and one company's supply.

Added 2026-09-08

141 GB at 4.8 TB/sHBM3E capacity and bandwidth on the NVIDIA H200NVIDIA H200 Tensor Core GPU datasheet · 2023-11-13

HBM (high-bandwidth memory)

HBM is memory stacked vertically and mounted right next to the processor, giving it a far wider road to the chip than ordinary memory. It is what lets an accelerator feed itself fast enough to be useful, and it is one of the scarcest, most expensive parts of an AI machine.

Added 2026-09-08

~300 FLOPs per byteratio of BF16 arithmetic (989 TFLOPS) to memory bandwidth (3.35 TB/s) on an H100 SXMNVIDIA H100 Tensor Core GPU datasheet · 2023-03-21

The memory wall

Processors got much faster at arithmetic than memory got at delivering numbers to work on, so modern accelerators spend most of their time waiting rather than computing. Almost every optimisation in AI serving is an attempt to move fewer bytes, not to do less maths.

Added 2026-09-08

460 TWh in 2022, 620–1,050 TWh by 2026global data centre electricity consumption and forecast rangeIEA, "Electricity 2024" report · 2024-01-24

Data centre power

AI capacity is limited less by chips than by electricity and the ability to connect to a grid. A modern AI hall draws as much power as a small town, and the wait for a grid connection is now measured in years, which is why builders chase places with spare power.

Added 2026-09-08

1.10Google fleet-wide trailing twelve-month power usage effectivenessGoogle data centre efficiency disclosures · 2023-12-31

PUE and cooling

PUE is the ratio of everything a data centre draws to the part that reaches the computers; 1.5 means half again is spent on cooling and losses. AI racks run so hot that air cooling is running out of room, which is pushing the industry to plumb liquid directly to the chips.

Added 2026-09-08

Engineering

21 million passagessize of the Wikipedia index the original retrieval systems searched per questionKarpukhin et al., "Dense Passage Retrieval for Open-Domain QA" (EMNLP 2020) · 2020-09-30

RAG (retrieval-augmented generation)

Instead of hoping the model memorised the right facts, you hand it the right documents at question time. The system searches your files, pastes the best passages into the prompt, and the model answers from those. It is why a chatbot can cite your company wiki without being retrained on it.

Added 2026-07-30

500 taskssize of SWE-bench Verified, the human-validated set of real GitHub issues agents are graded onOpenAI, "Introducing SWE-bench Verified" · 2024-08-13

AI agents

A chatbot answers you; an agent acts for you. Give it a goal and it runs a loop — pick a step, use a tool like search or code, check the result, adjust, repeat until the job is done. The bet is delegation: you review finished work instead of doing the work yourself.

Added 2026-07-30

57 subjectsbreadth of MMLU, the benchmark most model launches still quoteHendrycks et al., "Measuring Massive Multitask Language Understanding" (ICLR 2021) · 2021-01-12

AI evaluations (evals)

Evals are the test suite for an AI system: a fixed set of questions with known good answers, scored automatically every time something changes. Without them, you are guessing whether a new model or prompt actually made things better. Teams with sharp evals ship improvements; teams without them ship vibes.

Added 2026-07-30

1,536 dimensionsdefault vector length of OpenAI text-embedding-3-smallOpenAI, "New embedding models and API updates" · 2024-01-25

Embeddings

An embedding turns a piece of text into a long list of numbers that stands for its meaning, positioned so that passages about similar things end up near each other. Search then becomes geometry: find the nearest points. It is the machinery behind semantic search, recommendations and the retrieval half of RAG.

Added 2026-09-08

freeprice of the OpenAI moderation endpoint, the standard input filterOpenAI platform documentation, Moderation guide · 2024-01-25

Guardrails

Guardrails are the checks placed around a model rather than inside it: filters on what goes in, validators on what comes out, and limits on what the system is allowed to do. They exist because you cannot make a model refuse reliably, but you can refuse on its behalf.

Added 2026-09-08

ranked LLM01prompt injection's position at number one in the OWASP Top 10 for LLM applicationsOWASP Top 10 for Large Language Model Applications, 2025 edition · 2024-11-17

Prompt injection

A model cannot tell your instructions apart from the text it is reading — both arrive as the same stream of words. So a web page, email or document can carry instructions of its own, and a system that reads it may follow them. It is the defining security problem of AI that touches the outside world.

Added 2026-09-08

open-sourced 25 Nov 2024date Anthropic published the Model Context Protocol specificationAnthropic, "Introducing the Model Context Protocol" · 2024-11-25

MCP (Model Context Protocol)

MCP is a common plug format between AI assistants and the tools and data they use. Write one server for your system and any assistant that speaks the protocol can use it, instead of every product building a bespoke connector for every service.

Added 2026-09-08

34%accuracy of skilled non-experts with unrestricted web access on GPQA questions, against 65% for domain PhDsRein et al., "GPQA: A Graduate-Level Google-Proof Q&A Benchmark" · 2023-11-20

Benchmarks

Benchmarks are the standard exams models are graded on, and they are how launches get compared. They are useful for tracking the field over years and close to useless for choosing a model for your particular job, which your own test cases decide.

Added 2026-09-08

Economics

~17× cheaper inputgap between GPT-4o at $2.50/M and GPT-4o mini at $0.15/M input tokensOpenAI API pricing page · 2024-07-18

Model routing

Not every question needs the biggest, most expensive model. A router reads each request and sends easy ones to a small, cheap model and hard ones to the heavyweight. Users rarely notice the difference — the bill notices, which is why most serious AI products quietly do this.

Added 2026-07-30

10% of base input pricewhat Anthropic charges for a cache read against a written prompt cacheAnthropic, "Prompt caching with Claude" · 2024-08-14

Prompt caching

Most requests to an AI system start with the same long preamble — instructions, examples, a document. Prompt caching stores the model's half-finished reading of that fixed part so the next request skips re-reading it, cutting both the price and the wait before the first word appears.

Added 2026-09-08

700 million monthly usersthreshold above which the Llama 3.1 licence requires a separate agreementMeta, Llama 3.1 Community License Agreement · 2024-07-23

Open weights

An open-weights model is one whose trained numbers you can download and run on your own hardware. That is not the same as open source — the training data and code usually stay private, and the licence often carries conditions — but it does mean nobody can take the model away from you.

Added 2026-09-08

Which model should I use?

The living decision framework — updated as the desk tests.

DRAFT · v0 · reviewed 2026-07-30

Asked — reader questions

No questions answered yet — be the first.