Rung 13 Inference & Token-Pricing EconomicsSwitch rungClose
In scope: token pricing trends, inference gross margins, the AI-lab P&L question, and how memory, optics and foundry cost structure feed through to the price of a token.
| Type | Source | Published |
|---|---|---|
| REPORT | LLM Inference Prices Have Fallen Rapidly but Unequally Across Tasks Ben Cottier, Ben Snodin, David Owen, Tom Adamczewski · Epoch AI Median LLM inference price decline 50x/yr across 6 benchmarks (range 9x-900x/yr); GPT-4-level PhD-science (GPQA Diamond) milestone specifically 40x/yr, below the all-benchmark median; post-Jan-2024-only median rises to 200x/yr. | 2025-03-12 |
| REPORT | Token Price Index — GPT-5.6 holds the prior-generation price rungs MenFem Token Price Index (own dataset) · MenFem — research/inference/token-prices.csv GPT-5.6 (2026-07-16) holds the exact prior-generation per-token price at two of three tiers: Sol (frontier) $5/$30 = GPT-5.5's rung, Terra (strong) $2.50/$15 = GPT-5.4's rung; Luna (mid) $1/$6 is a new point. Capability up, price rungs held. | 2026-07-16 |
| REPORT | LLM Inference Handbook Modular · Modular The measurement vocabulary for inference cost: TPOT = (E2EL - TTFT)/(output tokens - 1), ITL, RPS, and GOODPUT (requests/sec meeting SLOs — the metric that stops a cost-per-token figure flattering a system that is fast and unusable). Plus GPU-memory and KV-cache calculators. | — |
| REPORT | OpenAI cuts GPT-5.6 Luna 80% and Terra 20%, and says why (announcement + API changelog + pricing page) OpenAI · OpenAI Luna -80% ($1.00/$6.00 -> $0.20/$1.20 per Mtok in/out) and Terra -20% ($2.50/$15 -> $2.00/$12) on 2026-07-30, with a vendor-named software cause: 20% lower serving cost from GPU kernel work and 15%+ better token generation from speculative decoding. Same changelog: launch 2026-07-09; Sol cut to $4/$20 on 2026-08-21 (promotional, at least through 2026-11-21); GPT-6 Sol $2/$10 on 2026-09-22. | 2026-07-30 |
| REPORT | Anthropic, Introducing Claude Fable 5.1: cache reads cut 75% Anthropic · Anthropic Fable 5.1 keeps $10/$50 input/output but cuts cache reads 75%, $1.00 -> $0.25 per Mtok (1/10 -> 1/40 of the input price); Anthropic estimates ~25% off typical workloads and up to ~45% off highly agentic ones. The price fall lands on the stored-context line item, not the headline rate. | 2026-09-01 |
| REPORT | Intercom, Fin AI Agent outcomes: $0.99 per outcome Intercom · Intercom The rung's first published price per task: $0.99 per outcome, charged once per conversation however many actions Fin takes (a resolution is one of five outcome types; a sales qualification is $9.99). Tokens, model and step count are all hidden inside the price. | 2026-09-23 |