The Price-Decline Distribution

There is no single rate at which inference prices fall. It is a distribution, and the shape of that distribution — not the headline number — is what the margin-durability question turns on. Epoch AI's cross-benchmark study is the load-bearing evidence: it measures how fast the price to hit a fixed performance bar has fallen, and finds the answer depends enormously on which capability you are buying.

The central figures (Epoch AI, methodology-documented): a median 50x/year decline across six benchmarks, with the rate ranging from 9x to 900x per year depending on the benchmark and threshold chosen. The often-quoted "GPT-4-level" milestone — GPQA Diamond, PhD-level science — specifically falls at 40x/year, below the all-benchmark median, not above it. Restricting the sample to models released after January 2024 pushes the median up to 200x/year, suggesting the pace has accelerated recently — but that faster regime, and the fastest ~900x tails, are drawn entirely from a short post-2024 window, so their persistence is unknown.

Why the distribution shape matters more than the median. The buyer's real question is not "how fast do prices fall on average" but "how fast do they fall for the capability I depend on." A task whose price collapses at 900x/year is commoditizing under your feet — value there migrates to whoever sits above the model. A task whose price falls at only 9x/year is scarce and defensible: the slow curves are the moat signal. Reading the median alone ("prices fall 50x/year, so inference is a commodity") throws away exactly the information — the spread — that tells you which slices of inference retain pricing power and which do not.

The spread resolves into tier half-lives (added 2026-07-23; re-read in full 2026-09-11). Tiered Super-Moore's Law (Mingdeng Du, arXiv 2603.28576, 2026-03-30 — full text, 23 pages, read 2026-09-11) gives the same "fast and unequal" decline some structure Epoch does not. Its price series ends 2026-03-28, so it is a rate with an as-of date, never today's price:

• Economy tier: ~1.10-year price half-life — but the fit explains under a fifth of the variation (R² = 0.192).
• Mid tier: ~1.55-year half-life (R² = 0.307).
• Flagship tier: 3.59-year half-life with a fit so weak (R² = 0.031) that the paper calls exponential decay "inapplicable". Reasoning models are part of why: they carry a price premium over non-reasoning models that averages ~31.5× but swings from 0.51× to 83.3× by quarter — in one quarter reasoning models were cheaper.
• Aggregate: about a 600-fold relative decline since 2020 (no dated $/M level anywhere in the paper).
• A structural break dated to May 2024 (Chow F = 5.74, p = 0.005, on 16 observations before and 44 after): before, the decline looks technology-driven; after, competition-driven. Market concentration fell (HHI 4,558 → 2,086; OpenAI's estimated share 65% → 34%, estimated without volume data).

Three things the full text adds that change how far to lean on these numbers. The trend rests on 62 hand-compiled milestones — the 318-model and 3,237-model datasets are snapshots, not history. The tiers are price bands (flagship above $5/M input, economy below $0.5/M), so a model that gets cheaper moves down a tier, which flatters the tiers above it. And a cross-section of the 318 models finds the typical model priced at about 11.5× its most efficient equivalent — the market is very dispersed, which may be more useful than the trend.

This is a more structured version of the KB's "slow tails are the moat signal" reading: the economy tier commoditizes on a ~1.1-year clock, the flagship tier resists because the reasoning premium dominates it. The 31.5x reasoning premium is why cost per task, not price per token, is the right unit for reasoning workloads — the premium is bought because accuracy lowers cost-per-correct-answer.

The decline is bifurcated by tier — a benchmark datapoint (added 2026-07-24). WorkBench Revisited (Styles & Miller, arXiv 2606.13715, 2026-06-10 — abstract-grade) re-runs a fixed workplace-agent benchmark two years on and observes the distribution's shape directly in cost-of-performance terms: the cost of a performance level that was once proprietary-only collapsed (the abstract says "drastically lowered"; a ~100× figure is attributed to the paper body, which has not been read), driven by open-weight models — while frontier serving costs stayed flat. Over the same window best-agent task completion rose 43% (GPT-4, Mar-2024) → 98% (Claude Fable 5, mid-2026) and unintended harmful actions fell 26% → 1.9% (capability and safety moved together). The load-bearing reading for this page: the price decline shows up in the open-weight rung, not the frontier rung — the fast tails Epoch measures and the near-zero flagship fit Du measures are the same asymmetry seen from a benchmark, and it locates the commoditization where an open-weight model can reach a once-proprietary bar. It is a direct, if abstract-grade, datapoint on the open-weight-vs-frontier question the frontier gaps track; the exact per-tier $ levels need the full text. (Absolute per-task levels for this workload class — $0.17–$6.55/task on STAGE-Claw — live on cost per task.)

⚠ The software-vs-hardware conflict (added 2026-07-23). Du's headline econometric result is that ~103.7% of the cost reduction comes from total-factor productivity / software-architectural innovation and ~–0.9% from GPU hardware. In plain terms: hardware is not where the falling prices come from. This cuts directly against the memory/hardware-scarcity framing of Matsuoka (the depreciation conveyor, on the capital-structure rung), in which hardware cost structure is the load-bearing variable. (This page once also named Dylan Patel here; that podcast source was culled from the KB on 2026-08-06.) The two can be partly reconciled — memory scarcity may gate supply/capacity while software drives price — but as stated they locate the cost lever in different layers, and this KB keeps both without resolving it. Provenance to weigh, now from the full text: the "103.7%" is a residual — everything the growth-accounting model does not otherwise explain — and it exceeds 100% only because the hardware term came out slightly negative. It measures the size of the unexplained part, not software efficiency directly. Single-author preprint, not peer-reviewed. Logged as frontier Conflict 7; the same conflict is recorded on the depreciation-conveyor page.

This page measures price, not cost — and the two have come apart (added 2026-07-22). Epoch's series is built from published prices, which is what makes it strong evidence. But a falling price tells you nothing about a falling cost: Patil's measurement (arXiv 2606.11690, 2026-06-10) finds effective cost on identical H100 hardware spanning $0.21-$15.25 per million output tokens across configurations at $6.98/GPU-hr (Azure list), and 17.5–36.3× for a single model between idle and full load. A price series declining 50x/yr against a cost basis that can move 36x with traffic shape cannot, on its own, support a statement about margins. Keep this page's claims strictly on the price side; the cost side is the cost-measurement problem.

Two mechanisms behind the decline, one of them newly named. Matsuoka (arXiv 2607.07207, 2026-07-08) projects a training-cost bifurcation — a luxury tier at $18-38B per frontier run by 2030 against a mass tier reaching previous-frontier parity via RL/distillation for a cost falling toward $5M (both modelled). If that holds, the price of yesterday's capability collapses not because anyone cut a price but because reproducing it becomes ~3-4 orders of magnitude cheaper, and frontier-capable open-weight models (Matsuoka cites GLM-5.2) put a floor-setting free option under every paid tier. That is a supply-side mechanism for exactly the fast tails Epoch measures.

A caution now attaches to the demand side of any commodity argument built on this page. Matsuoka additionally argues that public token trackers overstate monetizable demand, and that all pre-Q2-2026 projections predate the industry's shift from token maximization to token minimization. Epoch's price series is not a token-volume tracker and is not directly hit by that critique — but the demand figures this KB pairs with it (Goldman's 24x-by-2030, Gartner's 5-30x agentic per-task multipliers, both located on the web 2026-07-17 and never registered as sources) largely predate that line. Recorded as a live conflict: preprint critique vs bank/analyst forecast, both moderate, unresolved.

The relationship to the rung. This concept measures price falling for fixed capability (the buy-side view). Its sell-side mirror is price-rung persistence: providers hold nominal per-token price points constant while capability rises. They are the same phenomenon seen from opposite ends — if the frontier rung stays at $5/$30 while the model improves, then the price to buy yesterday's capability must collapse (you buy it a rung down, or from open source). The Epoch distribution is what that collapse looks like when measured across tasks.

See also — what has moved since the series ended (context, not this rung's evidence)

Added 2026-09-24. Every price-trend number on this page ends by March 2026. Two dated events since then bear on it, and neither is registered on this rung yet:

• OpenAI cut GPT-5.6 Terra 20% and Luna 80% on 2026-07-30, leaving the frontier tier (Sol) unchanged — a tier-selective cut that is exactly the "fall lands on the lower tiers" shape this page describes. OpenAI's own announcement, now ingested on this rung (raw, 2026-09-24), names the cause: 20% lower serving cost from GPU kernel work and 15%+ better token generation from speculative decoding — software, not silicon. But the frontier did not stay flat for long: Sol went to a promotional $4/$20 (from $5/$30) on 2026-08-21, and GPT-6 Sol launched at $2/$10 on 2026-09-22 (same source). The "flat at the frontier" half of this page's shape now needs a date on it.
• The compilers rung now holds kernel-level evidence for the "software, not silicon" reading. DeepSeek's FlashMLA FP8 decode kernel went from 250 to 410 TFLOPS (1.64×) on the same Hopper chip, owned entirely by the kernel (raw); FlashAttention-4 reached 71% utilisation on B200, 1.3× the vendor library (raw). These are throughput gains, not prices, but they are the kind of mechanism Du's residual would be made of.

Key Claims

• Median inference price decline: 50x/year, measured across six benchmarks and multiple thresholds. Evidence: strong (methodology-documented benchmark study) (Epoch AI)
• Range: 9x to 900x per year — the rate varies enormously by benchmark/threshold; there is no single "true" rate. Evidence: strong (Epoch AI)
• GPT-4-level PhD-science (GPQA Diamond): 40x/year — below the all-benchmark median, despite being the headline example most often quoted. Evidence: strong (Epoch AI)
• Post-Jan-2024-only median: 200x/year — the pace appears to have accelerated recently, but the fastest tails come entirely from a short window, so durability is unknown. Evidence: moderate (short track record) (Epoch AI)
• The slow curves (9x end) are the moat signal — where price falls slowly, capability is scarce and pricing power persists; where it falls fast (900x), commoditization is racing. Evidence: moderate (interpretation of the Epoch spread; not an Epoch claim)
• Tier half-lives: economy ~1.10yr (R²=0.192), mid ~1.55yr (R²=0.307), flagship 3.59yr with no usable fit (R²=0.031); reasoning premium ~31.5× on average but 0.51–83.3× by quarter; ~600× aggregate decline; series ends 2026-03-28. Evidence: moderate — full text read 2026-09-11; direction robust, precision weak; trend rests on 62 milestones; tiers are price bands (Du / Tiered Super-Moore's)
• A May-2024 structural break shifts the decline from technology-driven to competition-driven (Chow F=5.74, p=0.005); HHI falls 4,558 → 2,086. Evidence: moderate-weak — estimated econometrics; only 16 observations before the break (Du)
• CONFLICT (unresolved): ~103.7% of the price decline attributed to software/TFP, ~–0.9% to GPU hardware — hardware is not where falling prices come from, cutting against Matsuoka's memory-scarcity framing. Memory may gate capacity while software drives price. Evidence: estimated growth-accounting residual (full text read 2026-09-11) — the >100% is the unexplained part, not a direct measure of software efficiency; single-author preprint (Du)
• The decline is bifurcated open-weight vs frontier: on a fixed workplace-agent benchmark the cost of a once-proprietary performance level collapsed (size unconfirmed; ~100× attributed to the unread body) while frontier serving costs stayed flat; completion 43%→98% and harmful actions 26%→1.9% (Mar-2024→mid-2026). The price decline shows up in the open-weight rung, not the frontier rung. Evidence: moderate — measured longitudinal benchmark, abstract-grade; per-tier $ levels need the full text (WorkBench Revisited)
• A falling price series cannot support a margin claim on its own — effective cost on identical H100 hardware spans $0.21-$15.25/M output tokens (as of 2026-06) with traffic shape alone. Evidence: moderate (measured; the level is anchored to $6.98/GPU-hr Azure H100 NVL list, found in the full text 2026-07-23) (Patil)
• Training-cost bifurcation is a supply-side mechanism for the fast tails: $18-38B per frontier run by 2030 vs mass-tier previous-frontier parity falling toward $5M. Evidence: weak-moderate (modelled projection to 2030) (Matsuoka)
• Public token trackers overstate monetizable demand; pre-Q2-2026 demand projections predate the token-maximization → token-minimization shift. Evidence: moderate (preprint claim, magnitude not stated, not reproduced here); conflicts with the Goldman/Gartner demand figures logged 2026-07-17 (Matsuoka)

Benchmarks & Data

Metric	Value	Source
Median decline (all benchmarks)	50x / year	Epoch AI
Range across benchmarks/thresholds	9x – 900x / year	Epoch AI
GPT-4-level PhD-science (GPQA Diamond)	40x / year (below median)	Epoch AI
Median, post-Jan-2024 models only	200x / year	Epoch AI
Economy-tier price half-life	~1.10 year (R² 0.192)	Du / Tiered Super-Moore's
Mid-tier price half-life	~1.55 year (R² 0.307)	Du
Flagship-tier fit	3.59 yr, no usable fit (R² = 0.031)	Du
Reasoning per-token premium	~31.5× average; 0.51–83.3× by quarter	Du
Aggregate decline (relative)	~600x	Du
Structural break (tech → competition)	May 2024 (Chow F=5.74, p=0.005)	Du
Source of decline (estimated)	~103.7% TFP / –0.9% GPU hardware	Du
Cost of a fixed performance level, open-weight vs frontier	open-weight "drastically lower" (~100× attributed, unread) / frontier flat	WorkBench Revisited
Best-agent workplace-task completion	43% (Mar-2024) → 98% (mid-2026)	WorkBench Revisited

Epoch benchmarks in the study: MMLU (general knowledge), GPQA Diamond (PhD science), MATH-500 / MATH Level 5 (math), HumanEval (coding), Chatbot Arena ELO (chat). Du's figures are relative/econometric — no absolute dated $/M-token level appears anywhere in the full text, and the series ends 2026-03-28.

Open Questions

• Does the post-2024 200x/year pace persist, or was 2024-2025 a one-off efficiency unlock (MoE, quantization, speculative decoding) that won't repeat?
• Which specific tasks sit at the slow (9x) end of the distribution — the defensible slices — and does that set shrink over time?
• Does the distribution's shape change (does the spread widen or narrow) as the frontier advances, or only its median?
• Is there a matching cost-decline distribution? This page has a price series and no cost series. Until one exists, "prices fell 50x/yr" and "margins compressed" are unconnected statements.
• Does the Epoch series hold up under the token-minimization shift? If providers and buyers both optimise for fewer tokens per task, a per-token price series and a per-task cost diverge — and per-task is the figure that matters commercially.
• Is Epoch's quality-held 50x/yr the right denominator for a commodity argument? The blended-spot price a buyer actually pays fell only ~3x/yr (2025→2026) — quality-per-dollar and unit price are different metrics that differ by >1 order of magnitude (see frontier.md → Evidence Located).

Related Concepts

• Token Pricing & the Inference-Margin Question — the umbrella question this distribution feeds: fast price decline is the mechanism by which inference could become commodity infrastructure.
• Price-Rung Persistence — the sell-side mirror: providers hold the rung while capability rises, which is what forces the buy-side price collapse this page measures.
• The Cost-Measurement Problem — this page is a price series; that page is why it cannot be read as a cost or margin series.
• The Depreciation Conveyor & Vintage Economics (on capital-structure) — training-cost bifurcation and open-weight parity are the supply-side mechanisms behind the fastest tails here; also carries the same software-vs-hardware conflict from the cost-structure side.
• Cost Per Task (Cost-of-Pass) — the reasoning premium that scrambles the flagship tier is why cost-per-task, not price-per-token, is the right unit for reasoning workloads.

Backlinks

Pages that reference this concept:

• Token Pricing & the Inference-Margin Question
• Price-Rung Persistence

Changelog

• 2026-09-24 (source ingest) — OpenAI's GPT-5.6 price-cut announcement ingested on this rung: the July cut now has a primary source and a named software cause, and the frontier tier's later cuts (Sol 2026-08-21, GPT-6 Sol 2026-09-22) are recorded against the "flat at the frontier" shape.
• 2026-09-24 (recompile, no new source) — Brought the Tiered Super-Moore's section up to the 2026-09-11 full-text read: added the weak fits (economy R² 0.192), the flagship's 3.59-year half-life, the 0.51–83.3× quarterly range behind the 31.5× premium, the 62-milestone basis, the price-band tiers, and the 16-observation pre-break sample; restated the 103.7% as a residual. Softened WorkBench's ~100× to "attributed, unread". Removed the culled Patel citation and fixed the depreciation-conveyor link. Corrected the Patil claim that still said the $/GPU-hour input was undisclosed. Added dated context: the 2026-07-30 OpenAI tier cut (own-dataset observation; its primary source was ingested later the same day) and two compilers kernel results.
• 2026-07-24 — Compiled WorkBench Revisited (2606.13715, abstract-grade). Added the bifurcated-decline datapoint: on a fixed workplace-agent benchmark the cost of a once-proprietary performance level collapsed ~100x since 2024 (open-weight-driven) while frontier serving costs stayed flat — the price decline lands in the open-weight rung, not the frontier rung (completion 43%→98%, harmful actions 26%→1.9%). New section, one Key Claim, two data rows; per-tier $ levels owed to the full text. Absolute per-task levels ($0.17–$6.55/task, STAGE-Claw) routed to cost per task. No change to the Epoch/Du price series.
• 2026-07-23 — Compiled Tiered Super-Moore's Law (Du, 2603.28576, abstract-page). Added tier half-lives (economy 1.10yr / mid 1.55yr / flagship near-zero, R²=0.031, 31.5x reasoning premium), the May-2024 tech→competition structural break (HHI 4,558→2,086), and — prominently — the software-vs-hardware conflict (~103.7% TFP / –0.9% GPU hardware), recorded unresolved against Matsuoka/Patel memory-as-destiny. Linked the reasoning premium to cost-per-task. No change to the Epoch price series itself.
• 2026-07-22 — Compiled Patil (2606.11690) and Matsuoka (2607.07207) against this page: added the explicit price-is-not-cost boundary, training-cost bifurcation as a supply-side mechanism for the fast tails, and Matsuoka's demand-tracker critique as a recorded conflict with the Goldman/Gartner figures logged 2026-07-17.
• 2026-07-16 — Created from the Epoch AI source (already on the shelf, ingested 2026-07-14). Split out from the founding umbrella concept to make the price-decline distribution a first-class idea; paired with the new price-rung-persistence concept as its sell-side mirror.