Inference & Token-Pricing Economics — Research Frontier

What is moving, what is stale, and what would refresh it.

State of the rung, in plain words (2026-09-24)

Three new sources today, the first since 2026-08-06 — the price shelf is current again, the cost shelf is not. Approved, and ingested in this pass:

1. OpenAI's GPT-5.6 price cut (2026-07-30), from OpenAI's own forum announcement, API changelog and pricing page (raw). The prices the discovery pass took from secondary reporting are confirmed. The cause is named: kernels and speculative decoding. The same changelog shows the frontier model cut on 2026-08-21 and GPT-6 launched on 2026-09-22.
2. Anthropic's Claude Fable 5.1 cache-read cut (2026-09-01): cache reads −75%, headline rate unchanged (raw). Admitted on the boundary: we rejected two pricing-policy items on 2026-09-13 as drift toward the retired Token Price Index; this one clears only because the cache-read line is itself the mechanism.
3. Intercom Fin, $0.99 per outcome — the rung's first published price per task (raw). The originally proposed help page could not be read (HTTP 401); two other Intercom pages carry the price. No archive copy could be made; the close-read holds the verbatim quotes, fetch time and page hashes.

All three are vendor disclosures, not measurements. The rung's newest price is now September 2026; its newest price-trend series still ends 2026-03-28, and it still holds no first-hand cost figure. The earlier 2026-09-24 recompile (no new source) is recorded in the timeline.

What is stale — the dated observations and their as-of dates

Every figure below is a dated observation, not a current value. Cite it with its date or not at all.

Claim on the rung	As of	Why it is stale
Price of fixed capability falls at a median 50×/yr (9×–900×)	Epoch series published 2025-03-12	18 months old; the fastest rates came from a short post-2024 window
Tier half-lives: economy 1.10 yr, mid 1.55 yr, flagship unfittable	series ends 2026-03-28	predates every July–August 2026 price cut; weak fits
GPT-5.6 held two of three prior-generation rungs	2026-07-16	fully broken: Terra −20% and Luna −80% on 2026-07-30, Sol to $4/$20 (promotional) on 2026-08-21 — confirmed on OpenAI's own pages 2026-09-24
GPT-5.6 / GPT-6 list prices; Fable 5.1 cache read $0.25	2026-09-24 (read)	current today; list prices move within weeks — re-read before citing; Sol's $4/$20 is promotional to at least 2026-11-21
Intercom Fin $0.99 per outcome	2026-09-24 (read; page updated ~09-23)	vendor help page, no archive copy — cite with the date
Anthropic: frontier rung held, strong rung cut	2026-08-13	vendor-page and own-dataset reading; no raw close-read on the rung
$0.21–$15.25 per million output tokens on one H100 setup	June 2026 test window, $6.98/GPU-hr Azure list	one chip generation, one price basis
Independent-provider list prices, $0.65–$4.20 per million (Llama 4 70B)	Q2 2026 (published 2026-04-24)	five months old; one model
$0.17–$6.55 per agent task	paper dated 2026-06-09	still unconfirmed — search-surfaced, never read from the paper
Inference gross margin 40–50%, training excluded	2026-07-09	an essay's estimate with no cost basis
326 TWh → ~6.5×10¹⁷ tokens a year	paper dated 2026-02-23	order-of-magnitude, abstract-read

What would refresh it

The three sources approved on 2026-09-24 are ingested (see the top of this page); no proposal for this rung is now awaiting approval. The SemiAnalysis GPU cluster-cost piece stays held pending a ruling on whether it belongs here or on datacenters.

What else a real refresh needs, beyond approvals: full-text reads of STAGE-Claw (to confirm or drop $0.17–$6.55 per task), WorkBench Revisited (to size the open-weight-versus-frontier split) and Cost-of-Pass (to date "halved every few months"); and one run of vllm-cost-meter on real traffic, which would be the rung's first first-hand cost measurement.

What has changed since July — sibling rungs (context, not this rung's evidence)

Linked so a reader can see the neighbours moved even though this rung did not. None of these counts as a source here.

• serving — DigitalOcean's measured H200 cost ($20.32 → $0.45 per million output tokens, batch 1 → 128, 2026-07-08; raw) and Epoch's capacity model (capacity ~3.4×/yr vs demand ~10×; raw) were compiled on this rung in July and are now registered on serving. New there since: a 2.5× goodput gain from splitting prefill and decode on the same GPUs (raw).
• harnesses — a controlled orchestration swap cut cost per task $0.21 → $0.12 at equal quality (raw).
• compilers — kernel-only speed-ups on unchanged chips (FlashMLA 1.64× on Hopper; FlashAttention-4 at 71% utilisation on B200) — FlashMLA, FlashAttention-4.
• capital-structure — GPU-backed borrowing costs widened from SOFR+450 (May) to SOFR+550 (August) at CoreWeave (raw); the depreciation conveyor concept now lives there.
• energy — record generation forecast (EIA, 2026-09-09) against a Texas pause on connecting new data-centre load (ERCOT, 2026-08-03): plentiful energy, scarce power at the site (EIA, ERCOT).
• hardware — SK hynix's 76% Q2 2026 operating margin, memory scarcity stated as a margin (raw).

Active Frontiers

1. The cost denominator — unmeasured for any frontier model

Status: Active, and the rung's most important gap. Key sources: Patil · Matsuoka · Modular handbook Concept: The Cost-Measurement Problem

Cost per token depends on how busy the hardware is (17–36× for one model on one H100 setup), and nothing in the KB measures what a frontier model costs to serve. The measured cost points that exist — Patil's H100/A100 sweeps here, DigitalOcean's H200 on serving — are open models, single dates, and none is priced against goodput.

Open problems: a frontier-model cost; a cost series rather than points; any cost at a stated service level; whether $/PB works outside decode.

2. Does inference become cheap plumbing?

Status: Steady — undecidable on this evidence. Key sources: Evans · Matsuoka · Du Concept: the margin question

Matsuoka's scenario weights tie the two poles at 25% each (expert judgment, five rounds). Du's May-2024 break leans weakly toward competition compressing prices. The frontier move this page said to watch has happened: OpenAI put GPT-5.6 Sol on a promotional $4/$20 on 2026-08-21 and launched GPT-6 Sol at $2/$10 on 2026-09-22 (OpenAI). Both labs now hold a $10/$50 point above it (GPT-6 Astra, Fable). That leans toward compression — with the caveat that a list-price cut says nothing about margin until a cost figure exists.

3. Where the price fall lands

Status: Active. Key sources: Epoch AI · Du · WorkBench Revisited Concepts: price-decline distribution · price-rung persistence (dated archive)

Every trend source agrees on the shape — fast at the cheap and mid tiers, flat at the frontier — and the July cuts fit it. The August and September moves do not: the frontier model was cut too, and a new generation launched below it. Two further shapes are new: the fall landing on a single line item (Fable 5.1's cache reads, headline rate unchanged — Anthropic), and a price per task that hides the token price altogether (Intercom). None of the trend sources measures the pace after March 2026.

4. Cost per task

Status: Active — the right unit, with no confirmed number on this rung. Key sources: Cost-of-Pass · AI Tokenomics · STAGE-Claw Concept: cost per task

All three are abstract-read; STAGE-Claw's dollar range is unconfirmed. The best-controlled task-price result in the KB sits on harnesses, not here. The rung now holds its first published task price: Intercom Fin at $0.99 per outcome, once per conversation, with the token count hidden inside it (Intercom). It is a list price, not a measured cost, and "done" is Intercom's definition.

5. Energy and the macro channel

Status: Early — one abstract-read preprint each. Concepts: energy cost of tokens · macro channel

Energy is plentiful in total but power is scarce at the site; neither page has a $/kWh-to-$/token conversion. The macro channel rests on a stipulated 18% cost share.

Conflicts on the Record

Recorded, not decided. Where sources disagree, the rung keeps both.

1. Unit — per token versus $/PB of bandwidth. Matsuoka argues $/PB for decode; everything else here is per token. Unreconciled.
2. Method — margins without denominators. Patil's spread makes Evans' 40–50% uncheckable, not wrong.
3. Demand — trackers overstate versus bank forecasts. Matsuoka says public token trackers overstate paid demand; the Goldman and Gartner figures below (located on the web, never registered) say demand is exploding. Unresolved.
4. Rate — which price decline is real. Epoch's quality-held 50×/yr versus a located blended-spot ~3×/yr a buyer actually paid (2025→2026). Different measures; the gap between them is the rung-hold mechanic.
5. What drives the price fall — software or hardware. Du attributes ~103.7% to productivity and −0.9% to GPU hardware (a residual — the unexplained part — now read in full). Matsuoka's model runs on memory and hardware vintage. Both can be partly true (memory limits capacity, software drives price). OpenAI's own account of its 2026-07-30 cut (ingested 2026-09-24) names GPU kernels and speculative decoding — software on existing chips — as the cause: one vendor-stated case on Du's side. It does not settle the conflict: the percentages have no dollar base, and the 20% saving explains only Terra's 20% cut, not Luna's 80%.
6. GPT-5.6 launch date. Resolved 2026-09-24: OpenAI's own changelog dates the release 2026-07-09; the Token Price Index recorded the rows on 2026-07-16.
7. Energy abundant, power scarce. Photons = Tokens says energy does not limit token volume; the energy rung's sources show power delivery to sites is a live constraint. Different quantities; recorded as a tension.
8. Internal tension in Matsuoka. He prices solvency on token-demand growth while arguing the trackers that measure demand are inflated. The full read (via hardware) shows he does revise demand for his own critique — to 2–3×/yr, self-labelled an upper bound — but that figure is asserted, not computed, and it is what puts his central case on the solvency line.

Retired from this list 2026-09-24: the "Patel 70% versus reported ~50%" margin conflict — its source was culled from the KB on 2026-08-06.

Knowledge Gaps

• A frontier-model cost to serve. Nothing in the KB. Suggested search: "measured serving cost frontier model per million tokens 2026".
• Full-text reads of the three abstract-read task-price sources. STAGE-Claw, WorkBench Revisited, Cost-of-Pass.
• Zero peer-reviewed papers. 9 preprints, 3 technical reports, 2 analyses. A Joule/Cell paper on inference energy was blocked (403) on 2026-07-23 and has not been retried.
• Audited lab financials. Anthropic's confidential draft S-1 (reported ~2026-06-01) is the only dated path; this pass did not search for a public filing.
• Non-US inference pricing. DeepSeek, Qwen, GLM rungs are absent from this rung (Kimi K3 pricing sits on models).
• Paid versus list prices. Enterprise and committed-use pricing is invisible everywhere on the rung.
• A cost at a stated service level. No goodput-denominated cost on this rung.

Evidence Located (2026-07-17 reading-desk session)

Located on the web during a reading pass; none of it is registered as a source, all of it is secondary or reported, and it is kept as the record, not as evidence.

• Anthropic: confidential draft S-1 reported filed ~2026-06-01; run-rate revenue ~$47B (May 2026); present-period gross margin reported near 50%, ~77% projected for 2028 (investor-deck lineage, unaudited). (Yahoo Finance, Simon Willison, Forbes, Sacra)
• Demand: Goldman Sachs forecasts token use up 24× to ~120 quadrillion tokens a month by 2030 (Goldman Sachs); compounded, that is ~2.2×/yr over four years — right on Matsuoka's ~2× solvency line (desk arithmetic, 2026-07-22). Blended token price reportedly fell ~3× from Q1 2025 to Q1 2026 (NeuralWired/Gartner). Agentic work reportedly uses 5–30× more tokens per task ("tokenmaxxing", Forbes Tech Council).
• Memory supply: IDC 2026 DRAM supply +16%, NAND +17%; TrendForce HBM demand +130% (2025) and +70% (2026). (IDC, TrendForce) The hardware rung now holds primary memory evidence (SK hynix, above).

Corrections made on the 2026-09-24 ingest

• "OpenAI holds its rungs" / "the frontier rung held through August" → OpenAI cut every GPT-5.6 tier, the frontier one (Sol) on 2026-08-21, promotionally; GPT-6 Sol launched at $2/$10. Corrected on price-rung persistence, the OpenAI pricing page and the index.
• July cut prices marked [unverified] from secondary reporting → confirmed on OpenAI's own changelog and pricing page.
• "Intercom Fin, $0.99 per resolution" → Intercom prices per outcome; a resolution is one of five outcome types, and one type (sales qualification) is $9.99.
• GPT-5.6 launch date unsettled → 2026-07-09.

Corrections made in the 2026-09-24 recompile

• "Identical hardware, identical model, a 73× spread" → the 73× compares two different models; the same-model spread is 17.5–36.3×.
• Tiered Super-Moore's Law described as abstract-only in four places → read in full on 2026-09-11; fits are weak, tiers are price bands, the trend rests on 62 milestones, the 103.7% is a residual, and the page count is 23, not 26.
• "Absolute level rests on an undisclosed $/GPU-hour input" (Patil) → found in July: $6.98.
• "~100× since 2024" (WorkBench) presented as read → attributed to an unread paper body.
• STAGE-Claw's $0.17–$6.55 graded as measured → re-graded weak-moderate until confirmed.
• Matsuoka's weights "with no elicitation method" → expert judgment over five rounds.
• Phillips-curve details not in the full text (a Hansen J-test, "within one standard error", Wald p = 0.78) → removed.
• DigitalOcean, Epoch and the depreciation conveyor cited as this rung's own → linked to serving and capital-structure.
• Patel's 70%+ Anthropic margin used as evidence → flagged as culled; the strike-or-keep call on the entity page is still an open editorial decision.
• GPT-5.6's held rungs presented as current on the margin and OpenAI pages → the 2026-07-30 cuts added.
• Source counts — this file said 13 and 16 in different paragraphs; the registry holds 14, and raw/ holds 14 close reads plus the registry file.