Cost Per Task (Cost-of-Pass)

This topic has said, repeatedly, that the commercially meaningful unit is cost per task, not price per token — and until this compile it held no framework for it, only Gartner's second-hand 5–30x agentic multiplier. Two 2026 economics preprints supplied the missing layer: one gives the measurable definition (Cost-of-Pass), the other the theory of why the token is the wrong unit (AI Tokenomics). Both are single-author-or-small-team preprints, not peer-reviewed, and read from their abstract pages only — the measured trajectories are real, the frameworks are definitional.

As of the 2026-07-24 compile the topic's long-missing absolute dated level is finally in. STAGE-Claw (arXiv 2606.10394) measures ~$0.17–$6.55 per task across 11 frontier models on a 40-task state-based agent benchmark — the first concrete 2026 $/task range this KB holds — and WorkBench Revisited (arXiv 2606.13715) adds the two-year trajectory (task completion 43%→98%, cost of a given performance level collapsing at the open-weight tier while frontier costs stayed flat). Both are ingested at abstract grade; STAGE-Claw's specific cost table is search-surfaced, full-text confirmation owed. What the level does not yet cover: a frontier/proprietary per-task cost series, and any workload beyond personal-computing/workplace agents.

The definition — price a correct answer, not a token

Cost-of-Pass (Erol, El, Suzgun, Yuksekgonul & Zou; arXiv 2504.13359; submitted 2025-04-17, revised 2026-02-26) formalizes the unit:

• Cost-of-pass = "the expected monetary cost of generating a correct solution." Operationally, per-attempt inference cost divided by the probability the attempt is correct (accuracy). A cheap-per-token model that is often wrong can carry a higher cost-of-pass than an expensive model that is usually right.
• Frontier cost-of-pass = the minimum cost-of-pass across all available models — or across models and human expert(s). This makes "is the AI economically worth it vs a human, on this task?" a computable question per task type.

The distinction is not cosmetic. Per-token price and cost-per-correct-answer can move in opposite directions: a model can get cheaper per token and more expensive per solved task (if accuracy drops), or dearer per token and cheaper per task (if accuracy rises enough). That decoupling is exactly what the reasoning-model tier does — see below.

The measured findings (Cost-of-Pass, across real models over time)

• Task-type specialization is economic, not just capability. Lightweight models win on basic quantitative tasks; large models win on knowledge-intensive tasks; reasoning models win on complex quantitative problems despite higher per-token cost — because their accuracy lowers cost per correct answer. Evidence: measured.
• Frontier cost-of-pass for complex quantitative tasks "roughly halved every few months." Evidence: measured trajectory; the exact model set and window need the full PDF to date precisely.
• Model-level innovation — not inference-time techniques (sampling, budget-aware decoding) — drives the primary cost-efficiency gains. Budget-aware methods (e.g. TALE-EP) help at the margin, not as the main lever. Evidence: measured.

The measured absolute levels (added 2026-07-24)

Cost-of-Pass and AI Tokenomics gave the topic the unit but no number. Two agentic-benchmark preprints now attach one.

STAGE-Claw (Liang et al., arXiv 2606.10394, 2026-06-09) — a state-based personal-agent benchmark of 40 tasks run across 11 frontier models, scored on the correctness of the final system state rather than the text of the response. Its cost axis is why it matters here:

• Measured API cost per task ≈ $0.17 (cheapest, MiniMax-tier) to ≈ $6.55 (Claude-Opus-4.7-tier) — a ~38x spread across the 11 models on the same 40 tasks. This is the first dated, absolute, per-task dollar level for a real 2026 agentic workload anywhere in this KB. Evidence: measured, but search-surfaced from the paper's cost table (the fetched abstract references "costs" without tabulating them) — full-text confirmation owed; API list-price, personal-computing scenarios only.
• Time cost per task ≈ 3–15 minutes per model (same provenance caveat).

The ~38x model spread is the cost-of-pass thesis in dollars: the expensive-per-task model at the top of the range is the frontier reasoning tier that Cost-of-Pass says you buy because accuracy lowers cost-per-correct-answer — STAGE-Claw prices both ends of that trade for the first time.

WorkBench Revisited (Styles & Miller, arXiv 2606.13715, 2026-06-10) re-runs a fixed workplace-agent benchmark two years on and supplies the trajectory under the level:

• Best-agent task completion 43% (GPT-4, Mar-2024) → 98% (Claude Fable 5, mid-2026); unintended harmful actions 26% → 1.9% — capability and safety moved together, not as a trade-off. Evidence: measured (abstract-grade).
• The load-bearing cost finding: open-weight models collapsed the cost of a performance level that was once proprietary-only, while frontier serving costs stayed flat (the abstract says "drastically lowered"; a ~100× figure has been attributed to the paper body, which has not been read here, so treat the size as unconfirmed; the per-tier $ tables need the full text). This is a bifurcated decline — the same asymmetry recorded on the price-decline distribution, where WorkBench's main treatment lives.

Together: STAGE-Claw gives the absolute level ($0.17–$6.55/task, one date, one benchmark family), WorkBench gives the trajectory and its shape (falling fast at the open-weight tier, flat at the frontier). Neither closes the frontier-proprietary cost series that the frontier file still tracks — the per-task cost is now anchored for agentic benchmarks, not for GPT-5.x / Claude / Gemini serving as a time series.

See also — sibling rungs (context, not this rung's evidence)

Added 2026-09-24 as the honest answer to "what has changed since July". None of these is registered on this rung; each is linked to the rung that holds it.

• harnesses — The Harness Effect (Writer, arXiv 2607.06906, 2026-07-08, ingested 2026-09-11): raw close-read. A controlled swap of the orchestration layer across six models and 22 fixed tasks cut blended cost per task from $0.21 to $0.12 (−41%) and tokens per task from 14.2k to 8.8k, with quality held level. On that workload the harness moved cost per task more than the whole spread of the model menu did. It is the first controlled cost-per-task comparison anywhere in the KB — and it says the biggest lever on a task price may sit above the model, not in the token price. Caveat: the harness is the authors' own product.
• models — Kimi K3 at ~$0.94 per task (Artificial Analysis figure, recorded on Inference & Token Economics from the models rung's Kimi K3 source) — a third-party aggregator already quoting a per-task price, an early sign of the unit shifting.

A published task price — Intercom Fin (added 2026-09-24)

Everything above measures a cost per task. Intercom sells one. Its Fin support agent is priced at $0.99 per outcome, charged "for one outcome per conversation, even if Fin takes multiple actions" (close-read, vendor pages read 2026-09-24; entity: Intercom (Fin)). An outcome is mostly a resolution — the customer confirms the answer helped, or leaves without asking for more — and a resolution the customer later reopens is refunded. One outcome type, a sales qualification, costs $9.99.

What that does to this page's unit:

• Cost-of-pass, turned into a price. Cost-of-pass divides the cost of an attempt by the chance it succeeds. Intercom charges only for the successes, so the buyer pays a price per pass and the seller carries the failed attempts, the retries and the token count. The buyer's side of the sum is fixed at $0.99; all the variation sits on Intercom's side.
• The token price is hidden, and its fall accrues to the seller. When a token gets cheaper (OpenAI's July cuts, Anthropic's cache-read cut, both on this rung) Fin's price does not move. The gap between $0.99 and the token cost of an average outcome is Intercom's margin, and it is not disclosed.
• "Done" is the seller's definition. An assumed resolution is a customer who stopped asking, not one whose problem is confirmed solved. A price per task is only as good as the rule for what counts as a task finished.

Evidence: strong for the price as published; vendor help pages that can change without notice and could not be archived — cite as "as of 2026-09-24". No bet rests on it (brand only).

The theory — token expenditure ≠ economic value

AI Tokenomics (Quanyan Zhu; arXiv 2606.24616; 2026-06-10) proposes "AI tokenomics" as a formal field — "how tokens are generated, consumed, priced, allocated, and optimized" — and lands the load-bearing distinction that reframes the whole margin debate:

• Token expenditure and economic value are distinct quantities. Value depends on marginal productivity, workflow position, hidden reasoning activity (unbilled/internal tokens), risk, and downstream propagation — not on tokens burned.
• Therefore pricing by tokens processed (the universal convention — OpenAI, Anthropic, Google, xAI all do it) systematically prices the input, not the output. Cost-of-pass is the operational correction to that mispricing.
• The paper is explicitly theoretical — no measured figures — and names "hidden-token measurement" and "empirical calibration" as open directions. In doing so its own author concedes the theory has no measured cost/value series: it is a map of the gap, not a filling of it.

Why this is the unit the topic keeps pointing at

1. It is what "cost per task" actually means. MenFem's bet is that the edge sits in the intelligence + harness layer, and that cost per task is the discipline that proves or kills it. Cost-of-pass is the first-principles definition of that number; AI Tokenomics is the argument for why it, not price-per-token, is the thing to optimize.
2. It resolves the reasoning-premium paradox. The price-decline distribution records (via Tiered Super-Moore's Law, arXiv 2603.28576) that reasoning models carry a ~31.5x per-token price premium and resist the price-decline trend. Cost-of-pass explains why that premium is still bought: on hard tasks the accuracy gain lowers cost per correct answer even as cost per token rises. The two sources are complementary — one prices the token, the other prices the pass.
3. It is the variable the token-minimization shift optimizes. Matsuoka's "token maximization → token minimization" regime shift (the cost-measurement problem) is, precisely, a shift to minimizing cost-of-pass rather than maximizing tokens billed. Price/token can fall while cost/task rises; this is the framework that makes that distinction measurable.
4. Two independent sources locate the cost-reduction lever in the model, not the stack. Cost-of-Pass's "model-level, not inference-time" echoes Tiered Super-Moore's "software/architecture, not hardware" (recorded as the software-vs-hardware conflict) — a convergence worth noting, though neither is peer-reviewed.

Key Claims

• Cost-of-pass = per-attempt inference cost ÷ accuracy = the expected cost of a correct solution; frontier cost-of-pass = the min across models or human experts. Evidence: strong as a definition (analytic construct, not an empirical claim) (Cost-of-Pass)
• Frontier cost-of-pass for complex quantitative tasks roughly halved every few months. Evidence: moderate — measured across real models, but abstract-page read; exact window/model-set unread (Cost-of-Pass)
• Reasoning models win on complex quantitative tasks despite higher per-token cost, because accuracy lowers cost-per-correct-answer. Evidence: moderate (measured) (Cost-of-Pass)
• Model-level innovation, not inference-time techniques, drives the primary cost-efficiency gains. Evidence: moderate (measured) (Cost-of-Pass)
• First measured absolute level: a real 2026 agentic workload costs ≈ $0.17–$6.55 per task across 11 frontier models (~38x spread) on the 40-task STAGE-Claw state-based benchmark. Evidence: weak-moderate — the benchmark is measured, but these dollar figures were surfaced by search from the paper's cost table and have never been read here; full-text confirmation still owed as of 2026-09-24; API list-price, personal-computing scenarios only (STAGE-Claw)
• The per-task cost decline is bifurcated: open-weight models collapsed the cost of a once-proprietary performance level (size unconfirmed — ~100× is attributed to the unread body) while frontier serving costs stayed flat; over the same two years best-agent task completion rose 43%→98% and unintended harmful actions fell 26%→1.9%. Evidence: moderate — measured longitudinal benchmark, abstract-grade; per-tier $ tables need the full text (WorkBench Revisited)
• Token expenditure ≠ economic value; per-token pricing prices the input, not the output. Evidence: moderate as a framework claim (single-author preprint, explicitly theoretical, no measured figures) (AI Tokenomics)
• "Hidden reasoning activity" (unbilled internal tokens) is a named value-side cost. Ties to the reasoning per-token premium (0.51–83.3× by quarter, ~31.5× on average, per Tiered Super-Moore's full text) and to a 5–12× reasoning-mode energy draw that appears only in unregistered web literature, not in any source on this rung. Evidence: weak-moderate (theoretical, named not measured) (AI Tokenomics)
• The rung's first published task price: $0.99 per outcome (Intercom Fin), one charge per conversation however many actions the agent takes; a sales qualification is $9.99. The buyer never sees a token. Evidence: strong for the price as published (vendor help article + pricing page, read 2026-09-24); "outcome" is the vendor's own definition (Intercom)
• The theory concedes it has no measured cost/value series — "empirical calibration" is named as open. Evidence: strong (directly stated in the abstract) (AI Tokenomics)

Benchmarks & Data

As of 2026-07-24 the topic holds its first absolute dated cost-per-task level (STAGE-Claw); the older rows remain trajectories or definitions. Every figure carries its as-of date.

Quantity	Value	As of	Nature	Source
Absolute cost per task, 11 frontier models (40-task agentic benchmark)	~$0.17–$6.55 / task (~38x spread)	2026-06-09	measured (search-surfaced cost table; full-text confirmation owed)	STAGE-Claw
Best-agent task completion / harmful actions (workplace agents)	43%→98% / 26%→1.9%	Mar-2024 → mid-2026	measured trajectory (abstract-grade)	WorkBench Revisited
Cost of a fixed performance level, open-weight vs frontier	open-weight "drastically lower" (~100× attributed, unread) / frontier flat	Mar-2024 → mid-2026	direction from the abstract (per-tier $ owed)	WorkBench Revisited
Frontier cost-of-pass, complex quant tasks	"roughly halved every few months"	2025-04 orig / 2026-02 rev	measured trajectory (relative)	Cost-of-Pass
Reasoning per-token premium (cross-ref)	~31.5× average; 0.51–83.3× by quarter	series to 2026-03-28	measured, unstable (Tiered Super-Moore's, full text)	price-decline-distribution
Published price per task (customer-support agent)	$0.99 / outcome ($9.99 / sales qualification)	2026-09-24	list price, vendor-published (not a measured cost)	Intercom Fin
Absolute $/task for a FRONTIER/proprietary model, as a time series	[GAP — still none in KB]	—	—	—

Open Questions

• [MOSTLY ANSWERED 2026-07-24] What is the absolute, dated cost-per-task for a real 2026 agentic workload? STAGE-Claw now supplies it — ~$0.17–$6.55/task across 11 frontier models (search-surfaced, full-text confirmation owed) — and WorkBench Revisited adds the two-year trajectory. What remains open: a frontier/proprietary per-task cost time-series, and coverage beyond personal-computing / workplace agents (coding, research, long-context/reasoning workloads).
• What is the "halved every few months" window and model set precisely? Abstract-page read only; the full PDF is needed to date it.
• Can hidden reasoning tokens be measured? AI Tokenomics names "hidden-token measurement" as open; the topic keeps hitting unbilled internal tokens (reasoning premium, reasoning-mode energy) with no way to count them.
• Does frontier cost-of-pass fall as fast as per-token price? If per-token price falls 50x/yr (Epoch) but accuracy on hard tasks improves more slowly, cost-per-task and price-per-token diverge — and cost-per-task is the commercial number.

Related Concepts

• The Cost-Measurement Problem — cost-of-pass needs a per-attempt cost, which that page shows is unmeasured and utilization-dependent. Cost-per-task inherits the missing denominator.
• The Price-Decline Distribution — prices the token; this page prices the pass. The reasoning-premium paradox lives at the seam between them.
• Token Pricing & the Inference-Margin Question — the umbrella. Cost-per-task is the unit in which "durable margin or commodity?" should ultimately be settled.

Backlinks

Pages that reference this concept:

• The Cost-Measurement Problem
• The Price-Decline Distribution
• Token Pricing & the Inference-Margin Question

Changelog

• 2026-09-24 (source ingest) — Compiled Intercom's Fin outcome pricing, the rung's first published price per task: new section, Key Claim and data row; removed the "proposed, awaiting approval" note. Recorded that Intercom now prices per outcome ($0.99; $9.99 for a sales qualification), not per resolution as the proposal said.
• 2026-09-24 (recompile, no new source) — Re-graded STAGE-Claw's $0.17–$6.55/task to weak-moderate: the figures are still search-surfaced, still unconfirmed two months on. Corrected WorkBench's "~100× since 2024": the abstract says "drastically lowered"; the 100× is attributed to an unread body. Replaced the single 31.5× reasoning premium with its quarterly range from the Tiered Super-Moore's full text. Added dated sibling-rung context (the Harness Effect's controlled −41% cost per task on harnesses) and the pending Intercom Fin proposal, marked as not ingested.
• 2026-07-24 — Compiled STAGE-Claw (2606.10394) and WorkBench Revisited (2606.13715). The topic's long-missing absolute dated cost-per-task level is now in: STAGE-Claw's ~$0.17–$6.55/task across 11 frontier models (search-surfaced from the paper's cost table, full-text confirmation owed) fills the former "[GAP — none in KB]" row; WorkBench adds the two-year trajectory (43%→98% completion, 26%→1.9% harmful actions, open-weight-cheap / frontier-flat cost bifurcation) with its main treatment on the price-decline distribution. Added a measured-levels section, two Key Claims, three data rows, and downgraded the first Open Question to mostly-answered. Both abstract-grade preprints, neither peer-reviewed; the remaining gap is a frontier/proprietary per-task cost time-series.
• 2026-07-23 — Created from two newly compiled preprints (Cost-of-Pass 2504.13359 — the cost-per-task framework; AI Tokenomics 2606.24616 — the value≠cost theory spine). Establishes cost-per-task / cost-of-pass as a first-class concept: the unit the topic repeatedly named as "the one that actually matters." Both read abstract-page only, both preprints, neither peer-reviewed; no absolute dated cost-per-task level yet held.