Hardware & Computing

AI compute infrastructure in 2026 is defined by two simultaneous paradigm shifts. On the classical side, the unit of compute is no longer the GPU chip but the rack — NVIDIA's Vera Rubin (50 PFLOPS FP4 per GPU, 288 GB HBM4 at 22 TB/s, 72-GPU racks at 260 TB/s aggregate) set the ceiling and entered full production in May 2026. The foundry substrate shifted beneath it: TSMC N2 (2nm) in mass production Q4 2025, A16 (1.6nm) with backside-power Super Power Rail due late 2026. HBM underwent an architectural shakeup — base dies moved from DRAM to TSMC logic processes, enabling HBM4E (2.5× bandwidth) and C-HBM4E (near-memory compute). On the photonic side, three anchor results (imec, Nokia Bell Labs, Lightmatter) turned photonic compute from research curiosity into production substrate.

The story that moved in mid-2026 is that the constraint migrated up the stack, twice. It went compute (2018–23) → memory (2024–25) → packaging and lithography economics (2026), and then memory scarcity stopped being a supply problem and became a market-structure problem. Three things now gate an accelerator, and none of them is transistor density: an interposer slot (NVIDIA reportedly holds ~60% of TSMC CoWoS, ramping toward a 120–130K wafers/month target), HBM supply (2026 capacity reported pre-booked; HBM3E 12-Hi still carries ~2/3 of 2026 shipments, not HBM4), and grid power (12 GW of US data-center capacity announced for 2026 against ~5 GW under construction — a 7 GW shortfall). The discipline that unites all three: announced ≠ under construction ≠ delivered.

Alongside that, the serving layer got its own reframing: the KV cache is now a first-order memory object, not a per-request tensor — which is why decode is memory-bound, why processing-in-memory finally has a candidate killer workload (hot-KV attention), and why "which accelerator is faster" is not a well-formed question until you say which phase. The first independent phase-separated benchmark in this KB finds GPUs win compute-bound prefill while a non-GPU rack wins memory-bound decode at small batch.

On the quantum side, four anchor results define the landscape — Google Willow (below surface-code threshold, Λ=2.14, d=7 at 0.143% error/cycle), IBM Nighthawk + qLDPC (real-time decoding <480ns, 2029 fault-tolerance target), Quantinuum's iceberg codes (94 logical qubits from 98 physical, beyond break-even), and Besedin et al. (first superconducting lattice surgery). Google's April 2026 QuEra investment made neutral atoms a formal third architecture.

The custom-silicon insurgency is measurable in shipments: ASICs at 27.8% of 2026 AI server shipments, GPUs 69.7%, with a Broadcom ~60% / Marvell ~35% co-design duopoly and Anthropic's ~1M-TPU / >1 GW commitment as the demand anchor. But the newest evidence cuts against the simplest version of that story — a custom program removes NVIDIA's merchant margin and not the memory premium (HBM is 45–50% of a custom accelerator's BOM, priced by the same three suppliers), and it still queues behind NVIDIA for the same interposers.

Standing provenance caveat. Much of the inference-economics material on this KB derives from a single-author, non-peer-reviewed preprint (Matsuoka, arXiv:2607.07207) plus the desk's close read of it. Its numbers are model outputs under stipulated parameters, its DRAM/HBM inputs are TrendForce figures cited second-hand, and it contains no supplier-level memory-maker analysis at all. It is an argument to test, not a factual backbone. On memory makers, the picture split in two on 2026-07-23: the KB now holds its first primary memory-maker technical source — SK Hynix's StreamDQ (arXiv:2607.08993), which closes the gap on the architecture / near-memory axis — but still holds zero primary memory-maker supply/pricing sources, the load-bearing axis. That closes, or doesn't, at the SK Hynix (2026-07-29) and Samsung (2026-07-30) prints.

Frontier — What's Moving Now

• Memory scarcity as market structure — $/PB cost unit, the depreciation conveyor (entrant/incumbent gap 3.2× → 1.9× → 3–4×, never closing), and a 16-cell vintage-breakeven grid in which only 2027 is robust across all regimes.
• KV cache as a first-order memory object — 30+ systems, five archetypes; ownership explains most remaining design variance; seven evaluation measurements missing field-wide. C²KV now shrinks-and-reuses the object itself (composable + compressed non-prefix reuse, up to 17× long-context speedup).
• Phase-aware inference — GPUs win prefill, non-GPU racks win small-batch decode, GPUs regain decode at scale. Architecture advantage is phase-dependent.
• The base die stops being passive — and a memory maker joins the bet — three 2026 near-memory works span the spectrum: TokenStack adds PIM alongside the GPU, StreamDQ (SK Hynix — the KB's first primary memory-maker source) offloads weight dequant onto the base die (7.08× GEMM), and AMMA removes the GPU compute die from decode attention entirely (15.5× lower attention latency vs H100). Narrows the memory-maker gap on architecture, not supply.
• Three capacity gates, one accelerator — CoWoS interposers, HBM supply, grid interconnection.
• HBM3E, not HBM4, ships the 2026 volume — ~2/3 of shipments; HBM4 16-Hi was targeted Q4 2026.
• Custom silicon can't design around the memory premium — best-case custom build ~$0.082/PB vs a $0.022–0.037/PB depreciated-incumbent floor.
• Custom-silicon inflection measurable in shipments — ASICs 27.8% of 2026 AI servers; Broadcom ~60% / Marvell ~35% co-design duopoly.
• Vera Rubin in full production — confirmed May 2026, shipments fall 2026; Rubin CPX cancelled, Rubin Ultra (2027) stays.
• TSMC Angstrom era + lithography economics — N2 in volume (3% of Q2'26 wafer revenue), A16 late 2026, high-NA EUV deferred to ≥2029 on cost.
• Quantum: three architectures, same 2029 target — superconducting, trapped-ion, neutral atom.
• Photonic compute crossed to production — imec, Nokia Bell Labs, Lightmatter.

Concept Map

Quantum and photonic clusters are omitted from this map for readability — see the concept table.

Concepts

Entities

Timeline

See timeline.md for chronological developments.

Research Frontier

See frontier.md for active research directions, breakthroughs, and knowledge gaps.

Sources

Sources 28–30 were registered from a discovery sweep with abstracts verified against arXiv but full text never ingested — there is no raw/*.md file for them. Full-text ingest is a Tier 1 knowledge gap. Source 32 is the desk's own close read of source 31 and is authoritative wherever the two differ. Sources 33–35 (StreamDQ, AMMA, C²KV) have raw files but are preprints with authors' own simulated/abstract-level results — none independently replicated; StreamDQ (SK Hynix) is the KB's first primary memory-maker source but architecture-only. Source 36 is an analysis-tier projection with an explicit as-of date (2026-06-15), not realized output.