Hardware
The silicon rung of the inference stack, read memory-first.
Memory bandwidth of one H100 SXM's 80 GB of HBM3 — the wall the arithmetic waits behind
In scope: the memory hierarchy and the memory wall, HBM and DRAM scaling, bandwidth and capacity limits, processing-in-memory, accelerator and NPU architecture, advanced packaging, and interconnect including optical interconnect and co-packaged optics. Memory is the binding constraint of this cycle. Out of scope: photonic compute (computing in light, as distinct from communicating in light) and quantum computing — frontier science with no near-term effect on what a token costs — and consumer buying advice, which is an editorial beat rather than research.
| Type | Source | Published |
|---|---|---|
| PAPER | Memory Scarcity, Open Models, and the Restructuring of the AI Industry, 2026-2030 Satoshi Matsuoka · RIKEN Center for Computational Science (R-CCS) Quantitative scenario model showing the 2026 DRAM/HBM price surge creates a persistent entrant-incumbent inference-cost gap that never closes over 2026-2030 (3.2x in 2026, narrowing to ~1.9x in 2027, re-widening to 3-4x by 2029-30) via a depreciation-conveyor mechanism; five probability-weighted scenarios (Rotating Landlord Oligopoly 25%, Commoditization Crash 25%, Jevons Absorption 20%, System-Layer Re-differentiation 18%, Geopolitical Bifurcation 12%) map the resolution. | 2026-07-08 |