Hardware
The silicon rung of the inference stack, read memory-first.
Memory bandwidth of one H100 SXM's 80 GB of HBM3 — the wall the arithmetic waits behind
In scope: the memory hierarchy and the memory wall, HBM and DRAM scaling, bandwidth and capacity limits, processing-in-memory, accelerator and NPU architecture, advanced packaging, and interconnect including optical interconnect and co-packaged optics. Memory is the binding constraint of this cycle. Out of scope: photonic compute (computing in light, as distinct from communicating in light) and quantum computing — frontier science with no near-term effect on what a token costs — and consumer buying advice, which is an editorial beat rather than research.
| Type | Source | Published |
|---|---|---|
| ANALYSIS | Custom Silicon Inflection 2026: Hyperscaler ASICs vs NVIDIA GPU Wega Chu, Dylan Patel, Daniel Nishball et al. · SemiAnalysis Deep technical analysis arguing NVIDIA's rack-as-product co-design strategy deepens lock-in despite custom ASIC growth at 44.6% CAGR | 2026-02-25 |
| ANALYSIS | The Angstrom Era Arrives: TSMC Enters 2nm Mass Production and Unveils 1.6nm Roadmap TokenRing Research · TSMC / Industry Analysis TSMC N2 hit mass production Q4 2025 (65-75% yield, Apple got >50% of capacity); A16 (1.6nm) with backside-power Super Power Rail targets late 2026 volume — first departure from FinFET in a decade | 2026-01-21 |
| ANALYSIS | HBM Architectural Shakeup: HBM4, HBM4E, C-HBM4E — 3nm Base Dies Enable 2.5x Performance Anton Shilov · Tom's Hardware / TSMC / GUC HBM base dies move from DRAM to 3nm logic (TSMC N3P) — enables 2.5x bandwidth (3 TB/s per stack), 2x channels, C-HBM4E adds custom base dies with near-memory compute | 2025-12-02 |
| ANALYSIS | Memory-Centric Computing: A Paradigm Shift for Sustainable and Efficient Systems Onur Mutlu (ETH Zurich) — synthesis of public talks and papers · ETH Zurich / SAFARI Group Argues processor-centric computing is fundamentally broken: 60–90% of system energy is data movement, not compute; DRAM can compute (RowClone, Ambit); reliability threats (RowHammer, RowPress, column disturbance) force memory intelligence anyway; JEDEC + trillion-dollar incumbency is the bottleneck, not technology | 2026-04-21 |
| ANALYSIS | TSMC to Quadruple Advanced Packaging Capacity: Reaching 130,000 CoWoS Wafers Monthly by Late 2026 TokenRing / FinancialContent · FinancialContent TSMC CoWoS ramp: ~35K wafers/mo (late 2024) → 75K (end 2025) → 130K target (end 2026); NVIDIA secures ~60% of capacity. | 2026-02-05 |
| ANALYSIS | SK hynix 2026 Outlook: HBM3E Dominates, HBM4 Dual Strategy Amid 3 Market Headwinds TrendForce · TrendForce HBM3E 12-Hi 36GB ≈ 2/3 of 2026 HBM shipments; HBM4 16-Hi 48GB targets Q4 2026; SK Hynix ~60% share, 2026 capacity pre-booked by NVIDIA + OpenAI. | 2026-01-05 |
| ANALYSIS | Global AI Server Shipments Forecast to Grow Over 28% YoY in 2026, with a Rising Share of ASIC-Based Systems TrendForce · TrendForce TrendForce: 2026 AI server shipments +28% YoY; ASIC-based systems reach 27.8% of shipments (highest since 2023), GPUs 69.7%; top-5 NA CSP capex +40% YoY; Google TPUs sold externally (Anthropic). | 2026-01-20 |
| ANALYSIS | The Custom AI ASIC State of Play (May 2026) — Broadcom Deals, Google TPUs, Meta MTIA & Beyond Tom's Hardware · Tom's Hardware May-2026 ASIC state-of-play: Broadcom ~60% / Marvell ~35% of the custom-AI-ASIC co-design market; Marvell ~$11B AI ASIC revenue 2026; programs — TPU v7 Ironwood (4,614 FP8 TFLOPS, 192GB HBM3E, 7.37 TB/s), Maia 200, Trainium3 (Q2 2026), MTIA, OpenAI Titan. | 2026-05-26 |
| ANALYSIS | Close read: Vintage Breakeven (§6) and the Demand-Measurement Critique (§9.1/§9.3) — Matsuoka 2607.07207 MenFem Research Desk · MenFem Primary-text close read of the full 22-page PDF. Corrects the parent ingest on four points: author is RIKEN R-CCS not independent; vintage breakeven is a 16-cell grid (4 vintages x 2 regimes x 2 HBM branches) not 3 rows; the 2026 vintage's regime exposure was INVERTED (worse under sticky ~31.3%, not coupled ~24.5%); custom-silicon build cost is $0.082/PB not $0.072. AUTHORITATIVE wherever it differs from the parent. | 2026-07-20 |
| ANALYSIS | TSMC CoWoS Supply-Demand Gap Reportedly Narrowing from 20% to 10% by End-2026 as Capacity Expands TrendForce (citing Economic Daily News, Reuters, Commercial Times) · TrendForce June-2026 CoWoS refresh (projection, as-of 2026-06-15): TSMC monthly capacity 120K-140K wafers by end-2026 (industry ~200K incl. 50-60K OSAT); supply-demand gap narrowing ~20% to ~10% by end-2026, shortage moderating in 2027; >60% TSMC expansion by 2027, CAGR >80% 2022-27; CoPoS pilots mid-2027, MP 2028-29 with NVIDIA Feynman first customer. | 2026-06-15 |