HBM4 Memory Architecture

HBM4 is the memory breakthrough that unlocks the next generation of AI inference and training — but the bigger story is the architectural shakeup that accompanies it. Three generations matter now: HBM4 (2026), HBM4E (2026–2027), and C-HBM4E (2026–2027). Two structural shifts make this the largest HBM change since the standard was introduced:

1. Base dies move from DRAM to logic processes. HBM4 base dies are manufactured on TSMC 12FFC, N5, or N3P — not traditional DRAM nodes. This makes HBM base dies ~2× more power-efficient than HBM3E's and opens the door to embedding custom logic (C-HBM4E).
2. Interface doubles to 2,048 bits with 32 channels per stack, directly targeting the AI memory wall.

Debuting in NVIDIA's Rubin GPU, HBM4 delivers 22 TB/s per GPU (2.8× over Blackwell's 8 TB/s HBM3e) with 288 GB capacity. HBM4E pushes this to 3 TB/s bandwidth per stack — 2.5× over HBM3E — at up to 12.8 GT/s demonstrated speeds. The Rubin Ultra GPU is spec'd for 1 TB of HBM4E across 8 stacks, yielding potential 16 TB/s per accelerator.

C-HBM4E is the real structural change. It retains standard HBM4E devices but allows custom base dies in three escalating levels: (1) logic integration on the base die, (2) custom die-to-die interface with memory controller on the base die, and (3) near-memory compute — basic processing capabilities inside the memory device itself. NMC requires topology-aware software, runtime, and OS evolution for heterogeneous memory domains — but reframes memory from a passive bottleneck to an active compute substrate. One source calls this "potentially the biggest shift in how computers work in decades."

Key Claims

• 22 TB/s bandwidth per GPU (HBM4, Rubin) — 2.8× over Blackwell. Evidence: strong (NVIDIA Rubin)
• 288 GB capacity per GPU (HBM4, Rubin) — 1.5× over Blackwell's 192 GB. Evidence: strong (NVIDIA Rubin)
• HBM4E: 3 TB/s per stack — 2.5× bandwidth over HBM3E. Evidence: strong (HBM4 Shakeup)
• 2,048-bit interface — doubled vs HBM3E's 1,024. Evidence: strong (HBM4 Shakeup)
• 32 channels per stack — doubled concurrency. Evidence: strong (HBM4 Shakeup)
• 12.8 GT/s demonstrated (Cadence PHY) — above spec. Evidence: strong (HBM4 Shakeup)
• Base dies on TSMC logic processes — 12FFC / N5 / N3P; 2× power efficient vs DRAM-based HBM3E. Evidence: strong (HBM4 Shakeup)
• C-HBM4E with near-memory compute — basic processing inside memory devices; requires topology-aware software. Evidence: moderate (HBM4 Shakeup)
• Rubin Ultra: 1 TB HBM4E, 16 TB/s potential — 8 HBM4 stacks per accelerator. Evidence: strong (HBM4 Shakeup)
• Memory subsystems with 48 TB/s bandwidth possible — with custom D2D interfaces. Evidence: moderate (HBM4 Shakeup)
• HBM3E 12-Hi 36GB carries ~2/3 of 2026 HBM shipments; HBM4 16-Hi 48GB targets Q4 2026 production. Evidence: moderate — January-2026 vendor outlook, now six months stale, and the raw ingest is a search-snippet summary (WebFetch blocked) (SK hynix outlook)
• SK Hynix ~60% HBM share; its 2026 capacity pre-booked by NVIDIA + OpenAI. Evidence: moderate — same January-2026 outlook; supersede with the 2026-07-29 print (SK hynix outlook)
• Memory is 40–50% of accelerator BOM (45–50% for a custom accelerator); HBM took 23% of DRAM wafer output in Q1–Q2'26; DRAM contract prices rose ~90% QoQ in Q1'26. Evidence: weak — TrendForce figures cited inside a single-author preprint; summary-derived twice over, unverified against memory-maker filings (Matsuoka)
• The HBM4 logic base die can serve as a stack-local control plane (cross-layer DMA, address translation, gather/broadcast, inline quantization) — the first C-HBM4E-class design with a named workload. Evidence: moderate — academic architecture proposal, abstract-level only (TokenStack)

Benchmarks & Data

C-HBM4E Customization Levels

Roadmap

Manufacturers

• DRAM: Micron (high-volume HBM4 for NVIDIA Vera Rubin), SK Hynix, Samsung
• Base dies: TSMC (12FFC / N5 / N3P)
• IP: GUC (PHY), Rambus (controller + C-HBM4E guidance), Cadence (12.8 GT/s PHY), Siemens EDA, Synopsys

2026 Supply Mix — SK Hynix Outlook (HISTORICAL, January 2026)

Staleness flag. The source below is a January 2026 vendor-outlook analysis and is now six months old. Treat it as the entry state of 2026, not current posture. SK Hynix reports on 2026-07-29 and Samsung on 2026-07-30 — both dates verified against primary filings this week — and those prints supersede everything in this section.

• SK Hynix at ~60% of the HBM market by share; its 2026 HBM capacity reported pre-booked by NVIDIA + OpenAI — which means new entrants are gated on supply until 2027 regardless of design merit. See SK Hynix.
• Samsung + SK Hynix reportedly raised HBM3E prices ~20% heading into 2026 (Digitimes cross-reference).
• The headline "three market headwinds" in the source title were not captured in the ingest — the raw file records that WebFetch was blocked and its content is a search-snippet summary. That gap is unresolved.

The commercially important read is the mix, not the roadmap: the generation that actually ships the volume in 2026 is HBM3E 12-Hi, not HBM4. HBM4's bandwidth numbers above describe the leading edge; two-thirds of 2026 units are the prior generation.

Memory as a Priced Input, Not Just a Spec

HBM stopped being a bill-of-materials line and became the term that sets AI-buildout economics. The figures below travel with this page but are weakly sourced — they are TrendForce numbers cited inside a single-author preprint, i.e. summary-derived twice over, and none has been verified against a memory-maker filing by this KB:

The consequence developed on Memory Scarcity & Inference Economics: because HBM is "priced by the same three suppliers under the same shortage" regardless of who designs the logic die, a custom accelerator removes merchant GPU margin but not the memory premium. DRAM→HBM wafer reallocation is treated there as the root cause of the broader commodity-DRAM surge — HBM allocation competes with commodity DRAM for the same wafer starts.

The Base Die Becomes a Controller

The HBM4 logic base die was introduced above as a power/efficiency story. The first concrete architecture to treat it as a control plane is TokenStack (arXiv:2605.05639), which uses the HBM4 base die as a stack-local controller handling cross-layer DMA, layered address translation, attention-side gather/broadcast coordination, and inline quantization during migration — with the stack split into dense capacity layers and PIM-enabled compute layers. That is C-HBM4E level-3 near-memory compute with a named workload attached (hot KV-cache attention) rather than a roadmap slot. See KV Cache Management and Processing-In-Memory. It is an academic proposal, not a product.

Analyst View — Memory Shortage (Attributed, Unverified)

Dylan Patel (SemiAnalysis), on Podcast Alpha (2026-07-10), characterizes the current memory tightness as structural rather than cyclical: "Memory is not a cycle top. It is a multi-year structural shortage." He frames the driver as a growth-rate mismatch — memory capacity grows 20-30%/year while AI demand doubles (~100%/year). Evidence: weak — Patel claims/estimates, unverified, no primary data cited, disclosed conflict (SemiAnalysis is a reported Anthropic enterprise customer) (Dylan Patel / Podcast Alpha).

This is directionally consistent with this page's own capacity/demand claims (HBM4/HBM4E roadmap tightness, TSMC logic-process base-die bottleneck) but Patel's specific growth-rate figures (20-30%/yr vs ~100%/yr) are his own estimate, not sourced to a named dataset — treat as a claim to test against primary capacity data (Micron/SK Hynix/Samsung disclosures), not as a settled number.

Open Questions

• What do the SK Hynix (2026-07-29) and Samsung (2026-07-30) prints say about HBM4 mix, ASP and capacity — and how far has the January outlook drifted?
• Do the ~90% DRAM price rise / 23% HBM wafer share figures hold up against primary memory-maker disclosures, or are they TrendForce estimates that have been laundered into fact by repetition?
• What are HBM4 yield rates and cost premiums versus HBM3e?
• Does Patel's 20-30%/yr capacity-growth-vs-~100%/yr-demand-growth framing hold up against primary DRAM-maker capex/output disclosures?
• Does C-HBM4E near-memory compute find a real workload beyond simulation benchmarks?
• Can ASIC vendors (TPU, Trainium) get HBM4/HBM4E at competitive timelines vs NVIDIA?
• What programming models emerge for NMC — CUDA extensions, or a new stack entirely?
• Does 1 TB HBM4E per accelerator change model-parallelism tradeoffs enough to shrink racks?

Related Concepts

• Rack-Scale AI Compute — HBM4 bandwidth enables rack-scale serving
• Custom Silicon vs GPU — memory access is a key differentiator in the ASIC vs GPU debate
• Nanosheet GAA Transistor — logic process nodes (N3P) are the new home for HBM base dies
• Processing-In-Memory — C-HBM4E level-3 NMC is the commercial wedge for memory-centric computing
• KV Cache Management — hot KV attention is the first named workload for base-die compute
• Memory Scarcity & Inference Economics — HBM price is the exogenous input driving those cost models
• Advanced Packaging & CoWoS Capacity — the second capacity gate on the same accelerator

Backlinks

Pages that reference this concept:

• NVIDIA Vera Rubin Platform
• Custom Silicon Inflection 2026
• HBM4 Architectural Shakeup
• SK hynix 2026 HBM Outlook
• TSMC
• SK Hynix

Changelog

• 2026-07-22 — Added the 2026 supply mix from the SK hynix January outlook (HBM3E 12-Hi ~2/3 of shipments, HBM4 16-Hi Q4'26, ~60% share, capacity pre-booked) with an explicit six-months-stale flag pointing at the 2026-07-29/30 prints; added the "memory as a priced input" block (BOM share, DRAM price rise, HBM wafer share) labelled weak/twice-summary-derived; added the HBM4 base-die-as-controller reading from TokenStack. +3 sources.
• 2026-07-14 — Added Patel/Podcast Alpha attributed "multi-year structural shortage" claim (capacity +20-30%/yr vs AI demand doubling) as an Analyst View — unverified, flagged for cross-check against primary DRAM-maker data.
• 2026-04-09 — Initial compilation from NVIDIA Rubin + SemiAnalysis.
• 2026-04-17 — Major update: added HBM4E (2.5× bandwidth, 2,048-bit interface, 32 channels), C-HBM4E (3 customization levels including near-memory compute), and TSMC logic-process base dies. Cross-linked to Nanosheet GAA Transistor.