Rack Power Density

Rack power density is the amount of electrical power (and therefore heat) concentrated into a single server rack. The shift to Nvidia's GB300 NVL72 rack-scale systems is the density step-change driving the modern AI datacenter: the NVL72 is a single rack integrating 72 Blackwell GPUs as one unit, and these racks are now the atomic build-unit of the largest AI factories. The NVIDIA / DOE source quantifies the deployment scale — Lambda's AI Factory in Kansas City is 100+ MW with 10,000+ GB300 NVL72 GPUs, and Global AI's New York site is 128 GB300 NVL72 racks (9,000+ GPUs) — but it describes the systems in GPU-count and total-megawatt terms, not in per-rack kilowatts.

The reason density matters for the buildout is thermal. As power per rack climbs into a regime where air cannot remove the heat, liquid cooling becomes mandatory rather than optional — which is the mechanism that re-rates the power-and-cooling supply chain (Vertiv, Eaton, GE Vernova, Schneider Electric, and the rest of the NVIDIA-named vendor list). Higher density also concentrates the power problem: the same gigawatts of campus demand arrive in fewer, hotter racks, tightening the link between this concept and Power: The Binding Constraint. The frontier debate this feeds — "does liquid cooling become mandatory and therefore Vertiv-et-al a secular winner, or stay niche?" — turns on exactly how high NVL72-class density goes.

RESOLVED 2026-07-23 — the kW/rack figure is now sourced, closing the topic's oldest gap. A Lenovo vendor product guide for the GB300 NVL72 (doc updated 2026-07-15) states 135 kW rack TDP, up to ~155 kW peak (50V DC rail; 72 B300 GPUs + 36 Grace CPUs), with heat captured ~90% liquid / ~10% air. This is the first primary vendor datasheet in the topic and fills the gap that had been open across the prior 13 sources. Provenance: technical-report-tier, but a vendor document — authoritative for the spec, not independent. Cross-vendor corroboration from the same search (not ingested): HPE QuickSpecs cited 132–140 kW with EDPp provisioning up to ~192 kW; the Lenovo 135 kW TDP / 155 kW peak sits inside that band. At 135 kW nominal a single NVL72 rack draws roughly 18–19× a legacy ~7 kW enterprise rack — the physical reason the 90/10 liquid split is forced, not optional. (Note: the older "~120–140 kW" number previously cited secondhand is now corroborated by a primary vendor spec; the "peak up to 155 kW" is a ceiling, not a sustained draw.)

Key Claims

• GB300 NVL72 is the rack-scale build-unit of the largest 2025–26 AI factories. Evidence: moderate (technical-report — NVIDIA) (NVIDIA — Building America's AI Infrastructure)
• Lambda AI Factory (Kansas City): 100+ MW, 10,000+ GB300 NVL72 GPUs. Evidence: moderate (technical-report — NVIDIA) (NVIDIA — Building America's AI Infrastructure)
• Global AI (New York): 128 GB300 NVL72 racks (9,000+ GPUs). Evidence: moderate (technical-report — NVIDIA) (NVIDIA — Building America's AI Infrastructure)
• DOE supercomputers anchor on Blackwell at scale: Solstice 100,000 Blackwell GPUs (DOE's largest), Equinox 10,000 Blackwell (2026), combined 2,200 exaflops AI; xAI Colossus 2 (Memphis) 500,000+ GPUs. Evidence: moderate (technical-report — NVIDIA) (NVIDIA — Building America's AI Infrastructure)
• Per-rack kW figure is NOT in any ingested source — RESOLVED 2026-07-23. GB300 NVL72 is 135 kW rack TDP, ~155 kW peak per a Lenovo vendor product guide (as-of doc date 2026-07-15). Evidence: moderate (technical-report — vendor datasheet, not independent) (Lenovo GB300 NVL72)

Added 2026-09-24 — the population behind the 135 kW rack

• The typical rack passed 11 kW for the first time in 2026 (average of each site's most common rack density, n = 679), up from 9 kW in 2025. Most of the jump comes from a few new sites whose typical rack is above 30 kW; without them the average is 7.8 kW, up from 7.5 kW. The most common rack design, 4–5 kW, has not changed in over a decade. Evidence: moderate (self-reported operator survey) (Uptime 2026)
• 24% of sites now have some racks at 30 kW or more, up from 19% in 2025 (n = 676); about one in seven have some at 50 kW or more, including racks above 100 kW. 76% still have none at 30 kW. Evidence: moderate (self-reported survey) (Uptime 2026)
• AI inference now drives dense racks as often as AI training — 21% of respondents name each, the first time they are level. Inference is what serves tokens, so the dense rack is no longer a training-only object. Evidence: moderate (self-reported survey) (Uptime 2026)
• So the GB300 NVL72 is an outlier by more than an order of magnitude: at 135 kW it is roughly 12× the 2026 typical rack (11 kW) and 17× the typical rack outside the high-density sites (7.8 kW). The earlier "~18–19× a legacy ~7 kW rack" had no source for its 7 kW baseline; the survey now supplies one (7.8 kW outside the high-density sites), which puts the ratio nearer 17×, and against the all-site typical rack of 11 kW it is about 12×. Evidence: moderate (arithmetic on a vendor spec and a survey average)

Added 2026-07-23 (closes the topic's oldest gap)

• GB300 NVL72 rack power: 135 kW TDP, up to ~155 kW peak (50V DC rail; power shelves 6–8 × 33 kW = 198–264 kW installed headroom above nominal). Evidence: moderate (technical-report — Lenovo vendor guide, as-of 2026-07-15) (Lenovo GB300 NVL72)
• Compute per rack: 72 × B300 (Blackwell Ultra) GPUs, 186 GB HBM3e each + 36 × Grace CPUs. Evidence: moderate (technical-report — vendor guide) (Lenovo GB300 NVL72)
• Heat capture ~90% liquid / ~10% air at the rack — CPUs, GPUs, HBM, NVSwitch and NICs are liquid-cooled; only OSFP modules, drives and PDUs stay air-cooled. This is the sourced mechanism behind "liquid cooling becomes mandatory." Evidence: moderate (technical-report — vendor guide) (Lenovo GB300 NVL72)
• A GB300 NVL72 rack draws ~18–19× a legacy ~7 kW enterprise rack (baseline sourced 2026-09-24: ~17× the 7.8 kW typical rack outside high-density sites, ~12× the all-site 11 kW — see above), and facility provisioning runs above nameplate (HPE cross-ref ~192 kW busway for EDPp headroom). Evidence: moderate (derived from vendor spec + cross-vendor cross-reference) (Lenovo GB300 NVL72)
• ~7.4 racks per MW at 135 kW nominal — the unit that ties rack density to the capex/MW and grid-queue figures. Evidence: moderate (arithmetic on the vendor spec) (Lenovo GB300 NVL72)

Added 2026-07-22

• Cooling architecture is visibly changing generation-over-generation: Tesla's Cortex 2 doubles Cortex 1's cooling capacity (two chiller plants, ≥12 fan units) and moves cooling equipment inside structures rather than siting it at ground level — an independent, non-NVIDIA datapoint that density is forcing physical redesign, not just vendor selection. Evidence: weak (single analysis source, one site) (Tesla Cortex 2)
• Density is now supply-gated as well as thermally gated: NVL72-class rack deployment depends on CoWoS advanced packaging, reported sold out through end-2026 with lead times into 2027. Density can be designed for faster than racks can be delivered. Evidence: weak (single analysis source) (TSMC Q2 2026)
• kW/rack remains a gap after this compile. (True as of 2026-07-22; closed 2026-07-23 by the Lenovo GB300 NVL72 vendor spec — 135 kW TDP / 155 kW peak. See the 2026-07-23 block above.)

Benchmarks & Data

• GB300 NVL72: 72 Blackwell GPUs per rack (rack-scale system)
• Lambda AI Factory: 100+ MW, 10,000+ GB300 NVL72 GPUs
• Global AI (NY): 128 GB300 NVL72 racks, 9,000+ GPUs
• Solstice: 100,000 Blackwell GPUs (DOE's largest AI supercomputer)
• Equinox: 10,000 Blackwell GPUs (2026); Solstice+Equinox = 2,200 exaflops AI
• xAI Colossus 2 (Memphis): 500,000+ GPUs
• kW/rack (GB300 NVL72): 135 kW TDP, ~155 kW peak — filled 2026-07-23 (Lenovo vendor guide); ~12× the 2026 typical rack of 11 kW (Uptime); ~7.4 racks/MW
• Industry typical (modal) rack, 2026: >11 kW average (9 kW in 2025; 7.8 kW excluding high-density sites); 24% of sites have some racks ≥30 kW (Uptime, self-reported)
• GB300 NVL72 compute: 72 × B300 GPUs (186 GB HBM3e each) + 36 × Grace CPUs; ~90% liquid / ~10% air cooling; 50V DC rail

Added 2026-07-22:

• Tesla Cortex 2 cooling: 2 chiller plants, ≥12 fan units (~2x Cortex 1); cooling housed inside structures
• Tesla Cortex 2 campus power: 500MW
• NVL72 rack supply gate: CoWoS sold out through end-2026, lead times into 2027

Open Questions

• What is the actual kW/rack for GB200/GB300 NVL72? ANSWERED 2026-07-23: 135 kW TDP / ~155 kW peak (Lenovo vendor guide). Remaining refinement: an NVIDIA-primary or independent-lab figure on sustained vs peak draw, and the GB200-vs-GB300 delta (this source is GB300-specific).
• Is the density step-change relieved or worsened by the packaging constraint? If racks are supply-limited, operators may build for higher density per delivered rack, pushing kW/rack up faster than the cooling supply chain expects.
• Does liquid cooling become mandatory (making the cooling vendors a secular winner) or stay niche? (Frontier debate #3.)
• At what power-per-rack threshold does air cooling become physically infeasible?
• How does rising density change capex-per-MW economics for plant and cooling?

Related Concepts

• Power: The Binding Constraint — density concentrates the same gigawatts into fewer, hotter racks
• Hyperscaler Capex Cycle — the ~$85B networking/cooling slice of plant capex is what density pulls upward
• Vertically Integrated Compute — Cortex 1 → Cortex 2 is a documented density-driven cooling redesign
• Advanced Packaging Bottleneck — NVL72 racks cannot be deployed faster than CoWoS delivers them
• Power Usage Effectiveness — denser racks force liquid cooling, which moves both PUE and water use

Changelog

• 2026-06-24 — Created from 1 source (nvidia-doe-ai-infrastructure)
• 2026-07-22 — Added Tesla Cortex 2 as an independent non-NVIDIA datapoint on density-driven cooling redesign, and the CoWoS supply gate on NVL72 deployment. kW/rack gap re-confirmed as still unfilled across all 10 sources.
• 2026-07-23 — Closed the topic's oldest gap: GB300 NVL72 rack power = 135 kW TDP / ~155 kW peak from the first primary vendor datasheet (Lenovo product guide, 2026-07-15), with the ~90% liquid / ~10% air split, 72×B300 + 36×Grace compute, and the derived ~18–19× legacy-rack ratio and ~7.4 racks/MW. Provenance flagged as vendor-primary (authoritative for spec, not independent).
• 2026-09-24 — Added the industry population from Uptime's 2026 survey (typical rack >11 kW, 24% of sites with some racks ≥30 kW, inference level with training as a density driver). Gave the old "~18–19× a legacy ~7 kW rack" comparison a sourced baseline: ~17× the 7.8 kW typical rack outside high-density sites, ~12× the all-site 11 kW.