Rung 09 Networking & InterconnectSwitch rungClose
Networking & Interconnect
How chips and sites talk to each other.
Fourth-generation NVLink bandwidth per H100 GPU — what one accelerator can say to its neighbours

In scope: optical interconnect and co-packaged optics, scale-up versus scale-out fabrics, NVLink, InfiniBand and Ethernet economics, bandwidth per dollar, topology, cross-datacenter links. Out: memory bandwidth inside a package (hardware), photonic compute (out of the knowledge base entirely).
This rung asks how chips, racks and sites talk to each other, and what that costs: optical links
and co-packaged optics, the fabrics inside a rack and between racks, NVLink versus InfiniBand
versus Ethernet, bandwidth per dollar, and links between data centres. Memory bandwidth inside a
chip package belongs to hardware. Computing with light is out of the knowledge base entirely;
communicating with light is in.
What this rung prices
A data centre has a fixed power feed. Every watt spent moving bits between chips is a watt not spent on the chips doing the work, so the power cost of each bit moved sets how much of the feed reaches compute, and with it how many tokens a site can serve. The rung's three in-scope sources all make a claim on exactly that term. Broadcom and NVIDIA, the two largest switch vendors, each state that putting the optics inside the switch package ("co-packaged optics") uses about 3.5x less power for the optical links than plugging optical modules into the front of the switch (Broadcom Tomahawk 6 – Davisson; NVIDIA Spectrum-X / Quantum-X Photonics). NVIDIA's technical blog then gives the watts: pluggable optics "often 30W per interface", co-packaged "as low as 9W" (NVIDIA CPO blog). That is about 3.3x on the vendor's own numbers (our arithmetic), a typical case against a best case, on switches not shipping when it was written. It is the first figure on the rung that could be carried towards a cost per token, once someone says how many switch ports a site runs per GPU.
The load-bearing findings
- Two vendors land on about 3.5x, and the agreement is the only reason to use the number. Broadcom's BCM78919 switch moves 102.4 Tb/s through 16 optical engines of 6.4 Tb/s each at 200 Gb/s per link, with a 70% cut in optical link power, "more than 3.5x lower than traditional pluggable solutions". The 70% and the 3.5x are the same claim stated twice, not two pieces of evidence, and the saving is scoped to the optical links, not to the switch or the whole system (Broadcom). NVIDIA's release states 3.5x power efficiency, 4x fewer lasers and 1.3x faster deployment against pluggable optics (NVIDIA).
- The watts behind the multiplier, and why it is 3.3x rather than 3.5x. NVIDIA's blog gives "often 30W per interface" for pluggables and "as low as 9W" for co-packaged optics, and electrical loss of up to 22 dB against about 4 dB. 30 ÷ 9 is about 3.3x, and Broadcom's "70%" is also 3.3x; both vendors headline 3.5x without showing how. The 22 dB to 4 dB gap is 18 dB, a power ratio of about 63x — which is exactly where the "63x" / "64x" signal-integrity figures come from. Corrected 2026-09-24: this page and the NVIDIA release's close read had treated that multiplier as a marketing composite with no unit; it is a unit conversion of a best-case loss figure (NVIDIA CPO blog). "Interface" is not defined, so a per-bit figure (about 37 against 11 pJ/bit if it means an 800 Gb/s port) is unconfirmed.
- Neither part was shipping in volume on the date of its source. Broadcom's part was sampling at announcement (2025-10-08). NVIDIA's was a roadmap: InfiniBand "later in 2025" and Ethernet in 2026, announced 2025-03-18. Five months later NVIDIA's blog restated those as early 2026 and the second half of 2026, a slip. Both compare against an unnamed pluggable baseline. Sampling is not shipping, and a roadmap is not a measurement.
- The scale these switches claim. Broadcom: 512 accelerators in a single scale-up cluster and 100,000+ in a two-tier scale-out network. NVIDIA: Ethernet configurations of 100 Tb/s and 400 Tb/s, and a 144-port 800 Gb/s InfiniBand switch (Broadcom; NVIDIA).
- The supply chain is named by the vendor. NVIDIA's own partner list for these switches includes Lumentum, alongside TSMC, Coherent, Corning, Fabrinet, Foxconn and others. That is the vendor naming the supplier, a stronger kind of evidence than an analyst saying so (NVIDIA). It bears on the rung's one thesis, which has a live position behind it — see theses.
- Inside the rack, the concept page carries NVIDIA's platform figures. The
silicon photonics page, compiled from an NVIDIA technical
blog held on the
hardwarerung, records 102.4 Tb/s per Spectrum-6 switch and 260 TB/s of aggregate bandwidth across the 72 GPUs of an NVL72 rack. Its "64x signal integrity" figure and the release's "63x" are the same 22 dB to 4 dB loss gap, now explained on the concept page. - Two of the four sources are about computing with light, which is out of scope. A photonic tensor processor from the imec platform ran pretrained networks on a 9 × 3 optical crossbar (photonic tensor processor), and Nokia Bell Labs trained a small photonic network on the chip itself (2 inputs, 8 hidden, 1 output; 92.5% on a 2D classification task) (on-chip backprop). They came to this rung with the rescued thesis. They are counted here for accuracy, not for coverage.
Concept map
Concepts
| Concept | Sources | Evidence | Last Updated |
|---|---|---|---|
| Silicon Photonics | 4 (0 papers) | Moderate; all vendor sources | 2026-09-24 |
Entities
| Entity | Type | Sources |
|---|---|---|
| Nokia Bell Labs | Lab | 1 |
The Silicon Photonics page now carries the co-packaged-optics power figures. The Nokia Bell Labs page is about photonic computing, the out-of-scope half.
What we do not know yet
- Power per bit, independently measured. The only absolute figures are NVIDIA's own (30 W against 9 W per "interface", undefined). No picojoules-per-bit figure is stated, and no one but the vendors has measured anything.
- Whether either part shipped in volume, on time, at the stated figures. NVIDIA's own dates had already slipped once by August 2025. A later Broadcom volume claim was noted but not verified, because the product pages would not load.
- Whether co-packaged optics replaces pluggable modules or adds optical content per system. Broadcom's release is evidence for replacement at the switch face; nothing here measures the total.
- Most of the scope. Scale-up versus scale-out economics, NVLink against InfiniBand against Ethernet on cost, bandwidth per dollar, topology and links between sites all lack a source.
- Standing: read it. Every figure here is vendor-stated and read, not measured.
Timeline
See the timeline for chronological developments, the open questions for the open questions, and the theses for the rung's one thesis.
Sources
| # | Title | Type | Date | Status |
|---|---|---|---|---|
| 1 | NVIDIA Spectrum-X Photonics and Quantum-X Photonics | news | 2025-03-18 | compiled |
| 2 | Scaling AI Factories with Co-Packaged Optics (NVIDIA blog) | technical-report | 2025-08-18 | compiled |
| 3 | Broadcom Tomahawk 6 – Davisson | news | 2025-10-08 | compiled |
| 4 | Photonic Tensor Processor (imec) | paper | 2026-01-15 | compiled |
| 5 | Integrated Photonic Neural Network with On-Chip Backpropagation | paper | 2026-03-20 | compiled |
Read next
- Scaling AI Factories with Co-Packaged Optics for Better Power Efficiency — the watts behind the 3.5x, and why they give 3.3x.
- Broadcom Announces Tomahawk 6 - Davisson, the Industry's First 102.4-Tbps Ethernet Switch with Co-Packaged Optics — the power-per-bit claim on a named part, and why 70% and 3.5x are one number.
- NVIDIA Announces Spectrum-X Photonics, Co-Packaged Optics Networking Switches to Scale AI Factories to Millions of GPUs — the second vendor's claim, and the supplier list.
- the theses — the rescued interconnect thesis, and why it needs in-scope evidence.
Companies on this rung
All 13- Arista Networks
- Astera Labs
- AXT Inc
- Ayar Labs
Replacing copper with light. One chiplet at a time.
- Celestial AI
- Coherent Corp
- Credo Technology
- Eoptolink Technology
- Fabrinet
- Lightmatter
- Lumentum
- Nokia