DeepSeek

Type: Lab (model lab that writes and publishes its own kernels)

On this rung DeepSeek matters as the author of FlashMLA, its open-source attention kernel library, and of the most detailed public account in this KB of how a kernel bottleneck is found and removed. When DeepSeek-V3.2 doubled context length from 64K to 128K tokens, the key-value cache for one request grew to 8.72 GiB, so DeepSeek moved the cache to FP8 — and then found that converting it back to BF16 on H800 cost more cycles than the matrix maths it fed. Its fix, "crossover", has two thread blocks split the conversion and share the halves through Hopper's Distributed Shared Memory.

The write-up is unusual in two ways: it states its bottleneck as a cycle budget whose arithmetic reproduces exactly, and the mechanism can be checked against the code DeepSeek actually ships. The close-read did that, down to the PTX instruction. The performance numbers cannot be checked the same way — they are DeepSeek measuring DeepSeek's own kernel, powering its own model, at one configuration, with no accuracy figure for the FP8 cache format.

Key Contributions

• Decode throughput 250 → 410 TFLOPS (1.64×) on H800 SXM5 from a kernel change alone — silicon, algorithm, numerics and cache format unchanged. As of 2025-09-29; vendor self-measurement (FlashMLA)
• A named, costed bottleneck: 50 cycles of FP8→BF16 conversion vs 34 of tensor-core work per KV token (FlashMLA)
• An FP8 KV-cache format at 656 bytes per token vs 1,152 in BF16, keeping the position-encoding slice in BF16 (FlashMLA)
• A published portability data point against itself: the same sparse decode kernel at up to 350 TFLOPS on B200, below the H800 figure (FlashMLA)

Evidence caveat

Every performance figure is first-party and unreproduced; the README's figures have not been restated since 2025-10-01 while the repository kept moving (HEAD 2026-07-27 at read time). Standing: read it — no Hopper or Blackwell hardware is in reach to run these kernels.

Mentioned In

• The non-matmul bottleneck — the Hopper half of the evidence
• Quantized KV cache — the FP8 format and its conversion cost
• Kernel portability — the B200-slower-than-H800 result
• FlashMLA FP8 deep dive — the source

Related Entities

• NVIDIA — maker of the H800 and B200 the kernel runs on
• DeepSeek on the serving rung — its production speculative-decoding system
• DeepSeek on the models rung