Rung 07 Compilers & KernelsSwitch rungClose
In scope: CUDA and Triton, kernel fusion and autotuning, compiler stacks (XLA, TVM, Inductor), custom kernels for attention and quantization, portability across accelerators. Out: the serving system above it (serving), the chip below it (hardware).
| Type | Source | Published |
|---|---|---|
| REPORT | A Deep Dive Into The Flash MLA FP8 Decoding Kernel on Hopper Shengyu Liu (git author of both commits to this document); FlashMLA authors of record per the repository citation block: Jiashi Li, Shengyu Liu · DeepSeek (deepseek-ai) An FP8-KV-cache attention decoding kernel on Hopper is bound not by tensor-core throughput but by dequantization on the CUDA cores: 50 cycles of format conversion per KV token against 34 cycles of MMA. "Crossover" — two CTAs in a cluster of 2 each dequantize half the KV block and share it over Distributed Shared Memory — lifts measured decode throughput 250 -> 410 TFLOPS on H800 SXM5 (1.64x, as of 2025-09-29) with silicon, algorithm and numerics unchanged. Vendor self-measurement; mechanism verified in shipped source. Same kernel scores 350 TFLOPS on B200, below the older H800. Engineering blog in the repository, filed as technical-report (company research blog). No accuracy evaluation of the FP8 cache format. | 2025-09-29 |
| REPORT | Warp Specialization in Triton: Design and Roadmap Manman Ren, Nick Riasanovsky, Neil Dhar, Hongtao Yu, Jie Liu, Partha Kanuparthy, Shane Nay · Meta (Triton autoWS team), published on the PyTorch blog VENDOR-AUTHORED (the team shipping the feature). Design of automatic warp specialization in the Triton compiler — data partitioning, loop scheduler, partition scheduler, buffer creation, memory planner, channel/code partitioner. On B200 flash-attention forward: 1.5-2x over stock Triton, near Gluon/cuDNN, cuDNN still ahead by 10-20% (one sentence, no configurations or absolute TFLOPS). Experimental; attention forward only; Hopper and Blackwell only; AMD on the roadmap. | 2026-01-08 |
| REPORT | ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training Janghwan Lee, Sihwa Lee, Jinseok Kim, Yongjik Kim, Jieun Lim, Jinwook Oh, Jungwook Choi · Not stated on the arXiv abstract page FP4 reasoning failure concentrates on LOW-ENTROPY tokens (digits, operators) — shown by a logit-noise experiment where perturbing only low-entropy predictions collapses accuracy and perturbing high-entropy ones barely moves it. W4A4KV4 NVFP4 + ReQAT hits 65.94% on AIME-120 vs BF16 FT 65.46% on R1-Qwen-14B — but TRAILS BF16 FT on R1-Llama-8B (41.85% vs 48.75%). 3.9x throughput on DGX Spark, 3.1x on B200. ICML 2026. | 2026-06-14 |