Rung 02 Serving & RuntimeSwitch rungClose
Why the Same Box Costs 44x More
Quiz·Intermediate·prefill-decode-disaggregation, kv-cache, kv-cache-management, speculative-decoding, low-bit-quantization
Batching, the prefill/decode split, speculative decoding under load, and KV-cache tricks: the mechanisms behind the serving numbers, not the numbers alone