Kimi K3: Open Frontier Intelligence

Supersession note. kb/models/raw/kimi-k3-moonshot-open-weight-2026.md was ingested 2026-07-20 from an independent review because Moonshot's own announcement was unreachable. It carries "16 active of 896 experts, ~1.8% activation" — that ratio was computed against parameter counts that this report corrects. The report's own figures are 2.8T total / 104B activated ≈ 3.7% activation. Where the two disagree, this file is authoritative; the earlier entry stays for the pricing, availability and third-party leaderboard context the report does not carry.

Abstract (verbatim)

"We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence."

Architecture — the numbers that matter to this rung

Three named architectural components:

• Kimi Delta Attention (KDA) — improves information flow across sequence length; the long-context mechanism, and the one the report says required algorithm-system co-design to run at this scale.
• Attention Residuals — improves information flow across model depth (the other axis).
• Stable LatentMoE — the routing scheme that makes 16-of-896 activation stable, i.e. avoids the expert collapse and load imbalance that normally punish extreme sparsity.

The through-line number: ~3.7% activation. Capability is bought at 2.8T parameters; per-token compute is paid at 104B. That decoupling is the whole reason a token price can fall while model size rises, and this rung held no current sparsity ratio before now.

The scaling-efficiency claim

"Approximately 2.5x improvement in overall scaling efficiency over Kimi K2", attributed jointly to KDA, Attention Residuals, Stable LatentMoE and refined training/data recipes.

Read this carefully: it is a generation-over-generation ratio against the authors' own prior model, not a comparison to anyone else's curve, and "overall scaling efficiency" is not defined on the abs page. It is a real and citable claim about Moonshot's own progress; it is not evidence that the industry's scaling efficiency improved 2.5x, and it should never be cited that way.

Infrastructure (what running 2.8T required)

• Algorithm-system co-design for KDA.
• Perfectly balanced expert-parallel training with efficient memory management — the load balance problem is the practical ceiling on high-sparsity MoE, and they claim to have solved it at 896 experts.
• Million-token agentic RL with persistent rollout and sandbox states — RL episodes long enough to need durable environment state, which is a harness-shaped problem appearing inside a model-training pipeline.
• Unspecified "deployment innovations."

Post-training

RL across general, agentic and coding domains, with multiple reasoning-effort levels — the same tiering shape the rung already holds for GPT-5.6 — claimed to enable compositional generalisation and robust long-horizon execution.

Limitations

• Stated by the authors: overall performance still trails Claude Fable 5 and GPT-5.6 Sol. An open-weight frontier report that names the two models it loses to is the opposite of a marketing posture, and that candour is part of why this source clears gate 2.
• No benchmark numbers on the abs page. "Frontier-level performance across long-horizon coding, agentic, knowledge, reasoning and vision" is a claim without a table at this depth; the scores live in the full 47-page report and are not captured here.
• "Scaling efficiency" is undefined in the abstract — tokens per unit quality? FLOPs per unit quality? Loss per unit compute? Unresolved without the full text; cite the 2.5x with that caveat.
• Self-evaluated suite. "Consistently outperforms other open and proprietary models evaluated in our suite" is the vendor's own eval selection.
• No cost or price figures in the report; the earlier secondary entry carries API pricing.

How it bears on MenFem

• "Parameter Scaling Is Dead — Multi-Dimensional Scaling." 2.8T total against 104B activated is this thesis made numeric: the parameter count still grows, but it is no longer what is paid per token. Scaling moved to sparsity, context and post-training effort levels — three axes, not one.
• "Inference Costs Will Fall 90% by 2028." The mechanism half of that call is sparsity, and ~3.7% activation at frontier-adjacent quality is the strongest current datapoint for it. Pair with the serving rung's disaggregation and speculative-decoding sources for the runtime half.
• Open weights at this scale bear on the lock-in score: a 2.8T open model that trails the leaders but beats everything else sets a floor under what capability costs, independent of any vendor's pricing decision.