Kimi K3

Moonshot AI · Release date not pinnedNew parts in the block · after Kimi K2 in the atlas

RL step

Reports Kimi K3 technical report @ 3cb39df · Kimi K2.5 report (arXiv 2602.02276v2), for the RL step · no training code is published · How to read

Problemx x Previous policy

\{y_j\}_{j=1}^{{\htmlClass{vl-stream-k}{K}}}\sim\pi_{\mathrm{old}}

y[B, K, T]Rewards

\text{rules}\ \cdot\ \text{generative RM}\ \cdot\ \text{budget}

r[B, K]Subtract the mean

A_j=r_j-\bar r

A[B, K]Policy

w=\pi_\theta/\pi_{\mathrm{old}}

w[B, K, T]Clip to [α, β]

c=\operatorname{Clip}(w,\alpha,\beta)

c[B, K, T]Squared log-ratio

D=(\log w)^2

D[B, K, T]Objective

\mathcal L_{\mathrm{RL}}=\tfrac1N\sum_{j,i}\big(c_j^iA_j-\tau D_j^i\big)

ℒ_RL[1]hatched band: not updated in this step · solid band: what this stage trainsKimi K3’s report, of this step: partial rollouts, extended · over its token budget a trajectory’s reward is −1 · “forward-only non-policy models” in the loss

RL step

Per-token penalty

tensor

Zero where the policy gives a token the probability the previous policy gave it.

shape: $D$ [B batch, K, T positions]

Sources

Contextual evidence from Squared log-ratio

Kimi K2.5 report §4.4.2, Policy Optimization: for each problem x, K responses are generated using the previous policy π_old; the objective (eq. 1) averages over the N generated tokens of the batch the token-level ratio π_θ/π_old clipped to [α, β], times the reward less the mean reward of the K responses, less τ times the squared log-ratio. The clipping is “a simple gradient masking scheme”: gradients are computed normally for tokens with log-ratios within [α, β] and zeroed outside it, “regardless of the sign of the advantages”. The MuonClip optimizer minimizes this objective.

k25 · arXiv 2602.02276 / KimiK2.5-2602.02276v2.pdf
Pinned revision v2 ↗ · p. 8

Contextual evidence from Squared log-ratio

§4.1.2: K completions are sampled for each of N prompts, and generation pauses once a fraction λ of the trajectories completes (partial rollouts); policy optimization “follows the algorithm in Kimi K2.5”, with a per-token regularization that tolerates stale data. A trajectory whose tokens exceed τ·b₀(x) has its task reward overridden with −1. Tasks that cannot be verified take an agentic generative reward model with a tournament of binary comparisons.

k3 · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · p. 13