Kimi K3

Moonshot AI · Release date not pinnedNew parts in the block · after Kimi K2 in the atlas

RL step

Reports Kimi K3 technical report @ 3cb39df · Kimi K2.5 report (arXiv 2602.02276v2), for the RL step · no training code is published · How to read

Problemx x Previous policy

\{y_j\}_{j=1}^{{\htmlClass{vl-stream-k}{K}}}\sim\pi_{\mathrm{old}}

y[B, K, T]Rewards

\text{rules}\ \cdot\ \text{generative RM}\ \cdot\ \text{budget}

r[B, K]Subtract the mean

A_j=r_j-\bar r

A[B, K]Policy

w=\pi_\theta/\pi_{\mathrm{old}}

w[B, K, T]Clip to [α, β]

c=\operatorname{Clip}(w,\alpha,\beta)

c[B, K, T]Squared log-ratio

D=(\log w)^2

D[B, K, T]Objective

\mathcal L_{\mathrm{RL}}=\tfrac1N\sum_{j,i}\big(c_j^iA_j-\tau D_j^i\big)

ℒ_RL[1]hatched band: not updated in this step · solid band: what this stage trainsKimi K3’s report, of this step: partial rollouts, extended · over its token budget a trajectory’s reward is −1 · “forward-only non-policy models” in the loss

RL step

Problem

tensor

A problem $x$ sampled from the training data.

shape: $x$ [B batch]

Sources1

Contextual evidence

Kimi K2.5 report §4.4.2, Policy Optimization: for each problem x, K responses are generated using the previous policy π_old; the objective (eq. 1) averages over the N generated tokens of the batch the token-level ratio π_θ/π_old clipped to [α, β], times the reward less the mean reward of the K responses, less τ times the squared log-ratio. The clipping is “a simple gradient masking scheme”: gradients are computed normally for tokens with log-ratios within [α, β] and zeroed outside it, “regardless of the sign of the advantages”. The MuonClip optimizer minimizes this objective.

k25 · arXiv 2602.02276 / KimiK2.5-2602.02276v2.pdf
Pinned revision v2 ↗ · p. 8