Kimi K3 · training steps · Clip to [α, β]

RL step

Clip to [α, β]

no parameters

The ratio is clipped token by token. The report describes the effect on this term as a mask on the gradient: tokens whose log-ratio lies in $[\alpha,\beta]$ , as its prose has it, pass theirs, the others pass none, whatever the sign of the advantage.

c_j^i=\operatorname{Clip}\big(w_j^i,\alpha,\beta\big)

in: $w$ [B batch, K, T positions]
out: $c$ [B batch, K, T positions]

Eq. 1 clips the ratio to $[\alpha,\beta]$ , with $\alpha,\beta>0$ ; the prose places the log-ratio in $[\alpha,\beta]$ . The report does not reconcile the two; the plate draws the equation.

Explanation

The report names this as its distinction from the clipping of PPO: it “relies strictly on the log-ratio to explicitly bound off-policy drift, regardless of the sign of the advantages”.

The Kimi K3 report credits “a per-token regularization” that constrains updates “within a localized neighborhood”; it does not say whether that is this clip, the squared log-ratio, or both.

Sources1

Direct evidence

Kimi K2.5 report §4.4.2, Policy Optimization: for each problem x, K responses are generated using the previous policy π_old; the objective (eq. 1) averages over the N generated tokens of the batch the token-level ratio π_θ/π_old clipped to [α, β], times the reward less the mean reward of the K responses, less τ times the squared log-ratio. The clipping is “a simple gradient masking scheme”: gradients are computed normally for tokens with log-ratios within [α, β] and zeroed outside it, “regardless of the sign of the advantages”. The MuonClip optimizer minimizes this objective.

k25 · arXiv 2602.02276 / KimiK2.5-2602.02276v2.pdf
Pinned revision v2 ↗ · p. 8