Kimi K3

All models, with previews →

Moonshot AI · Release date not pinnedNew parts in the block · after Kimi K2 in the atlas

MOPD reward

Reports Kimi K3 technical report @ 3cb39df · Kimi K2.5 report (arXiv 2602.02276v2), for the RL step · no training code is published · How to read

MOPD reward

Student writes

×

learned · not counted here

The student writes the response the reward is computed on. This is concluded: the report does not say directly who samples it.

y∼πθ(⋅∣e,x){\htmlData{light=response rollout}{y}}\sim\pi_\theta(\cdot\mid {\htmlData{light=effort}{e}},{\htmlData{light=query}{x}})
in
xx [B batch]ee [B batch]
out
yy [B batch, T positions]
Explanation

Three places of the report bear on it. The method is named on-policy distillation, which in the work the report cites means the student’s own samples; the report does not define it.

The reward “seamlessly integrates into our RL framework, naturally enabling infrastructure-level optimizations such as partial rollout training”: partial rollouts are the mechanism of the policy’s own rollouts.

And, of the experts: “Trajectories produced by the resulting experts at all reasoning levels are jointly collected for supervised fine-tuning and multi-teacher on-policy distillation.”

Read literally, the third makes the experts’ trajectories an input of this stage, and the report does not say as what. Who samples the response the reward is computed on is not said directly. The plate draws the student, from the first two, with a dashed frame.

Sources3

Direct evidence

§4.1.3, Multi-Teacher On-Policy Distillation: for a domain d and a sampled reasoning effort e ∈ {low, high, max}, optimization is guided by the teacher of that pair among the nine experts. The per-token reward on y_t is the stop-gradient log-ratio of the teacher’s probability to the student’s, clipped to ±R_max (eq. 15). The dense reward integrates into the RL framework, partial rollouts included.

mopd · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · pp. 13–14

Direct evidence

§4.1.2, Reasoning Effort RL: “Trajectories produced by the resulting experts at all reasoning levels are jointly collected for supervised fine-tuning and multi-teacher on-policy distillation.”

trajectories · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · p. 13

Direct evidence

§4.1.2, Algorithm: policy optimization “follows the algorithm in Kimi K2.5”, with a per-token regularization; this report does not write it.

rl · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · p. 13