DeepSeek-V4.1-Flash · training steps · Effort-conditioned reward

out: $A$ [B batch, |𝔅|, M]

Explanation

Responses at different efforts are not directly compared: a subgroup is one prompt at one effort. The effort is in the prompt, and within a subgroup the behaviour it asks for is induced by the length component of the reward, which is heavy at a low effort and light at a high one.

The plate stops at the advantage. The report names GRPO only through its group size and writes no objective, clipping or KL term for this model.

The trained effort levels, $M_b$ , $k_0$ , $\lambda$ , $C_{\max}$ and $L_{\mathrm{norm}}$ are not disclosed. In deployment the tiers max, high and low are the efforts 100, 75 and 50, and values between the trained levels can be requested.

Sources5

Direct evidence

§5.1.4, Controllable Reasoning Effort in RL: a scalar effort b ∈ {1, …, 100} is prepended to the system prompt. For each prompt x, M_b responses are sampled at each effort level b ∈ B (eq. 8). Responses sharing (x, b) form a subgroup, within which rewards are mean-centered to compute group-relative advantages. A length-penalty term is added to the reward: −min{C_max, k(b)·ℓ/L_norm} (eq. 9), with k(b) = k₀·exp(−(b − b_min)/τ), τ = λ·Δb̄, Δb̄ the average spacing between the training effort levels (eq. 10).

effort · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 29

Direct evidence

§5.1.1: “We formalize each task as a triplet (problem, environment, verification system)”.

tasks · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 25

Direct evidence

§5.2.1: prompts are dispatched by “the GRPO group size”; the report writes no objective, clipping or KL term.

group · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 30

Direct evidence

§5.1.4, Table 2: the public API’s tiers max, high and low are the effort values 100, 75 and 50; training uses a finite set of effort levels.

tiers · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 30

Direct evidence

Appendix C: k₀ is the penalty coefficient at the lowest effort level b_min.

k0 · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 49