DeepSeek V4.1

All models, with previews →

DeepSeek · Sep 2026New parts in the block · after DeepSeek-V3.2-Exp in the atlas

Effort-conditioned reward

Reports DeepSeek-V4.1 technical report @ 2cba9e4 · no training code is published · How to read

Effort-conditioned reward

Add length penalty

×

no parameters

The length-penalty term is added to the reward: the number of reasoning tokens ℓb,j\ell_{b,j} over a reference length LnormL_{\mathrm{norm}}, times the coefficient, taken off; Cmax⁡C_{\max} caps the deduction.

rb,jlen=−min⁡{Cmax⁡, k(b) ℓb,jLnorm}r^{\mathrm{len}}_{b,j}=-\min\Big\{C_{\max},\ {\htmlData{light=k coefficient}{{\htmlClass{vl-stream-k}{k}}}}(b)\,\frac{\ell_{b,j}}{L_{\mathrm{norm}}}\Big\}
in
rr [B batch, |𝔅|, M]kk [|𝔅|]zz [B batch, |𝔅|, M]
out
RR [B batch, |𝔅|, M]

R=r+rlenR=r+r^{\mathrm{len}}: RR and rr are this plate’s names; the report writes the penalty term rb,jlenr^{\mathrm{len}}_{b,j} alone.

Sources1

Direct evidence

§5.1.4, Controllable Reasoning Effort in RL: a scalar effort b ∈ {1, …, 100} is prepended to the system prompt. For each prompt x, M_b responses are sampled at each effort level b ∈ B (eq. 8). Responses sharing (x, b) form a subgroup, within which rewards are mean-centered to compute group-relative advantages. A length-penalty term is added to the reward: −min{C_max, k(b)·ℓ/L_norm} (eq. 9), with k(b) = k₀·exp(−(b − b_min)/τ), τ = λ·Δb̄, Δb̄ the average spacing between the training effort levels (eq. 10).

effort · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 29