Training

Pinned sources DeepSeek-V4.1 technical report @ 2cba9e4 · no training code is published · How to read

  1. Start of training
  2. Start of the encoder’s training
  3. Vision encoder: contrastiveDeepSeek-ViT≈ 47B pairsnot statedContrastiveSigLIPViT
  4. Vision encoder: autoregressiveDeepSeek-ViT236Bnot statedNTPViT
  5. DeepSeek-ViT
  6. Pre-training 0–28TDeepSeek-V4.1-Flash28T64KNTPBalanceweight 0.0001not ViTsparse from scratch
  7. Pre-training 28–34TLR decay begins6T64KNTPBalanceweight 0.0001all
  8. Pre-training 34–45TDeepSeek-V4.1-Flash11T1MNTPBalanceweight 0.0001all
  9. DeepSeek-V4.1-Flash-Base
  10. DSpark stageDeepSeek-V4.1-Flashnot statednot statednot statedDSparkDSpark added
  11. SFTDeepSeek-V4.1-Flashnot statednot statedSFTnot statedpost-training changes
  12. RLDeepSeek-V4.1-Flashnot statednot statedRLgroup-relativenot stated
  13. On-policy distillationDeepSeek-V4.1-Flashnot statednot statedOPDover 40 teachersnot stated
  14. DeepSeek-V4.1-Flash
  • tokens, log scale
  • derived: computed from stated numbers
  • context length
  • all parameters train
  • a part trains
  • all but a part trains
  • NTPdashed: concluded, no source names it
  • not stated by the sources
  • the architecture changes
  • checkpoint
  • checkpoint, weights published
  • a join the sources leave unexplained

DeepSeek-V4.1-Flash-Base

×

Checkpoint

The report’s name for the pre-trained model (Figure 6). It does not place the name relative to the DSpark stage, which leaves the backbone unchanged.

Weights
no pinned source shows them published
Sources1
Key
stage
tokens, log scale
a total no source states, computed from steps × batch × length
checkpoint
checkpoint, weights published
a join the sources leave unexplained
stages of a released model, folded into one row
derived
computed from reported values; the arithmetic is shown
not disclosed
no pinned source states it; nothing is filled in

DeepSeek V4.1

All models, with previews →

DeepSeek · Sep 2026New parts in the block · after DeepSeek-V3.2-Exp in the atlas