LDLMIn construction

All models, with previews →

Meshchaninov et al. · May 2026Diffusion language model

Sampler

Pinned implementation and paper LDLM @ e18c42a, not yet public · arXiv:2605.07933v1 · OpenWebText · 1024 · checkpoint, tokenizer and saved statistics not verified · How to read

TrainingGeneration
Latents at time tz[B, T, 768]Previous estimateẑsc[B, T, 768]Adaptive time grid1,024 stepst[B]
DenoiserOpen →
z0[B, T, 768]
ϵ^\hat\epsilon==(zt−αtz^0)/σt({\htmlData{light=state}{z_t}}-\alpha_t{\htmlData{light=prediction denoiser}{\hat z_0}})/\sigma_tss==−ϵ^/σt-\hat\epsilon/\sigma_tff==−12βtzt−βts-\tfrac12\beta_t{\htmlData{light=state}{z_t}}-\beta_tsmt′{\htmlData{light=step-mean mean}{m_{t'}}}==zt+f (t′−t){\htmlData{light=state}{z_t}}+f\,(t'-{\htmlData{light=time grid}{t}})zt′{\htmlData{light=next-state diffuse}{z_{t'}}}==mt′+βt (t−t′)  ξ{\htmlData{light=step-mean mean}{m_{t'}}}+\sqrt{\beta_t\,({\htmlData{light=time grid}{t}}-t')}\;\xi
mt′[B, T, 768]zt′[B, T, 768]starts as Gaussian noisereplaced by the next latentsstarts as zerosreplaced by this call’s predictionthe next call’s latentsreturned by the last callcall 1 of 1,024 · m: the step’s mean, before its noise · t′: the grid’s next time · α, σ, β: the schedule at t · ξ: fresh Gaussian noise

Sampler

Noise estimate

×

no parameters

The noise that, with the predicted clean latents, would give the current latents.

ϵ^=zt−αt z^0σt{\htmlData{light=noise-estimate}{\hat\epsilon}}=\frac{{\htmlData{light=state}{z_t}}-\alpha_t\,{\htmlData{light=denoiser}{\hat z_0}}}{\sigma_t}
in
ztz_t [B batch, T positions, 768 d]z^0\hat z_0 [B batch, T positions, 768 d]tt [B batch]
out
ϵ^\hat\epsilon [B batch, T positions, 768 d]
Explanation

αt=1/1+9tan⁡2(πt/2)\alpha_t=1/\sqrt{1+9\tan^2(\pi t/2)} and σt=1−αt2\sigma_t=\sqrt{1-\alpha_t^2}, the schedule’s coefficients at the current time.

Sources2

Direct evidence

The denoiser’s clean prediction is converted into the noise estimate and the score with the marginal’s mean and standard deviation.

The implementation, pinned at revision e18c42acdfb7, not yet public

Direct evidence

The tangent schedule: α = 1/√(1 + d² tan²(πt/2)), σ = √(1 − α²); β(t) is evaluated with t clipped at 0.95.

The implementation, pinned at revision e18c42acdfb7, not yet public