LDLMIn construction

All models, with previews →

Meshchaninov et al. · May 2026Diffusion language model

Pinned implementation and paper LDLM @ e18c42a, not yet public · arXiv:2605.07933v1 · OpenWebText · 1024 · checkpoint, tokenizer and saved statistics not verified · How to read

TrainingGeneration

Gaussian prior

×

no parameters

Draw every latent feature independently from a standard Gaussian: 1,024 latents of 768 features for each sequence.

z1∼N(0,I){\htmlData{light=noise prior}{z_1}}\sim\mathcal N(0,I)
out
z1z_1 [B batch, T positions, 768 d]
Explanation

Generation is unconditional: nothing but this noise and the noise the sampler adds at each step decides the text. The number of latents fixes the length, 1,024 tokens.

Sources2

Direct evidence

The prior is a standard Gaussian.

The implementation, pinned at revision e18c42acdfb7, not yet public

Direct evidence

Sampling starts from the prior with a zero self-conditioning estimate, uses the averaged denoiser, takes the adaptive grid when it is requested and the sampler has been updated and a uniform grid otherwise, steps N times carrying the state and the clean estimate, and returns the last step’s mean.

The implementation, pinned at revision e18c42acdfb7, not yet public