LDLMIn construction

Meshchaninov et al. · May 2026Diffusion language model

Pinned implementation and paper LDLM @ e18c42a, not yet public · arXiv:2605.07933v1 · OpenWebText · 1024 · checkpoint, tokenizer and saved statistics not verified · How to read

Training Generationcall1

1 2 512 1,024

Gaussian prior

z_1\sim\mathcal N(0,I)

z₁[B, T, 768]

Sampler Open →

Latent decoder Open →

Token decoder Open →

L[B, T, 50259]Choose tokens

\arg\max_k L

I[B, T]Tokenizer datafixed assets Detokenizer Generated text

Choose tokens

no parameters

At every position take the token with the largest logit.

I_{b,t}=\arg\max_{{\htmlClass{vl-stream-k}{k}}}L_{b,t,{\htmlClass{vl-stream-k}{k}}}

in: $L$ [B batch, T positions, 50259 vocabulary]
out: $I$ [B batch, T positions]

Explanation

Nothing is sampled here: all the randomness of generation is in the latents. Every position is decided at once, from one pass of the decoders.

Sources1

Direct evidence

Generation: the sampled latents pass the latent decoder and the token decoder, each position takes the token of the largest logit, and the tokens are decoded to text, special tokens kept unless truncation is on.

The implementation, pinned at revision e18c42acdfb7, not yet public