LDLMIn construction

Meshchaninov et al. · May 2026Diffusion language model

Pinned implementation and paper LDLM @ e18c42a, not yet public · arXiv:2605.07933v1 · OpenWebText · 1024 · checkpoint, tokenizer and saved statistics not verified · How to read

Training Generationcall1

1 2 512 1,024

Gaussian prior

z_1\sim\mathcal N(0,I)

z₁[B, T, 768]

Sampler Open →

Latent decoder Open →

Token decoder Open →

L[B, T, 50259]Choose tokens

\arg\max_k L

I[B, T]Tokenizer datafixed assets Detokenizer Generated text

Detokenizer

no parameters

Turn each sequence of token IDs back into text.

in: $I$ [B batch, T positions]

Explanation

truncate_eos is false in this configuration: the whole sequence is decoded, special tokens included.

Sources3

Direct evidence

Generation: the sampled latents pass the latent decoder and the token decoder, each position takes the token of the largest logit, and the tokens are decoded to text, special tokens kept unless truncation is on.

The implementation, pinned at revision e18c42acdfb7, not yet public

Direct evidence

Generation: sequences are not truncated at the end token, and the trained adaptive time grid is used.

The implementation, pinned at revision e18c42acdfb7, not yet public

Direct evidence

The tokenizer is loaded by name, set up, and gains a `<mask>` token when it has none; the vocabulary size is its length.

The implementation, pinned at revision e18c42acdfb7, not yet public