LDLMIn construction

All models, with previews →

Meshchaninov et al. · May 2026Diffusion language model

Time embedding

Pinned implementation and paper LDLM @ e18c42a, not yet public · arXiv:2605.07933v1 · OpenWebText · 1024 · checkpoint, tokenizer and saved statistics not verified · How to read

TrainingGeneration

Sampler › Denoiser › Time embedding

Contraction

×

learned

Map the 1,536 features back to 768.

e↓=eσW2+b2{\htmlData{light=time-down}{e^{\downarrow}}}={\htmlData{light=time-silu}{e^{\sigma}}}W_{2}+b_{2}
in
eσe^{\sigma} [B batch, 1536 features]
out
e↓e^{\downarrow} [B batch, 768 features]
parameters
1,180,416
Parameters

1,180,416 parameters owned by this component

Exact per-instance accounting
ComponentCalculationParameters
weights + bias1,536×768+7681{,}536 \times 768 + 7681,180,416
Per-instance total1,180,416

Across 1024 shared calls: 1,180,416×1=1,180,4161{,}180{,}416 \times 1 = 1{,}180{,}416 parameters.

Derived from the pinned implementation’s constructors and configuration. The checkpoint’s tensors have not been counted. 306,011,162 of these parameters are trained; frozen GPT-2 and the decoder’s mask vector, 124,442,112, are not.

Sources2

Direct evidence

The denoiser owns the time embedding’s two affine layers, one rotary table for all blocks, an affine projection of the concatenated input, and the stack.

The implementation, pinned at revision e18c42acdfb7, not yet public

Direct evidence

The time embedding passes its MLP and a SiLU; the noised latents are concatenated with the previous clean estimate, zeros when there is none, and projected; the stack runs in bfloat16.

The implementation, pinned at revision e18c42acdfb7, not yet public