Sampler › Denoiser
Predicted clean latents
×tensor
The denoiser’s estimate of the clean latents.
- shape
- [B batch, T positions, 768 d]
Sources
Contextual evidence from Denoiser
The denoiser owns the time embedding’s two affine layers, one rotary table for all blocks, an affine projection of the concatenated input, and the stack.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Denoiser
The time embedding passes its MLP and a SiLU; the noised latents are concatenated with the previous clean estimate, zeros when there is none, and projected; the stack runs in bfloat16.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Denoiser
The denoiser’s stack applies the blocks in order and then the output layer, each with the time condition.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Denoiser
Diffusion: self-conditioning on, `t_min` 0.05 and `eps` 10⁻⁵; the adaptive time sampler has 100 bins, a minimum weight of 0.0001, an EMA of 0.9 and updates every 5,000 steps; the denoiser’s EMA decay is 0.9999; scheduler `tanh` with d = 3, solver `euler`, `ode_sampling` false, N = 128 by default; the denoiser has 12 blocks of width 768 with 12 heads, time conditioning `adaln`, a standard MLP, softcap 50 and self-conditioning by concatenation; the timestep embedding’s `max_period` is 10.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Noise estimate
The denoiser’s clean prediction is converted into the noise estimate and the score with the marginal’s mean and standard deviation.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Noise estimate
The tangent schedule: α = 1/√(1 + d² tan²(πt/2)), σ = √(1 − α²); β(t) is evaluated with t clipped at 0.95.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Output projection
The output layer: gain-only LayerNorm, a shift and a scale from the time condition, and an affine projection initialized at zero.
The implementation, pinned at revision e18c42acdfb7, not yet public