Sampler › Denoiser
Previous estimate
×supplied · no parameters
The clean latents as estimated by an earlier call; zeros when there is none.
- shape
- [B batch, T positions, 768 d]
Explanation
This is self-conditioning: the denoiser sees its own earlier answer beside the noised latents.
Sources
Contextual evidence from Denoiser
The denoiser owns the time embedding’s two affine layers, one rotary table for all blocks, an affine projection of the concatenated input, and the stack.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Denoiser
The time embedding passes its MLP and a SiLU; the noised latents are concatenated with the previous clean estimate, zeros when there is none, and projected; the stack runs in bfloat16.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Denoiser
The denoiser’s stack applies the blocks in order and then the output layer, each with the time condition.
The implementation, pinned at revision e18c42acdfb7, not yet public
Contextual evidence from Denoiser
Diffusion: self-conditioning on, `t_min` 0.05 and `eps` 10⁻⁵; the adaptive time sampler has 100 bins, a minimum weight of 0.0001, an EMA of 0.9 and updates every 5,000 steps; the denoiser’s EMA decay is 0.9999; scheduler `tanh` with d = 3, solver `euler`, `ode_sampling` false, N = 128 by default; the denoiser has 12 blocks of width 768 with 12 heads, time conditioning `adaln`, a standard MLP, softcap 50 and self-conditioning by concatenation; the timestep embedding’s `max_period` is 10.
The implementation, pinned at revision e18c42acdfb7, not yet public