How to read
Shapes
- Tensor
- A capsule. Its symbol is its name; its shape is written beside it, and in full in its card.
- Learned operation
- A frame with a band along its top: it owns weights.
- Reuses weights
- A frame with an outlined band: its weights belong to another operation.
- Operation without parameters
- A frame with a cut corner.
- Combination
- A circle where tensors meet: a sum, a product, a weighted sum.
- Composite
- A frame with an offset outline: it opens to a plate of its own.
- Repetition
- Brackets with a count: what stands between them is repeated, each time with its own weights unless it says otherwise.
- Derivation
- A formula drawn in place; every term opens its card.
- Training only
- A dashed frame: the step runs in training and is the identity otherwise.
- Fixed resource
- A dotted frame: files a step reads and never changes.
- State kept between calls
- A dotted, rounded frame: a cache.
- Absent in this mode
- Dashed and faint: the step is not run in the execution mode shown.
Kinds
On a plate that shows parts of different kinds (a model, a stack, a layer, a block) a node with weights carries the colour of its kind: on its band, its frame and faintly inside it. These colours are darker than any stream’s.
- Attention
- Violet.
- Feed-forward, mixture of experts
- Rust.
- Embedding, projection onto the vocabulary
- Teal. A projection that reuses the embedding’s weights has an outlined teal band.
- A layer or a stack
- Its band names what it holds: half violet and half rust for attention with a feed-forward.
- Other learned steps
- Grey band: a normalization, a mix of stored states, a table that belongs to none of the three.
- Steps without weights
- No colour, whatever they serve: position IDs, the steps that build the RoPE coefficients and the causal mask, a sum, the tokenizer.
Inside one component (an attention, a feed-forward) every node is neutral, and a short band of the component’s colour stands over the plate’s title.
Lines
- Data flow
- An arrow carries a tensor from the step that makes it to the step that reads it.
- Fork
- A dot on a line: from there the same tensor goes on to several steps. Two lines that cross without a dot stay separate.
- Hidden-state trunk
- A heavy line: the hidden state on its way through the layers, which each layer reads and adds its update to.
- What the plate is about
- These tensors and their lines keep full colour at rest. One without a hue of its own is drawn in ink and heavier.
- Reads a fixed resource
- A dotted line.
- Same weights
- A double line with no arrowhead, bridging above the row.
Streams
A line’s colour says which stream of data it carries. A colour is always backed by a name: the tensor’s symbol, a route’s label or a frame’s title.
- hidden state (x, normalized x, Δ)
- tokens and vocabulary
- queries
- keys
- values
- keys and values before they split (latent)
- positions, mask
- inside the feed-forward (MLP, experts, router)
- feature space (h, reconstructed h)
- diffusion time and its conditioning
- everything else: scores, weights, indices
At rest a line is drawn in a softer step of its colour; the lines of the node under the pointer take the full colour.
Axes
- B
- batch
- T
- positions in the sequence
Every other axis (heads, a head’s width, the model’s width, experts) is named in the card of the tensor that has it.
Pointer and keys
- Click
- opens a node’s card under the diagram
- Double-click
- opens a composite’s own plate
- ← →
- walk to the neighbouring node
- Esc
- closes the node’s card; the plate’s own card returns
- Hover
- lights a node with what it reads and feeds