Read straight down: explanation → mathematics → code → lab → review are now visible on one page, without separate tabs. Content status: this is still a concise legacy draft, not yet the complete research-level rewrite. Read the rebuilt Session 01 as the depth benchmark.
Explore the model: click any block
The diagram is interactive. Click a stage to open plain-English detail in the side card. The stages are conceptual, not a diagram of every tensor operation.
The idea in one minute
A modern pre-norm decoder block applies normalization, causal attention, residual addition, another normalization and a gated feed-forward network. Multiple blocks produce contextual token states.
Open deeper explanation
First name the input and the output. Then identify what is learned, what stays fixed and which signal supplies supervision. During training, compare the predicted quantity against its target; at inference, explain the operations that remain. Apply this method to the model above and inspect the mathematical tab.
When should I use this?
Trace the exact tensor shape through a block and identify both residual paths. The right approach depends on memory, latency, data quality, and the required evaluation metric.
Core mathematical model
Read every symbol as a quantity with a shape or a probability. For a working implementation, confirm the definition, units, and dimensions before changing hyperparameters.
How to reason about the equation
1. Identify the fixed input. 2. Identify learned quantities and the target. 3. Check which axis is normalized or reduced. 4. Test limiting cases. 5. Compare memory and time scaling as sequence length grows.
Minimal Python / PyTorch example
import torch
x=torch.randn(2,16,512)
norm=torch.nn.RMSNorm(512)
print(norm(x).shape)Runnable learning snippet; some lessons use a tiny conceptual example rather than a full production implementation.
From toy snippet to a real experiment
Add deterministic seeds, input assertions, tests for tensor shapes, a timing measurement and a meaningful baseline. For GPU work, record VRAM, throughput and quality separately.
Interactive lab: information budget
Illustrative budget visualization. Filled tiles represent retained units; this is a conceptual slider, not an execution of the named model.
Research exercise
Trace the exact tensor shape through a block and identify both residual paths.
What to report
Baseline, changed variable, expected effect, observed effect, failure case and next experiment. Avoid claiming a result until you run and measure it.
Self-check and spaced review
Question 1: What is the main input and output?
Explain them using the architecture diagram, and name their shapes where possible.
Question 2: What changes at training time?
Identify learned parameters, targets, loss function and optimizer or supervision signal.
Question 3: Which edge case breaks this method?
Design one counterexample and a metric that can detect it.
Review this lesson on days 1, 3, 7, 14 and 30 after completion.