Variational autoencoders and latent spaces
A variational autoencoder squeezes an image into a small, smooth latent code and learns to decode it back. This session derives its training loss (the ELBO), shows the reparameterization trick that makes it trainable, explains posterior collapse and blurry samples, and shows why Stable Diffusion, SD3, FLUX and video generators all run diffusion inside a VAE latent space.
What you will learn
- Why a plain autoencoder cannot generate, and what the prior, the decoder and the encoder each do in a VAE
- How to derive the ELBO, why it is a lower bound on log p(x), and what the gap between them measures
- The KL term between diagonal Gaussians in closed form, and how a Gaussian decoder turns reconstruction into MSE
- The reparameterization trick, and why its gradients have much lower variance than the score-function estimator
- Posterior collapse: when and why the latent is ignored (with an exact linear-VAE result), and the standard fixes
- Why VAE samples are blurry, and how perceptual and adversarial losses fix this for the autoencoders of latent diffusion
- The compression arithmetic of latent diffusion (512 x 512 x 3 to 64 x 64 x 4), latent scaling factors, and the channel trade-off of SD3
Why this matters
Stable Diffusion, SDXL, SD3, FLUX and most video generators do not run diffusion on pixels. They first compress the image with a variational autoencoder (VAE), run the expensive generative model on the small latent code, and decode the result back to pixels at the very end.
The numbers show why. A 512 × 512 RGB image has \(512 \cdot 512 \cdot 3 = 786{,}432\) numbers. The Stable Diffusion 1.x VAE turns it into a latent of shape 64 × 64 × 4, which is \(16{,}384\) numbers: 48 times fewer. [Certain] (its config has 4 latent channels and 3 down-sampling steps, so \(512/8 = 64\)). Every denoising step of the diffusion model now works on 48 times less data, and a transformer on that latent sees far fewer tokens.
The VAE is also the cleanest example of an idea you will meet again and again in this track: you cannot compute the likelihood \(p(\mathbf{x})\) of your model, so you maximize a lower bound on it instead. The diffusion loss in session 225 is the same kind of bound. And the VAE has two famous failure modes, posterior collapse and blurry samples, which explain why modern image models use a VAE only as a compressor and leave the actual generation to diffusion.
In session 222 you saw the VAE as one family among five. This session opens it up. The next session, session 224, replaces the continuous latent with a discrete codebook (VQ-VAE, VQGAN), which is how images become tokens for an LLM.
From an autoencoder to a generative model
An autoencoder has two networks. The encoder maps an input \(\mathbf{x}\) (an image) to a short vector \(\mathbf{z}\), the latent code. The decoder maps \(\mathbf{z}\) back to a reconstruction \(\hat{\mathbf{x}}\). Both are trained together to make \(\hat{\mathbf{x}}\) close to \(\mathbf{x}\). Because \(\mathbf{z}\) is much smaller than \(\mathbf{x}\), the network must keep only what matters. This is useful compression.
But a plain autoencoder cannot generate. Nothing tells us where in latent space the codes of real images live. They may sit on a few thin, curved strips with empty space between them. If you pick a random \(\mathbf{z}\) and decode it, you usually land in that empty space, and the decoder was never trained there. The output is garbage.
A VAE fixes this by starting from a probabilistic model of the data. We first choose a simple distribution for the latent, the prior \(p(\mathbf{z}) = \mathcal{N}(\mathbf{0}, I)\). Then a neural decoder defines \(p_\theta(\mathbf{x} \mid \mathbf{z})\), the probability of an image given a code. Together they define a probability for every image:
- Read the symbols
- \(p(\mathbf{z})\)
- the prior: a standard Gaussian over latent codes; this is where we will sample from
- \(p_\theta(\mathbf{x} \mid \mathbf{z})\)
- the decoder, with weights \(\theta\): a distribution over images for one code (for example a Gaussian centered on the decoded image)
- \(p_\theta(\mathbf{x})\)
- the likelihood (also called evidence) of an image: the average of the decoder over all codes
Generation is now easy: draw \(\mathbf{z} \sim \mathcal{N}(\mathbf{0}, I)\) and decode. Training is the hard part. We want to maximize \(\log p_\theta(\mathbf{x})\) over the training images, but the integral in (223.1) runs over all of latent space. For almost every \(\mathbf{z}\), the decoded image looks nothing like \(\mathbf{x}\), so random sampling of \(\mathbf{z}\) would need a huge number of samples to find the few codes that matter. We need to know which codes are likely for a given image. That is the posterior \(p_\theta(\mathbf{z} \mid \mathbf{x}) = p_\theta(\mathbf{x} \mid \mathbf{z})\, p(\mathbf{z}) / p_\theta(\mathbf{x})\), and it contains the same impossible integral in its denominator.
The encoder guesses the posterior
The VAE’s answer (Kingma and Welling, 2013; Rezende et al., 2014) is to train a second network to guess the posterior. The encoder \(q_\phi(\mathbf{z} \mid \mathbf{x})\), with weights \(\phi\), reads an image and outputs a Gaussian over codes:
So the encoder outputs two vectors: a mean \(\mu\) (where the code is) and a standard deviation \(\sigma\) (how unsure it is), one number per latent dimension. In practice the network outputs \(\log \sigma^2\), which can be any real number. One encoder serves all images; this is called amortized inference, because the cost of finding a posterior is paid once, during training, instead of once per image. Figure 1 runs the whole model on a tiny example. It uses the two loss formulas of the next sections, (223.4) and (223.5); you can step through it now and come back to it after Algorithm 1.
- input: image \(\mathbf{x}\) (4 pixels), \(\beta\), decoder noise \(s^2 = 0.5\)
- \(\mu \leftarrow W_\mu \mathbf{x} + b_\mu\) // encoder head 1
- \(\log\sigma^2 \leftarrow W_v \mathbf{x} + b_v\); \(\sigma \leftarrow \exp(\tfrac12\log\sigma^2)\)
- \(\varepsilon \sim \mathcal{N}(\mathbf{0}, I)\) // outside noise, no gradient
- \(\mathbf{z} \leftarrow \mu + \sigma \odot \varepsilon\) // (223.6)
- \(\hat{\mathbf{x}} \leftarrow W_d \mathbf{z} + b_d\) // decoder
- \(R \leftarrow \lVert \mathbf{x} - \hat{\mathbf{x}} \rVert^2 / (2s^2) + \tfrac{D}{2}\log(2\pi s^2)\) // (223.5)
- \(K \leftarrow \tfrac12 \sum_j (\mu_j^2 + \sigma_j^2 - 1 - \log\sigma_j^2)\) // (223.4)
- \(\text{loss} \leftarrow R + \beta K\) // one-sample \(-\text{ELBO}\) when \(\beta = 1\)
- backward through \(\mathbf{z}\): \(\partial\text{loss}/\partial\mu\), \(\partial\text{loss}/\partial\sigma\) // (223.7)
How to read Figure 1
Press Step to run one line of the algorithm at a time; the line in use is highlighted, and the panel next to it shows the formula with the current numbers. In the latent plane (middle), the grey circles are the prior at radius 1 and 2, the blue ellipse is \(q(\mathbf{z} \mid \mathbf{x})\) with radii \(\sigma_1, \sigma_2\), and the grey dot is \(\varepsilon\). On line 5 the dot is first stretched by \(\sigma\) and then shifted by \(\mu\): that is the reparameterization. Press “New ε” to draw another noise vector: \(\mathbf{z}\), \(\hat{\mathbf{x}}\) and \(R\) change, but \(\mu\), \(\sigma\) and \(K\) do not, because they depend only on \(\mathbf{x}\). The readout compares this one-sample loss with its exact average over all \(\varepsilon\). The fixed weights are \(W_\mu = \begin{pmatrix} 1 & -1 & 0.5 & -0.5 \\ 0.5 & 0.5 & -0.5 & -0.5 \end{pmatrix}\), \(b_\mu = \mathbf{0}\), \(W_v = \begin{pmatrix} -0.5 & 0.5 & -0.5 & 0.5 \\ 0.25 & 0.25 & 0.25 & 0.25 \end{pmatrix}\), \(b_v = (-1, -1)\), \(W_d = \begin{pmatrix} 0.3 & 0.2 \\ -0.3 & 0.2 \\ 0.25 & -0.2 \\ -0.25 & -0.2 \end{pmatrix}\), \(b_d = (0.5, 0.5, 0.5, 0.5)\). Pixels are numbered left to right, top to bottom.
The ELBO: a lower bound we can compute
Here is the key derivation. Take any image \(\mathbf{x}\) and any distribution \(q(\mathbf{z})\), here \(q_\phi(\mathbf{z} \mid \mathbf{x})\). Since \(\log p_\theta(\mathbf{x})\) does not depend on \(\mathbf{z}\), we may average it over \(q\), and then use \(p_\theta(\mathbf{x}) = p_\theta(\mathbf{x}, \mathbf{z}) / p_\theta(\mathbf{z} \mid \mathbf{x})\):
A KL divergence is never negative (session 3). So the first term is always at most \(\log p_\theta(\mathbf{x})\). It is called the evidence lower bound (ELBO). The gap between the bound and the true log-likelihood is exactly the KL divergence from the encoder’s guess to the true posterior. The bound is tight when the encoder guesses the posterior perfectly.
Now write \(p_\theta(\mathbf{x}, \mathbf{z}) = p_\theta(\mathbf{x} \mid \mathbf{z})\, p(\mathbf{z})\) and split the logarithm. We get the form that every VAE implementation uses:
- Read the symbols
- reconstruction
- sample codes from the encoder, decode them, and score how well the original image is explained; high when the code carries the details of \(\mathbf{x}\)
- regularizer
- how far the encoder’s Gaussian for this image is from the prior \(\mathcal{N}(\mathbf{0}, I)\); it is the price, in nats, of the information the code carries about \(\mathbf{x}\)
- training
- maximize the ELBO over \(\theta\) and \(\phi\) together, averaged over the data; the loss is the negative ELBO
The two terms pull in opposite directions. Reconstruction wants each image to get a sharp, unique code. The KL term wants every code distribution to look like the same standard Gaussian. The compromise is a latent space where each image owns a small, fuzzy region, and the regions overlap and fill the prior. That is exactly the property a plain autoencoder lacks.
Figure 2 makes (223.2) concrete with a model small enough to solve exactly: one latent number \(z \sim \mathcal{N}(0, 1)\) and a linear decoder \(p(x \mid z) = \mathcal{N}(x;\ 2z,\ 1)\). For this model the true posterior is a Gaussian we can write down, so we can measure the gap.
How to read Figure 2
Move the encoder mean and variance. The dashed line \(\log p(x)\) never moves, because it does not depend on \(q\). The ELBO bar is always below it, and the gap shrinks to 0 only when the blue curve covers the orange one exactly (press the button). Notice the trade inside the ELBO: a narrow \(q\) far from 0 makes reconstruction better but the KL to the prior worse. Change \(x\): a large \(|x|\) moves the posterior away from the prior, and \(\log p(x)\) drops because such an \(x\) is rare under the model.
Model: \(z \sim \mathcal{N}(0,1)\), \(x = 2z + \text{noise}\) with noise variance 1. Then \(x \sim \mathcal{N}(0,\ 2^2 + 1) = \mathcal{N}(0, 5)\), and for \(x = 1.5\): \(\log p(x) = -\tfrac12 \log(2\pi \cdot 5) - 1.5^2/10 = -1.949\) nats.
The exact posterior is Gaussian with mean \(2 \cdot 1.5 / 5 = 0.6\) and variance \(1/5 = 0.2\).
Take the encoder guess \(q = \mathcal{N}(0.3,\ 0.5)\). Reconstruction: \(\E_q[\log p(x \mid z)] = -\tfrac12\log 2\pi - \tfrac12\big[(1.5 - 2 \cdot 0.3)^2 + 4 \cdot 0.5\big] = -2.324\). KL to the prior (formula 223.4 below): \(\tfrac12(0.09 + 0.5 - 1 - \log 0.5) = 0.142\). ELBO \(= -2.324 - 0.142 = -2.466\). The gap is \(-1.949 - (-2.466) = 0.517\) nats, and computing \(\KL(q \,\|\, p(z \mid x))\) directly also gives \(0.517\). With \(q = \mathcal{N}(0.6, 0.2)\) the gap is 0.
The two terms in closed form
The KL term
Both the encoder’s distribution and the prior are Gaussians with diagonal covariance, so the KL term has a closed form, summed over the \(d\) latent dimensions:
Each dimension pays for two things. \(\mu_j^2\): the code is moved away from the center. \(\sigma_j^2 - 1 - \log\sigma_j^2\): the code is more certain (narrower) than the prior; this part is 0 at \(\sigma_j = 1\) and grows fast as \(\sigma_j \to 0\). A dimension with \(\mu_j = 0,\ \sigma_j = 1\) for every image costs nothing, and also carries no information. Remember this: it is what posterior collapse looks like.
The encoder outputs \(\mu = (1, 0)\) and \(\sigma = (0.5, 1)\). Dimension 1: \(\tfrac12(1 + 0.25 - 1 - \log 0.25) = \tfrac12(0.25 + 1.386) = 0.818\) nats. Dimension 2: \(\tfrac12(0 + 1 - 1 - 0) = 0\). Total KL \(= 0.818\) nats. Only dimension 1 carries information about this image.
The reconstruction term
The decoder outputs an image \(\hat{\mathbf{x}} = D_\theta(\mathbf{z})\), and we must choose what \(p_\theta(\mathbf{x} \mid \mathbf{z})\) means. The common choice for real-valued pixels is a Gaussian centered on \(\hat{\mathbf{x}}\) with a fixed variance \(s^2\) for each of the \(D\) pixels:
So the reconstruction term is a squared error, scaled by \(1/(2s^2)\). For binary pixels, a Bernoulli decoder gives binary cross-entropy instead. The scale matters a lot: \(s^2\) sets the exchange rate between pixel error and KL nats. A large \(s^2\) means “pixel errors are cheap”, so the model will spend fewer nats in the code. We will see this cause posterior collapse below.
The reparameterization trick
We train with gradient descent, so we need \(\nabla_\phi\) of the reconstruction term \(\E_{q_\phi(\mathbf{z} \mid \mathbf{x})}[f(\mathbf{z})]\), where \(f(\mathbf{z}) = \log p_\theta(\mathbf{x} \mid \mathbf{z})\). The trouble: \(\phi\) changes the distribution we sample from. A sample \(\mathbf{z}\) drawn by a random number generator has no gradient with respect to \(\phi\).
The trick is to move the randomness outside the network. Draw noise that does not depend on \(\phi\), then build the sample with a differentiable formula:
Here \(\odot\) is element-wise multiplication. The sample \(\mathbf{z}\) has exactly the distribution \(\mathcal{N}(\mu, \operatorname{diag}\sigma^2)\), but now it is a plain function of \(\mu\) and \(\sigma\), so backpropagation runs through it like any other layer: \(\partial \mathbf{z}/\partial \mu = 1\) and \(\partial \mathbf{z}/\partial \sigma = \varepsilon\). One sample of \(\varepsilon\) per image is usually enough.
Let us write the gradients out for one image. Call \(\mathbf{g}_z = \partial R / \partial \mathbf{z}\) the gradient that the decoder sends back to the sample, where \(R = -\log p_\theta(\mathbf{x} \mid \mathbf{z})\). The chain rule through \(\mathbf{z} = \mu + \sigma \odot \varepsilon\), plus the derivative of the KL formula (223.4), gives for the loss \(R + \beta K\):
- Read the symbols
- \(\mathbf{g}_z\)
- how the reconstruction loss changes when the sample moves; for the Gaussian decoder \(\hat{\mathbf{x}} = W_d\mathbf{z} + b_d\) of Figure 1 it is \(-W_d^\top(\mathbf{x} - \hat{\mathbf{x}})/s^2\)
- \(\beta\mu\)
- the KL term pulls the mean back toward 0
- \(\mathbf{g}_z \odot \varepsilon\)
- the reconstruction part of the \(\sigma\) gradient: it uses the same noise that built \(\mathbf{z}\), because \(\partial \mathbf{z} / \partial \sigma = \varepsilon\)
- \(\beta(\sigma - 1/\sigma)\)
- the KL term pushes \(\sigma\) toward 1: negative (grow \(\sigma\)) when \(\sigma \lt 1\), positive when \(\sigma \gt 1\)
Image A is \(\mathbf{x} = (0.9, 0.1, 0.8, 0.2)\). The encoder gives \(\mu = (1.1, 0)\), \(\log\sigma^2 = (-1.7, -0.5)\), so \(\sigma = (0.427, 0.779)\). The first noise vector is \(\varepsilon = (-0.357, 0.700)\), so \(\mathbf{z} = (1.1 - 0.427 \cdot 0.357,\ 0.779 \cdot 0.700) = (0.947, 0.545)\).
The decoder gives \(\hat{\mathbf{x}} = (0.893, 0.325, 0.628, 0.154)\), with squared error 0.0824. With \(s^2 = 0.5\): \(R = 0.0824 / 1 + 2\log\pi = 0.082 + 2.290 = 2.372\). KL: \(\tfrac12(1.21 + 0.183 - 1 + 1.7) + \tfrac12(0 + 0.607 - 1 + 0.5) = 1.046 + 0.053 = 1.100\). With \(\beta = 1\), loss \(= 3.471\).
Backward: \(\mathbf{g}_z = (-0.202, 0.175)\). By (223.7), \(\partial\text{loss}/\partial\mu = (-0.202 + 1.1,\ 0.175 + 0) = (0.898, 0.175)\) and \(\partial\text{loss}/\partial\sigma = (0.072 - 1.912,\ 0.122 - 0.505) = (-1.840, -0.383)\). The reconstruction part alone (\(\beta = 0\)) would make \(\sigma_1\) smaller (gradient \(+0.072\)), the KL part makes it larger, and here the KL part wins.
This loss is a one-sample estimate. Its exact average over \(\varepsilon\) is \(\E[R] + K = 2.453 + 1.100 = 3.553\) (the expected squared error adds \(\sum_j \lVert W_{d,:j} \rVert^2 \sigma_j^2\) to the error at the mean). Other noise draws give 3.402 and 3.423, so single samples scatter around the true value.
Without the trick, there is a general alternative, the score-function (REINFORCE) estimator, which works even for discrete \(\mathbf{z}\) (session 64 uses it for RL):
It is unbiased, but it only uses the value of \(f\), not its slope, so its variance is much higher. Figure 3 compares the two on \(f(z) = (z - 2)^2\) with \(z \sim \mathcal{N}(\mu, \sigma^2)\), where the true gradient \(\partial \E[f]/\partial\mu = 2(\mu - 2)\) is known.
How to read Figure 3
Each dot is one gradient estimate a training step would use. The readout gives the exact standard deviation of one-sample estimates: \(2\sigma\) for the reparameterization estimator, and \(\sqrt{(d^4 + 18 d^2\sigma^2 + 15\sigma^4)/\sigma^2 - 4d^2}\) with \(d = \mu - 2\) for the score-function one. At \(\mu = 0, \sigma = 1\) these are 2 and 9.33, a variance ratio of about 22. To match the reparameterization estimator with \(N = 1\), the score-function estimator needs about 22 samples. Small \(\sigma\) makes it worse, not better, because of the \(1/\sigma\) in \(\nabla_\mu \log q\).
- input: a mini-batch of images \(\mathbf{x}_1, \dots, \mathbf{x}_B\); encoder weights \(\phi\); decoder weights \(\theta\); weight \(\beta\) on the KL (1 for the true ELBO)
- \((\mu_i, \log\sigma_i^2) \leftarrow \text{Encoder}_\phi(\mathbf{x}_i)\) for each image
- \(\varepsilon_i \sim \mathcal{N}(\mathbf{0}, I)\); \(\;\mathbf{z}_i \leftarrow \mu_i + \exp(\tfrac12 \log\sigma_i^2) \odot \varepsilon_i\) // formula (223.6)
- \(\hat{\mathbf{x}}_i \leftarrow \text{Decoder}_\theta(\mathbf{z}_i)\)
- \(R_i \leftarrow -\log p_\theta(\mathbf{x}_i \mid \mathbf{z}_i)\) // summed over pixels, formula (223.5)
- \(K_i \leftarrow \tfrac12 \sum_j (\mu_{ij}^2 + \sigma_{ij}^2 - 1 - \log\sigma_{ij}^2)\) // formula (223.4)
- \(\text{loss} \leftarrow \frac1B \sum_i (R_i + \beta K_i)\) // negative ELBO when \(\beta = 1\)
- backpropagate through decoder, the sampling formula and encoder; update \(\theta, \phi\) with Adam
Figure 1 runs exactly these lines for one image, with numbers. Step through it once more now that you know where (223.4) to (223.7) come from: watch the noise point move from \(\varepsilon\) to \(\mathbf{z}\), the two loss terms appear, and the gradient arrow on \(\mu\) at the last line.
Sampling and the shape of the latent space
After training, the encoder is only needed to encode existing images. To generate, we use the prior and the decoder alone:
- generate: \(\mathbf{z} \sim \mathcal{N}(\mathbf{0}, I)\); return \(D_\theta(\mathbf{z})\) // the mean of \(p(\mathbf{x} \mid \mathbf{z})\); adding pixel noise only adds grain
- reconstruct: \(\mu \leftarrow \text{Encoder}_\phi(\mathbf{x})\); return \(D_\theta(\mu)\) // use the mean, not a sample, for a stable result
- interpolate between images \(\mathbf{x}_a, \mathbf{x}_b\): encode both to \(\mu_a, \mu_b\)
- for \(t\) in \(0, 0.1, \dots, 1\): return \(D_\theta\big((1-t)\mu_a + t\mu_b\big)\) // spherical interpolation keeps the norm typical of the prior
Because the KL term forces neighboring codes to overlap, small moves in latent space give small, meaningful changes in the image, and a straight line between two codes passes through plausible images. This smoothness is the reason a VAE latent is a good place to run a second generative model.
Figure 4 trains a real VAE in your browser and shows its latent space while it learns. The data are the 16 bar images of the PyTorch section below: 8 × 8 pixels, one bright row (8 images) or one bright column (8 images). The encoder is 64 → 16 (tanh) → 4 numbers (\(\mu\) and \(\log\sigma^2\) for a 2-D latent); the decoder is 2 → 16 (tanh) → 64 logits with a Bernoulli likelihood, so \(R\) is the binary cross-entropy summed over pixels. Each step runs Algorithm 1 on all 16 images with one fresh \(\varepsilon\) per image, and Adam with learning rate 0.01. Everything is seeded, so the same settings always give the same run.
- initialize encoder \(\phi\) and decoder \(\theta\) (seed 7); Adam, learning rate 0.01
- for step \(t = 1, 2, \dots\)
- \((\mu_i, \log\sigma_i^2) \leftarrow \text{Enc}_\phi(\mathbf{x}_i)\) for all 16 images
- \(\varepsilon_i \sim \mathcal{N}(\mathbf{0}, I)\); \(\mathbf{z}_i \leftarrow \mu_i + \sigma_i \odot \varepsilon_i\)
- \(R_i \leftarrow \text{BCE}(\mathbf{x}_i, \operatorname{sigmoid}(\text{Dec}_\theta(\mathbf{z}_i)))\), summed over 64 pixels
- \(K_i \leftarrow \tfrac12 \sum_j (\mu_{ij}^2 + \sigma_{ij}^2 - 1 - \log\sigma_{ij}^2)\)
- \(\text{loss} \leftarrow \bar{R} + \beta_t \bar{K}\) // bars: mean over the 16 images
- backpropagate (223.7 and the networks); Adam update of \(\phi, \theta\)
How to read Figure 4
Press Play. At first all 16 means sit near the origin with \(\sigma \approx 1\) and the decoder draws the same grey average everywhere (\(R \approx 45\) nats). Then the points spread out, the ellipses shrink, and each region of the grid starts to show one bar. With \(\beta = 1\), after 1,000 steps the mean reconstruction is about 4.3 nats and the KL about 4.1 nats; the 16 regions tile the prior with small gaps. Now Reset with \(\beta = 4\): after 1,500 steps the KL stays near 0.2 nats, all points stay near the origin with \(\sigma \approx 0.9\), the grid shows only blurry averages, and reconstruction stays near 25 nats. That is posterior collapse in a real nonlinear network. With \(\beta = 0.1\) the ellipses become tiny and the points move far out: good reconstructions, but the codes no longer fill the prior. Back steps one training step back by replaying from a saved checkpoint.
Two limits are worth knowing. First, the ELBO only asks that each \(q_\phi(\mathbf{z} \mid \mathbf{x})\) be close to the prior; their average over the data, the aggregate posterior, may still leave regions of the prior empty. Samples from those “holes” decode to poor images. Second, a richer prior helps: hierarchical VAEs such as NVAE stack many latent layers, and latent diffusion goes further by learning the prior with a diffusion model.
β-VAE: the rate–distortion knob
Higgins et al. (2017) put a weight \(\beta\) on the KL term:
The names come from information theory. The KL term (the rate) bounds how many nats the code carries about the image. The reconstruction term (the distortion) measures what is lost. \(\beta \gt 1\) buys a more compressed, more prior-like latent at the cost of worse reconstructions; Higgins et al. report that this tends to give disentangled dimensions (one dimension per factor such as position or size) on simple data sets. [Likely] \(\beta \lt 1\) gives sharper reconstructions and a latent that is less like the prior. Latent diffusion autoencoders use \(\beta\) around \(10^{-6}\), as we will see.
Posterior collapse
Posterior collapse means the encoder gives the same distribution, the prior, for every input: \(q_\phi(\mathbf{z} \mid \mathbf{x}) \approx p(\mathbf{z})\) for all \(\mathbf{x}\). The KL term is then 0, and the decoder learns to ignore \(\mathbf{z}\). You get a model that may still produce reasonable outputs, but the latent code is useless: you cannot encode, edit or interpolate.
Why would training choose this? Because carrying information in \(\mathbf{z}\) costs KL nats. If the decoder can explain the data almost as well without \(\mathbf{z}\), the cheapest solution is to send nothing. This happens in two classic settings:
- A decoder that is too strong. Bowman et al. (2015) trained a VAE for sentences with an LSTM decoder. An autoregressive decoder can model \(p(\mathbf{x})\) on its own, token by token, so the KL term quickly went to 0. The same risk exists for any VAE whose decoder is an LLM.
- A KL price that is too high relative to the reconstruction. This is the large-\(s^2\) or large-\(\beta\) case. Lucas et al. (2019) showed it already happens in a linear VAE, where everything can be solved exactly.
Here is that exact result, for one latent dimension per data direction. Let a data direction have variance \(\sigma_x^2\), the decoder be \(x = w z + \text{noise}\) with fixed noise variance \(s^2\), and the KL weight be \(\beta\). Maximizing the expected objective (223.9) over the encoder and over \(w\) gives:
- Read the symbols
- \(w^\star\)
- the best decoder weight; \(w^\star = 0\) means the decoder ignores this latent dimension (collapse)
- \(\beta s^2\)
- the threshold: a direction is encoded only if its data variance is larger than this
- \(R\)
- the KL (in nats) the dimension pays at the optimum
- \(D_{\text{mse}}\)
- the expected squared reconstruction error along this direction
So a VAE behaves like PCA with a noise floor: directions with more variance than \(\beta s^2\) get a latent dimension, and all weaker directions collapse. Raising \(\beta\) or \(s^2\) collapses more dimensions. Figure 5 shows it for six data directions with variances 4, 2, 1, 0.5, 0.25 and 0.125.
How to read Figure 5
A direction is encoded only when its bar rises above the orange threshold line \(\beta s^2\). At the default (\(\beta = 1\), \(s^2 = 0.5\)), three directions are active, the rate is \(\tfrac12(\log 8 + \log 4 + \log 2) = 2.079\) nats and the squared error is \(3 \times 0.5 + 0.5 + 0.25 + 0.125 = 2.375\). Push \(\beta\) to 4: only the variance-4 direction survives. This is posterior collapse, caused not by a bad optimizer but by the objective itself. Lower \(s^2\) (a more confident decoder) and more directions wake up.
The same thing happens in the nonlinear VAE of Figure 4, where nothing is solved exactly. Reset it with \(\beta = 4\): after 1,500 steps the mean KL is about 0.23 nats (against about 4.4 nats with \(\beta = 1\)), every \(\sigma\) stays near 0.9, all means stay close to the origin, and the reconstruction loss stays near 25 nats instead of about 3. Now tick KL warm-up and Reset, still with \(\beta = 4\). The decoder learns to use \(\mathbf{z}\) during the first 500 cheap steps, and at step 1,500 the mean KL is about 1.4 nats and the reconstruction about 19 nats. The collapse is only partly undone, because the final price \(\beta = 4\) is the same; but the run ends in a much better place. This is why warm-up is a standard fix.
Fixes that are used in practice
- KL annealing (warm-up): start with \(\beta = 0\) and raise it to 1 over the first thousands of steps, so the decoder learns to use \(\mathbf{z}\) before the KL price is charged (Bowman et al.).
- Free bits: do not charge for the first \(\lambda\) nats of KL in each latent group (Kingma et al., 2016). A dimension below that budget has no reason to collapse:
- Weaken the decoder: word dropout (replace some input tokens of the decoder by an unknown token, Bowman et al.), a smaller receptive field, or fewer layers, so the decoder must rely on \(\mathbf{z}\).
- Learn the decoder variance \(s^2\) instead of fixing it too high; or simply lower \(\beta\).
- Monitor it: log the KL per latent dimension and count active units, the dimensions whose mean \(\mu_j(\mathbf{x})\) actually varies across inputs (Burda et al. use variance above 0.01). [Likely] A total KL near 0 is not “good regularization”; it is collapse.
Why VAE samples are blurry
With the Gaussian decoder (223.5), the reconstruction loss is a squared error. The squared error has a simple optimum: if, for a given code, several sharp images are possible, the best single prediction is their average. The average of several sharp images that differ in small details (the exact position of an edge, the pattern of hair, the texture of grass) is a blurry image.
And in a VAE, the code is designed to leave details out: every nat of detail costs KL. Fine detail is high in entropy and low in squared-error importance, so it is the first thing the model drops. The decoder must then guess it, and a squared-error decoder guesses the average. On top of that, sampling the Gaussian \(p(\mathbf{x} \mid \mathbf{z})\) would only add independent pixel noise, so people show the mean \(D_\theta(\mathbf{z})\), which is smooth.
Figure 6 shows the effect on the simplest possible case: a one-dimensional edge whose exact position the code does not know.
How to read Figure 6
Grey lines are possible true images. Blue is what the decoder outputs. Under squared error the blue line is a ramp whose 10%–90% rise takes \(0.8\times\) the uncertainty window: the blur grows with the uncertainty. Under absolute error the output is sharp, but it is still just one choice, not a sample: no decoder trained with a pixel-wise loss can show you the full range of possibilities. That is what a GAN loss, or a separate generative model such as diffusion, adds.
The fixes used by real image autoencoders all change what “good reconstruction” means:
- Perceptual loss (LPIPS): compare deep features of a pretrained network instead of pixels, so a slightly shifted texture is not punished like a large error.
- Adversarial (GAN) loss: a discriminator punishes outputs that look unreal, so the decoder must commit to one sharp detail instead of averaging (VQGAN; see session 224).
- Hierarchical latents (NVAE and similar) carry more detail with less blur, at the cost of more complex models.
- Move generation elsewhere: keep the VAE as a near-lossless compressor and let a diffusion model learn the distribution over latents. This is latent diffusion.
VAEs as the first stage of latent diffusion
Rombach et al. (2021) observed that most bits of an image are imperceptible detail, while the generative model should spend its effort on semantics (layout, objects, style). So they split the work. Stage 1 is an autoencoder that removes imperceptible detail (perceptual compression). Stage 2 is a diffusion model trained on the latents (semantic compression), see session 228. The stage-1 model is a VAE in name, but it is trained very differently from the VAE of Kingma and Welling:
In the public latent-diffusion config for the f = 8, 4-channel autoencoder, \(\lambda_{\text{KL}} = 10^{-6}\), the GAN loss has weight 0.5 and only starts after 50,001 steps. [Certain] With such a tiny KL weight, this VAE is almost a plain autoencoder. That is on purpose:
- We never sample from its prior. The diffusion model is the prior. So we do not need \(q_\phi(\mathbf{z} \mid \mathbf{x})\) to match \(\mathcal{N}(\mathbf{0}, I)\); a small KL is enough to keep the latent values bounded and the space smooth, with no wild scale.
- Reconstruction quality is the ceiling for the whole system: the diffusion model can never produce detail the decoder cannot draw. So rate is cheap and distortion is expensive: \(\beta\) near 0.
- L1 + LPIPS + GAN avoid the blur of Figure 6, so the decoder draws sharp textures from a small code.
The compression arithmetic
An autoencoder with down-sampling factor \(f\) (each side shrinks by \(f\)) and \(c\) latent channels maps an \(H \times W \times 3\) image to \(\tfrac{H}{f} \times \tfrac{W}{f} \times c\). The compression ratio is
SD 1.x: \(f = 8, c = 4\), so \(r = 48\). SD3 and FLUX keep \(f = 8\) but use \(c = 16\), so \(r = 12\). A diffusion transformer then cuts the latent into \(p \times p\) patches, one token each, so the token count is \(\tfrac{H}{fp} \cdot \tfrac{W}{fp}\): 1,024 tokens for a 512 × 512 image with \(f = 8, p = 2\), and 4,096 tokens at 1024 × 1024. FLUX, for example, packs 2 × 2 latent patches into one token in its pipeline. [Certain]
Why did SD3 go from 4 to 16 channels? Because the decoder can only draw what the latent remembers. The SD3 paper (Esser et al., 2024) measured reconstruction for \(f = 8\) and 4, 8 or 16 channels: [Certain]
| Latent channels | Reconstruction FID ↓ | Perceptual similarity ↓ | SSIM ↑ | PSNR (dB) ↑ |
|---|---|---|---|---|
| 4 | 2.41 | 0.85 | 0.75 | 25.12 |
| 8 | 1.56 | 0.68 | 0.79 | 26.40 |
| 16 | 1.06 | 0.45 | 0.86 | 28.62 |
The catch, stated in the same paper: a richer latent is harder for the diffusion model to predict, so the 16-channel latent pays off only with a large enough diffusion model. This is the core trade-off of latent design: reconstruction quality versus how easy the latent is to model.
Latent scaling
Diffusion noise schedules assume data of roughly unit variance. The latents of these VAEs are not unit-variance (the KL weight is too small to force it), so pipelines rescale them:
The scaling factor \(s\) is 0.18215 for SD 1.x and 0.13025 for SDXL; newer VAEs such as those of SD3 and FLUX also use a shift \(b\). [Certain] For SD 1.x, \(1/0.18215 = 5.49\), which suggests a raw latent standard deviation of about 5.5; the factor was chosen to bring the latents to unit variance. [Likely] Forgetting this step is a classic bug: the diffusion model then sees data at the wrong noise level.
How to read Figure 7
Try SD 1.x (512, \(f = 8\), \(c = 4\)): 64 × 64 × 4, ratio 48. Then SD3 / FLUX at 1024 (\(f = 8\), \(c = 16\)): 128 × 128 × 16, ratio 12, and 4,096 tokens with \(p = 2\), 16 times the attention cost. Raising \(f\) to 16 cuts tokens by 4 but each latent cell must summarize 256 pixels, so reconstruction suffers unless \(c\) grows; the ratio \(3f^2/c\) stays the same when \(c\) grows by \(f^2\). Pixel-space diffusion corresponds to \(f = 1\), which this figure does not even offer: at 512 × 512 with \(p = 2\), that would be 65,536 tokens.
Video VAEs
Video generators compress time as well. CogVideoX uses a 3D causal VAE with 16 latent channels, 8 × 8 spatial and 4 × temporal compression. [Certain] “Causal” means a latent frame only looks at the current and earlier video frames, so the first frame is encoded on its own and the VAE can also encode a single image. For a 49-frame clip this gives \(1 + 48/4 = 13\) latent frames. [Likely] At 480 × 720, the latent is 13 × 60 × 90 × 16 = 1,123,200 numbers, versus 50,803,200 for the pixels: 45 times fewer. See session 231.
Why diffusion works well in this latent space
- Size: 12 to 48 times fewer numbers per image, and \(f^2 = 64\) times fewer transformer tokens than the same patch size on pixels, so both training and every sampling step are much cheaper.
- Smoothness: the small KL term and the convolutional encoder keep the latent locally smooth and bounded; nearby latents decode to similar images, so the step-by-step denoising has a well-behaved target.
- Right division of labor: the decoder (trained with LPIPS and GAN) handles textures, which squared-error diffusion objectives model poorly; the diffusion model handles layout and content.
- Reuse: one VAE is trained once and shared by many diffusion models, fine-tunes, ControlNets and LoRAs, because they all live in the same latent space.
A VAE in PyTorch
A complete, deterministic VAE on a toy data set: 16 images of 8 × 8 pixels, each with one bright horizontal or vertical bar. It uses a 2-D latent, a Bernoulli decoder (binary cross-entropy), the reparameterization trick, KL warm-up, and then checks the closed-form KL against a Monte Carlo estimate, samples from the prior, and interpolates. It runs in a few seconds on a CPU.
import torch, torch.nn as nn, torch.nn.functional as F torch.manual_seed(0) # toy data: 8x8 images with one bright bar (8 rows + 8 columns = 16 images), as 64-dim vectors bars = torch.zeros(16, 8, 8) for i in range(8): bars[i, i, :] = 1.0 # horizontal bar in row i bars[8 + i, :, i] = 1.0 # vertical bar in column i data = bars.view(16, 64) class VAE(nn.Module): def __init__(self, d_in=64, d_h=128, d_z=2): super().__init__() self.enc = nn.Sequential(nn.Linear(d_in, d_h), nn.ReLU(), nn.Linear(d_h, 2 * d_z)) self.dec = nn.Sequential(nn.Linear(d_z, d_h), nn.ReLU(), nn.Linear(d_h, d_in)) def forward(self, x): mu, logvar = self.enc(x).chunk(2, dim=-1) # q(z|x) = N(mu, diag(exp(logvar))) eps = torch.randn_like(mu) # noise from N(0, I), no gradient needed z = mu + torch.exp(0.5 * logvar) * eps # reparameterization, formula (223.6) logits = self.dec(z) # Bernoulli decoder p(x|z) return logits, mu, logvar def loss_terms(x, logits, mu, logvar): rec = F.binary_cross_entropy_with_logits(logits, x, reduction="none").sum(-1) # -log p(x|z), summed over pixels kl = 0.5 * (mu.pow(2) + logvar.exp() - 1 - logvar).sum(-1) # formula (223.4) return rec.mean(), kl.mean() model = VAE() opt = torch.optim.Adam(model.parameters(), lr=3e-3) for step in range(3000): x = data[torch.randint(0, 16, (64,))] logits, mu, logvar = model(x) rec, kl = loss_terms(x, logits, mu, logvar) beta = min(1.0, step / 1000) # KL warm-up against posterior collapse loss = rec + beta * kl # negative ELBO when beta = 1 opt.zero_grad(); loss.backward(); opt.step() with torch.no_grad(): logits, mu, logvar = model(data) rec, kl = loss_terms(data, logits, mu, logvar) print(f"reconstruction {rec:.2f} nats, KL {kl:.2f} nats, -ELBO {rec + kl:.2f} nats") print(f"best possible -log p(x) = log 16 = {torch.log(torch.tensor(16.)):.2f} nats") # closed-form KL vs Monte Carlo estimate for the first image m, s = mu[0], torch.exp(0.5 * logvar[0]) z = m + s * torch.randn(200000, 2) logq = torch.distributions.Normal(m, s).log_prob(z).sum(-1) logp = torch.distributions.Normal(0., 1.).log_prob(z).sum(-1) print(f"KL closed form {0.5*(m**2 + s**2 - 1 - 2*s.log()).sum():.3f}, Monte Carlo {(logq - logp).mean():.3f}") # generation: sample z from the prior and decode x_new = torch.sigmoid(model.dec(torch.randn(4, 2))).view(4, 8, 8) print("pixels above 0.5 in 4 samples:", (x_new > 0.5).sum(dim=(1, 2)).tolist()) # interpolation between a horizontal and a vertical bar in latent space for t in [0.0, 0.5, 1.0]: zt = (1 - t) * mu[0] + t * mu[8] img = torch.sigmoid(model.dec(zt)).view(8, 8) print(f"t={t}: row0 mean {img[0].mean():.2f}, col0 mean {img[:, 0].mean():.2f}")
On PyTorch 2.8 (CPU) this prints a reconstruction of 0.07 nats, a KL of 4.11 nats, so \(-\text{ELBO} = 4.18\) nats, against the best possible \(-\log p(\mathbf{x}) = \log 16 = 2.77\) nats. The bound is valid (4.18 ≥ 2.77), and the 1.41-nat difference is partly the ELBO gap and partly an imperfect model. The closed-form KL of the first image is 4.242 and the Monte Carlo estimate 4.241. Each prior sample has 8 or 9 bright pixels, so it decodes to a clean bar. The interpolation starts at the row-0 bar (row mean 1.00) and ends at the column-0 bar (column mean 1.00); halfway it passes through a different bar, because in a 2-D latent the 16 images sit side by side. Other versions or hardware may print slightly different numbers.
Where this is used
| Setting | How the VAE is used | What matters |
|---|---|---|
| Text-to-image (SD 1.x, SDXL, SD3, FLUX) | KL-regularized autoencoder, \(f = 8\); 4 channels (SD 1.x, SDXL) or 16 (SD3, FLUX); diffusion runs on the scaled latent | Reconstruction quality caps image quality; scaling and shift factors must match the VAE |
| Video generation (CogVideoX and similar) | 3D causal VAE: 4× in time, 8 × 8 in space, 16 channels | Temporal compression makes long clips affordable; flicker can come from the VAE itself |
| Image editing, inpainting, super-resolution | Encode the input, edit or denoise in latent space, decode | The VAE round trip alone changes fine details such as small text and faces |
| Video retrieval and near-duplicate search | Compact latents or encoder means as features; mostly replaced by contrastive encoders such as CLIP for retrieval [Likely] | VAE latents keep appearance, not semantics; use them for pixel-level similarity, not meaning |
| Anomaly detection (industrial images, video surveillance frames) | Train on normal data; a high reconstruction error or low ELBO flags anomalies | Known pitfall: likelihood models can give high likelihood to simple out-of-distribution images [Likely] |
| Text VAEs and sentence latents | LSTM or transformer decoder conditioned on \(\mathbf{z}\) | Posterior collapse is the main problem; KL annealing, free bits, word dropout |
| Discrete visual tokens for LLMs | VQ-VAE / VQGAN: a codebook replaces the Gaussian latent | Next session: session 224 |
Common mistakes
- Mixing mean and sum. Averaging the reconstruction over pixels but summing the KL over dimensions (or the reverse) changes the effective \(\beta\) by thousands. This alone can cause posterior collapse or a latent that ignores the prior.
- Predicting \(\sigma\) directly. A linear layer can output a negative number. Predict \(\log\sigma^2\) and use \(\exp(\tfrac12 \log\sigma^2)\); clamp it to a sane range for stability.
- Reading a KL near 0 as success. It means the latent carries no information. Check active units and per-dimension KL.
- Expecting sharp samples from an MSE decoder. Pixel-wise losses give the average of possible details (Figure 6). Use perceptual and adversarial losses, or a second-stage generative model.
- Forgetting the latent scaling factor (223.15) when training or sampling a latent diffusion model, or using the factor of a different VAE (SD 1.x and SDXL differ: 0.18215 vs 0.13025).
- Swapping the VAE without retraining. A diffusion model only works with the latent space it was trained on; a VAE with a different channel count or scale needs a new (or adapted) diffusion model.
- Decoding in low precision without checking. Some VAE decoders overflow in FP16 and return NaN or black images; decode in FP32 or BF16, or use a fixed variant. [Likely]
- Comparing ELBOs across setups. An ELBO depends on the decoder variance and on units (nats vs bits per dimension); compare like with like, and remember it is only a lower bound on \(\log p(\mathbf{x})\).
Interview questions
What is the difference between an autoencoder and a VAE? Can you sample from a plain autoencoder?basic
Derive the ELBO. What is the gap between the ELBO and \(\log p(\mathbf{x})\)?core
Why do we need the reparameterization trick? When can’t we use it?basic
Write the KL between \(\mathcal{N}(\mu, \operatorname{diag}\sigma^2)\) and \(\mathcal{N}(\mathbf{0}, I)\). Which dimensions carry information?basic
Why does the score-function (REINFORCE) estimator have higher variance than the reparameterized one?advanced
What is posterior collapse, why does it happen, and how do you fix it?core
Why are VAE samples blurry?core
Why does Stable Diffusion run diffusion in a VAE latent space, and why is its KL weight only 1e-6?advanced
SD3 moved from 4 to 16 latent channels. What is the trade-off?advanced
What does \(\beta\) do in a \(\beta\)-VAE, in rate–distortion terms?core
How would you detect that the latent of your VAE is not being used?core
Exercises
- Compute the KL from \(\mathcal{N}((0.5, -1),\ \operatorname{diag}(1, 0.25))\) to \(\mathcal{N}(\mathbf{0}, I)\). Which dimension costs more, and why?
- In Figure 2, set \(q\) to the true posterior for \(x = 1.5\). What are the ELBO, \(\log p(x)\) and the gap?
- A linear VAE has a data direction with variance 3, decoder noise \(s^2 = 0.5\) and \(\beta = 2\). Is this direction encoded? If yes, find \(w^\star\) and the KL it pays.
- A 768 × 768 image goes through an \(f = 8\), \(c = 4\) VAE, then a DiT with patch size 2. Give the latent shape, the compression ratio and the number of tokens.
- A 49-frame, 480 × 720 video goes through a causal video VAE with 4× temporal, 8 × 8 spatial compression and 16 channels. How many numbers does the latent have, and how many times smaller than the pixels is it?
- Explain in two sentences why raising the decoder noise \(s^2\) can cause posterior collapse, using formula (223.5).
Answerscheck
Key takeaways
- A VAE is a latent-variable model \(p(\mathbf{z})p_\theta(\mathbf{x} \mid \mathbf{z})\) (223.1) plus an encoder \(q_\phi(\mathbf{z} \mid \mathbf{x})\) that guesses the intractable posterior.
- \(\log p(\mathbf{x}) = \text{ELBO} + \KL(q \,\|\, p(\mathbf{z} \mid \mathbf{x}))\) (223.2); the ELBO is reconstruction minus a KL to the prior (223.3), with a closed-form KL (223.4) and a squared-error reconstruction for a Gaussian decoder (223.5).
- The reparameterization \(\mathbf{z} = \mu + \sigma \odot \varepsilon\) (223.6) gives low-variance gradients; the score-function estimator (223.8) is the high-variance fallback.
- \(\beta\) trades rate against distortion (223.9). Posterior collapse switches off latent dimensions whose data variance is below \(\beta s^2\) (223.10, 223.11); KL annealing, free bits (223.12) and weaker decoders fight it.
- Pixel-wise losses predict the average of possible details, so VAE samples are blurry; LPIPS and GAN losses fix the decoder.
- Latent diffusion uses a VAE with tiny KL weight (\(10^{-6}\)), L1 + LPIPS + GAN losses (223.13), compression \(3f^2/c\) (223.14: 48 for SD 1.x, 12 for SD3 and FLUX) and a latent scaling factor (223.15).
Key formulas
References for this session
- Kingma & Welling (2013). Auto-Encoding Variational Bayes
- Rombach et al. (2021). High-Resolution Image Synthesis with Latent Diffusion Models
- Noyan, Farré, Marafioti, Zohar, Vision Language Models (O’Reilly, 2026), chapter 10: Any-to-Any Models
- Kingma & Welling (2013). Auto-Encoding Variational Bayes
- Rezende, Mohamed & Wierstra (2014). Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Rombach et al. (2021). High-Resolution Image Synthesis with Latent Diffusion Models
- Noyan, Farré, Marafioti, Zohar, Vision Language Models (O’Reilly, 2026), chapter 10: Any-to-Any Models
- Bowman, Vilnis, Vinyals, Dai, Jozefowicz & Bengio (2015). Generating Sentences from a Continuous Space
- Kingma, Salimans, Jozefowicz, Chen, Sutskever & Welling (2016). Improving Variational Inference with Inverse Autoregressive Flow
- Lucas, Tucker, Grosse & Norouzi (2019). Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
- Higgins et al. (2017). beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
- Burda, Grosse & Salakhutdinov (2015). Importance Weighted Autoencoders
- Vahdat & Kautz (2020). NVAE: A Deep Hierarchical Variational Autoencoder
- Zhang, Isola, Efros, Shechtman & Wang (2018). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- Esser et al. (2024). Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
- Yang et al. (2024). CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
- van den Oord et al. (2017). Neural Discrete Representation Learning
- Esser et al. (2020). Taming Transformers for High-Resolution Image Synthesis