LLM & VLM courseBlog
56Track 4: Post-training and RL · session 4 of 25

The PEFT family: adapters, BitFit, prefix tuning, prompt tuning, P-Tuning v2

Full fine-tuning makes a new copy of every weight for every task. Parameter-efficient fine-tuning (PEFT) freezes the pretrained model and trains a very small set of new or chosen numbers instead. This session builds the first PEFT family from the ground up: adapters, BitFit, prompt tuning, prefix tuning, P-Tuning and P-Tuning v2, with their math, their real costs, and a clear way to choose between prompting, a linear probe, PEFT and full fine-tuning.

Time: 2.5 to 3 hours (reading 2 hours, exercises 45 minutes)Level: BeginnerBefore this: Sessions 17, 25, 33, 54 and 55

What you will learn

  • Why PEFT exists: storage, memory and serving costs when one base model must serve many tasks
  • Adapters (Houlsby): a small bottleneck with a skip connection inside each block, started as the identity, and its parameter count 2md + d + m
  • BitFit: train only the bias terms, why it works on BERT, and why it cannot be used on Llama-style models that have no biases
  • Prompt tuning and prefix tuning: learned virtual tokens at the input versus learned keys and values at every layer, and the exact attention identity that explains prefix tuning
  • P-Tuning (a prompt encoder) and P-Tuning v2 (deep prompts plus a classification head) and why they close the gap at small model sizes
  • How to choose between prompting, a linear probe, PEFT and full fine-tuning using data size, number of tasks, latency and model size

Why this matters

In session 55 we trained every weight of a model for one task. That works, but it has a price: each task gets its own full copy of the model. An 8B model in bf16 is about 16 GB. One hundred tasks are about 1.6 TB of weights, and each one needs its own GPU memory when it serves traffic. Parameter-efficient fine-tuning (PEFT) keeps one frozen copy of the pretrained model and trains only a tiny set of numbers per task, often less than 1% of the model.

This session covers the first PEFT family, the methods that came before LoRA. They are still asked about in interviews, they are still used, and they explain why LoRA was designed the way it was (session 57):

  • Adapters (Houlsby et al., 2019): add a small bottleneck network inside every block and train only that.
  • BitFit (Ben Zaken et al., 2021): add nothing; train only the bias terms that already exist.
  • Prompt tuning (Lester et al., 2021) and P-Tuning (Liu et al., 2021): learn a few “virtual tokens”, vectors placed in front of the input.
  • Prefix tuning (Li & Liang, 2021) and P-Tuning v2 (Liu et al., 2021): learn virtual keys and values at every layer.

The three families change three different places in the network: new layers (adapters), existing parameters (BitFit), or the input and the attention context (prompts and prefixes). The last section answers the practical question an engineer actually faces: should I write a better prompt, train a linear probe, use PEFT, or fine-tune everything?

Real systems: the Hugging Face peft library implements prompt tuning, prefix tuning and P-Tuning next to LoRA, so one base model can load many small task files. [Likely] Prompt-based PEFT also lets one batch mix requests for different tasks, because the shared weights do not change; only the prepended vectors differ per request. [Certain] The VLM book (Noyan et al., chapter 5) uses PEFT for vision-language post-training for the same reason: the base model is large and the task data is small.

The PEFT idea: freeze the base, train a small piece

Full fine-tuning starts from the pretrained weights \(\theta_0\) and changes all of them. PEFT freezes \(\theta_0\) and trains a small set of task parameters \(\phi\). The model for task \(n\) is the pair \((\theta_0, \phi_n)\). The storage cost for \(N\) tasks then changes like this:

\[ \text{full fine-tuning: } N\,|\theta_0| \qquad\qquad \text{PEFT: } |\theta_0| + N\,|\phi|, \quad |\phi| \ll |\theta_0| \tag{56.1} \]
Read the symbols
\(\theta_0\)
the pretrained weights; \(|\theta_0|\) is how many numbers they contain
\(\phi\)
the small set of task parameters that PEFT trains (adapters, biases, prompts, prefixes)
\(N\)
number of tasks (or customers, or languages) that share the base model

There are three ways to choose \(\phi\), and this session follows them in order:

  1. Add new modules to the network and train only them (adapters).
  2. Select a small subset of the existing weights and train only that subset (BitFit trains the biases).
  3. Add new inputs: learned vectors that the frozen network reads like tokens (prompt tuning, P-Tuning, prefix tuning, P-Tuning v2).

Almost every method also trains a new task head when the task is classification, as in session 55. For example, Houlsby et al. train the adapters, the layer-norm parameters and the final classification layer. [Certain] When we count “trained parameters” below, we count the backbone part and say “plus the head”.

Figure 1 shows where each method lives in one transformer block, and how many numbers it trains for three real models. Choose a method and a model.

trained for this taskfrozen (shared by all tasks)bias vector (BitFit trains these)where the count is now
  1. \(n \leftarrow 0\) // full fine-tuning: every box counts
  2. input: if prompt: \(n \mathrel{+}= p\,d\) // (56.5)
  3. for \(\ell = 1, \dots, L\):
  4. LN 1: if BitFit: \(n \mathrel{+}= \) its bias
  5. attention: if BitFit: \(n \mathrel{+}= \mathbf{b}_Q, \mathbf{b}_K, \mathbf{b}_V, \mathbf{b}_O\); if prefix: \(n \mathrel{+}= 2\,p\,d_{kv}\) // (56.6)
  6. if adapters: \(n \mathrel{+}= 2md + d + m\) // (56.3)
  7. LN 2 and MLP: if BitFit: \(n \mathrel{+}= \) their biases
  8. if adapters: \(n \mathrel{+}= 2md + d + m\)
  9. output side: other biases (BitFit); the new task head is counted apart
Figure 1. One transformer block (repeated \(L\) times) and the task head. The highlighted boxes are trained; everything else is frozen and shared. The readout counts the trained backbone parameters with formulas (56.3), (56.5) and (56.6), using the real sizes of each model from its Hugging Face config. Play runs the counting loop in the code box: a marker walks through layer 1 box by box, the counter \(n\) grows only where the chosen method trains something, and the strip at the bottom fills as layers 2 to \(L\) repeat the same count.
How to read Figure 1

Start with “Full fine-tuning”: every box is trained, and the per-task file is the whole model (16 GB for Llama 3.1 8B). Click “Adapters”: only the two small orange boxes in each block are trained, about 0.4% of Llama 3.1 8B at \(m = 64\). Click “BitFit” with BERT-base: about 0.09% (the small dots). Now switch the model to Llama 3.1 8B: the count drops to zero, because Llama has no bias vectors at all. Qwen2.5 has biases only on the query, key and value projections. Click “Prompt tuning” and “Prefix”: move the length slider and compare. Prompt tuning trains \(p \times d\) numbers; prefix tuning trains keys and values in every layer, so it is about \(2L\) times larger (less with grouped-query attention). Press Play with “BitFit” and BERT-base: layer 1 adds 768 (norm 1), 3,072 (Q, K, V, O biases) and 4,608 (norm 2 and the two MLP biases), 8,448 in total; the other 11 layers add 92,928 more, and the embedding norm and pooler biases bring it to 102,912. With “Full fine-tuning” on Llama 3.1 8B, one layer is 218,112,000 numbers and the embeddings plus output layer are 1,050,677,248.

Adapters: a small bottleneck inside each block

An adapter is a tiny two-layer network: it projects the hidden vector of size \(d\) down to a small size \(m\) (the bottleneck), applies a nonlinearity, and projects back up to \(d\). It has its own skip connection, so its output is “input plus a small correction”.

\[ \mathrm{Adapter}(\mathbf{h}) = \mathbf{h} + W_{\text{up}}\, f\!\left(W_{\text{down}}\,\mathbf{h} + \mathbf{b}_{\text{down}}\right) + \mathbf{b}_{\text{up}} \tag{56.2} \]
Read the symbols
\(\mathbf{h} \in \R^{d}\)
the output of a sub-layer (attention or MLP) for one token
\(W_{\text{down}} \in \R^{m \times d}\)
the down-projection to the bottleneck size \(m\), with bias \(\mathbf{b}_{\text{down}} \in \R^{m}\)
\(W_{\text{up}} \in \R^{d \times m}\)
the up-projection back to size \(d\), with bias \(\mathbf{b}_{\text{up}} \in \R^{d}\)
\(f\)
a nonlinearity such as GELU or ReLU
\(m\)
the bottleneck size, much smaller than \(d\) (Houlsby et al. chose from 8, 64 and 256)

Where it goes. Houlsby et al. add the adapter twice to each layer: once after the attention sub-layer and once after the feed-forward sub-layer. Each adapter is applied to the sub-layer output after its projection back to size \(d\), and before the residual addition. [Certain] So the block becomes \(\mathbf{x} \leftarrow \mathbf{x} + \mathrm{Adapter}(\mathrm{Attn}(\mathrm{LN}(\mathbf{x})))\), and the same for the MLP. (The original BERT is post-LN; the placement relative to the skip connection is the part that matters.)

How many numbers. One adapter has two matrices and two bias vectors:

\[ |\phi_{\text{adapter}}| = 2md + d + m, \qquad |\phi_{\text{model}}| = 2L\,(2md + d + m) \tag{56.3} \]

The first formula is from the paper. [Certain] The second counts two adapters in each of \(L\) layers. Houlsby et al. used adapters with about 0.5 to 8% of the original parameters, and the bottleneck \(m\) is the single knob that trades quality for size. [Certain]

Start as the identity. Because of the inner skip connection, an adapter with near-zero weights returns \(\mathbf{h}\) almost unchanged. So at the start of training, the network with adapters computes the same function as the pretrained network, and training only has to learn a small correction. Houlsby et al. draw the adapter weights from a zero-mean Gaussian with standard deviation \(10^{-2}\), truncated at two standard deviations, and found quality is robust below \(10^{-2}\) but drops when the initialization is too large. [Certain] Many later implementations simply set \(W_{\text{up}} = 0\), which makes the start exactly the identity (the same trick appears in LoRA). Figure 2 shows why the scale matters.

Figure 2. One adapter in BERT-base (\(d = 768\)). Left: the bottleneck, drawn to scale on a log axis. Right: the size of the correction \(\lVert \mathrm{Adapter}(\mathbf{h}) - \mathbf{h} \rVert / \lVert \mathbf{h} \rVert\) at initialization, computed live for a fixed random hidden vector and seeded random weights with the chosen standard deviation (both projections, truncated at \(2\sigma\)).
How to read Figure 2

At the paper's setting (\(\sigma = 10^{-2}\), \(m = 64\)) the correction is about 1% of the input: the network starts almost exactly as the pretrained one. Move \(\sigma\) to \(10^{-1}\): the correction becomes as large as the input itself, so the adapter destroys the pretrained features before training even begins. Because both projections scale with \(\sigma\), the correction grows like \(\sigma^2\): each step of 0.5 on the slider multiplies it by about 10. The bottleneck slider changes the parameter count (formula 56.3) roughly in proportion to \(m\). Houlsby et al. found mean validation accuracy almost flat across sizes 8, 64 and 256 (86.2%, 85.8%, 85.7%), so small adapters are often enough. [Certain]

Worked example: adapter size in BERT-base and Llama 3.1 8B

BERT-base (\(d = 768\), \(L = 12\), 109,482,240 parameters). With \(m = 64\), one adapter has \(2 \cdot 64 \cdot 768 + 768 + 64 = 99{,}136\) parameters. Two per layer and 12 layers give \(24 \times 99{,}136 = 2{,}379{,}264\), or 2.17% of the model.

Llama 3.1 8B (\(d = 4096\), \(L = 32\), 8,030,261,248 parameters). With \(m = 64\): \(64 \times (2 \cdot 64 \cdot 4096 + 4096 + 64) = 33{,}820{,}672\) parameters, 0.42% of the model. In bf16 that is a 67.6 MB file per task, compared with 16.06 GB for a full copy.

Results. On GLUE with BERT-large, adapters with sizes 8 to 256 scored 80.0 against 80.4 for full fine-tuning, training 3.6% of the parameters per task, and needing 1.3 times the parameters of one model for all 9 tasks instead of 9 times. [Certain]

The cost: latency. Adapters are extra layers in series. They cannot be merged into the frozen weights, because of the nonlinearity \(f\). At large batch sizes the extra work is small, but at batch size 1 it shows. The LoRA paper measured one forward pass of GPT-2 medium on one GPU: 19.8 ms without adapters and 23.9 to 25.8 ms with two adapter variants (+20.7% and +30.3%) at batch size 1 and length 128, but only +2.2% and +3.0% at batch size 32 and length 512. [Certain] This latency is the main reason LoRA was designed as a linear update that can be merged (session 57).

Which layers matter? Houlsby et al. removed trained adapters one layer at a time. Removing adapters from the lower layers (0 to 4) on MNLI barely changed the result; the upper layers carried most of the effect. Removing all adapters dropped MNLI to 37%, the majority-class score. [Certain] Later adapter variants put one adapter only after the MLP (Lin et al., 2020, called AdapterL in the LoRA paper) to save latency. [Certain]

BitFit: train only the biases

Every linear layer computes \(\mathbf{y} = W\mathbf{x} + \mathbf{b}\). BitFit (“bias-terms fine-tuning”) freezes every weight matrix \(W\) and every layer-norm gain, and trains only the additive bias vectors \(\mathbf{b}\): the biases of the query, key, value and output projections, of the two MLP layers, and of the layer norms. [Certain] Nothing new is added to the network, so inference cost does not change at all.

\[ \mathbf{y}_t = W\mathbf{x}_t + \mathbf{b}, \qquad \frac{\partial \mathcal{L}}{\partial \mathbf{b}} = \sum_{t} \frac{\partial \mathcal{L}}{\partial \mathbf{y}_t} \tag{56.4} \]
Read the symbols
\(W\)
a frozen weight matrix (no gradient is stored for it)
\(\mathbf{b}\)
the bias vector, the only trained part of the layer
\(\partial \mathcal{L} / \partial \mathbf{y}_t\)
the gradient arriving from above at token \(t\); the bias gradient is just its sum over tokens

A bias adds the same shift to every token. So BitFit can move each feature up or down and change which neurons in the MLP are active, but it cannot rotate or mix features. The surprise is how far that goes. On BERT-base, biases are 0.09% of the parameters, and the GLUE validation average was 82.4 for BitFit against 82.3 for full fine-tuning; a frozen model with only a trained head scored 62.1. [Certain] On BERT-large (biases 0.08%) BitFit scored 84.2 against 84.1. [Certain] The authors read this as evidence that fine-tuning mostly exposes knowledge the model already learned in pretraining, more than it teaches new knowledge. [Certain]

Two more findings are useful. Training only two bias vectors per layer, the query bias \(\mathbf{b}_q\) and the bias of the first MLP layer (\(\mathbf{b}_{m2}\) in the paper's notation, intermediate.dense.bias in Hugging Face BERT), keeps most of the quality with 0.04% of the parameters: 81.1 on average. [Certain] And training the same number of randomly chosen weights instead of biases is clearly worse (78.5). [Certain] BitFit also needs a larger learning rate: the paper used \(10^{-4}\) to \(10^{-3}\) for bias-only training against \(10^{-5}\) to \(5 \times 10^{-5}\) for full fine-tuning, and reports that optimization was more stable. [Certain]

Training cost. BitFit saves optimizer memory (no Adam states for \(W\)) and the weight-gradient matrix multiply in every layer. But the backward pass must still carry \(\partial \mathcal{L} / \partial \mathbf{x}\) down through every layer, because every layer has a trained bias. So activation memory is about the same as full fine-tuning. This is true for all methods here: if the first trained parameter sits in layer 1, backpropagation must reach layer 1.

Prompt tuning: learn the input, not the model

With prompting, we write words in front of the input (“Classify the sentiment:”). Each word becomes an embedding vector, and the model reads those vectors. Prompt tuning skips the words: it learns the vectors directly. These learned vectors are called a soft prompt or virtual tokens, because they live in the same space as token embeddings but do not need to be the embedding of any real token.

\[ X' = \big[\, P \,;\, E(x_1), \dots, E(x_T) \,\big] \in \R^{(p + T) \times d}, \qquad P \in \R^{p \times d}, \qquad |\phi| = p\,d \tag{56.5} \]
Read the symbols
\(P\)
the soft prompt: \(p\) trainable vectors of size \(d\), the only trained numbers
\(E(x_t)\)
the frozen embedding of real input token \(x_t\)
\([\,\cdot\,;\,\cdot\,]\)
concatenation along the sequence axis: the prompt vectors come first
\(p\)
prompt length (number of virtual tokens)

The model is not changed at all. Training sends gradients back through the whole frozen network into \(P\) only. Lester et al. used T5 models up to XXL (11B parameters), a default of 100 virtual tokens, a learning rate of 0.3 with Adafactor, and 30,000 steps per task. [Certain] That learning rate is huge compared with fine-tuning (about \(10^{-5}\)); the soft prompt is a small set of free vectors and needs large steps to move.

The scale result. The key finding: prompt tuning gets closer to full model tuning as the model grows, and at T5-XXL (11B) it matches it on SuperGLUE. [Certain] With 5 tokens on T5-XXL (\(d = 4096\)) the prompt has only 20,480 parameters, over five orders of magnitude fewer than the 11B model. [Certain] For small models the gap is large, as Figure 4 shows.

Details that matter (all from the paper's ablations [Certain]):

  • Length. Going from 1 token to more is critical for most sizes; beyond 20 tokens gains are small. The XXL model works even with a single token.
  • Initialization. Random initialization is worst. Initializing from embeddings of real vocabulary words, or of the class label words, is better, and the difference disappears at XXL size.
  • The base model's objective. T5 was pretrained with span corruption, which is a poor fit. Continuing T5's training with a plain language-model objective for up to 100K steps (“LM adaptation”) made prompt tuning work much better.
  • Mixed-task batches. Because the model is shared, one batch can hold examples for many tasks, each with its own prompt. The paper also ensembles five prompts on one frozen model.

P-Tuning (v1). Liu et al. (2021, “GPT Understands, Too”) had the same idea for knowledge probing and SuperGLUE, with two changes. First, the virtual tokens are placed inside a template next to real “anchor” words, for example around “The capital of [Britain] is [MASK]”. Second, the vectors are produced by a small prompt encoder (an LSTM or an MLP) instead of being free parameters, which makes training more stable. [Certain] They motivate it with how fragile manual prompts are: changing a single word in a prompt could drop LAMA accuracy by 20 points. [Certain] Unlike Lester et al., P-Tuning was also used together with fine-tuning the model.

Prefix tuning: virtual keys and values at every layer

Prompt tuning changes only the input layer. Its vectors must travel through all the frozen layers before they affect the output, and each layer can transform them in ways the prompt cannot control. Prefix tuning gives every layer its own learned vectors. In each attention layer \(\ell\), it adds \(p\) learned keys and \(p\) learned values in front of the real ones:

\[ K'_\ell = \big[\, P^{K}_{\ell} \,;\, X_\ell W_K \,\big], \qquad V'_\ell = \big[\, P^{V}_{\ell} \,;\, X_\ell W_V \,\big], \qquad \mathrm{Attn}\big(X_\ell W_Q,\; K'_\ell,\; V'_\ell\big) \tag{56.6} \]
Read the symbols
\(X_\ell\)
the hidden states of the real tokens entering layer \(\ell\)
\(W_Q, W_K, W_V\)
the frozen query, key and value projections of that layer
\(P^{K}_{\ell}, P^{V}_{\ell} \in \R^{p \times d_{kv}}\)
the learned prefix keys and values of layer \(\ell\); \(d_{kv}\) is the total key width (\(n_{kv}\) heads times head size \(d_h\))

Every real token can attend to the prefix as if \(p\) extra tokens came first, but the prefix itself has no query and is not computed from anything: it is just stored. The trained size is \(|\phi| = 2\,L\,p\,d_{kv}\). For Llama 3.1 8B with grouped-query attention (8 key-value heads of size 128, so \(d_{kv} = 1024\)) and \(p = 20\), this is \(2 \cdot 32 \cdot 20 \cdot 1024 = 1{,}310{,}720\) parameters, 0.016% of the model. A model with full multi-head attention would need 4 times more. At inference the prefix is exactly a pre-filled part of the KV cache (session 26): 2.5 MiB per sequence in bf16 for this example.

Li & Liang applied it to GPT-2 for table-to-text and to BART for summarization. Training about 0.1% of the parameters matched full fine-tuning with all the data, and did better in low-data settings and on topics not seen in training. [Certain] Best prefix lengths were about 10 for table-to-text and 200 for summarization; longer prefixes gave lower training loss but slightly worse test scores. [Certain]

Why a prefix works: one exact identity

He et al. (2021) showed that adding a prefix to attention is the same as mixing the original attention output with an attention over the prefix alone. Take one query \(\mathbf{q}\), real keys and values \(K, V\), and prefix keys and values \(P^K, P^V\). Split the softmax sum into its prefix part and its real part:

\[ \mathrm{Attn}\big(\mathbf{q}, [P^K; K], [P^V; V]\big) = (1 - \lambda)\, \mathrm{Attn}(\mathbf{q}, K, V) + \lambda\, \mathrm{Attn}(\mathbf{q}, P^K, P^V) \tag{56.7} \]
\[ \lambda(\mathbf{q}) = \frac{\sum_{j} e^{\mathbf{q} \cdot \mathbf{p}^K_j / \sqrt{d_h}}}{\sum_{j} e^{\mathbf{q} \cdot \mathbf{p}^K_j / \sqrt{d_h}} + \sum_{i} e^{\mathbf{q} \cdot \mathbf{k}_i / \sqrt{d_h}}} \tag{56.8} \]
Read the symbols
\(\lambda(\mathbf{q})\)
the share of attention that this query gives to the prefix (between 0 and 1); it depends on the query, so each token mixes differently
\(\mathbf{p}^K_j\)
prefix key \(j\); \(\mathbf{k}_i\) is real key \(i\)
\(d_h\)
the head size used to scale scores

The proof is two lines: the softmax denominator over all keys is the sum of the two partial sums, so the weights on the real keys are the old weights times \(1 - \lambda\), and the weights on the prefix are the prefix-only weights times \(\lambda\). So a prefix adds a vector \(\mathrm{Attn}(\mathbf{q}, P^K, P^V)\) to each token's attention output, with a gate \(\lambda\) chosen by the query. That is very close to what an adapter does: add a small learned correction to a sub-layer output. He et al. use this view to put adapters, prefix tuning and LoRA in one framework. [Certain] Figure 3 lets you check the identity.

prefix keys (learned)real keysoutput vector
Figure 3. One query attends to three real tokens (scores 2, 1, 0) and to \(p\) prefix keys that all have the chosen score. Left: the attention weights. Right: the 2-D value space. The real values give the original output (blue point); the prefix values give the prefix output (orange point); the new output (green) lies on the line between them at position \(\lambda\), exactly as formula (56.7) says.
How to read Figure 3

With \(p = 0\) the output is the original one. Add prefix keys: the real weights shrink by the same factor \(1 - \lambda\), so their ratios do not change, and the output slides toward the prefix output. A higher prefix score or a longer prefix both increase \(\lambda\). The readout computes the output twice, directly with one softmax over all keys and through formula (56.7), and prints the difference: it is at the level of rounding error. In a real model, the prefix keys are learned so that \(\lambda\) is large for the queries that need a change and small for the others.

Worked example: a prefix moves one attention output

A query has scores 2, 1 and 0 with three real keys, whose (1-D) values are 1, 0 and −1. Softmax weights: \(e^2, e^1, e^0\) divided by \(e^2 + e + 1 = 11.107\), so 0.665, 0.245 and 0.090. The original output is \(0.665 - 0.090 = 0.575\).

Add two prefix keys with score 1.5 each and value −2. Their partial sum is \(2e^{1.5} = 8.963\), so \(\lambda = 8.963 / (8.963 + 11.107) = 0.447\) by (56.8). The prefix-only output is −2. By (56.7) the new output is \((1 - 0.447)(0.575) + 0.447 \cdot (-2) = -0.575\). Two stored vectors flipped the sign of this token's attention output, and nothing in the frozen model changed.

Reparameterization. Li & Liang found that training \(P\) directly was unstable and very sensitive to the learning rate and initialization. [Certain] So during training they produce it from a smaller matrix through an MLP:

\[ P_\theta[i, :] = \mathrm{MLP}_\theta\big(P'_\theta[i, :]\big), \qquad P'_\theta \in \R^{p \times k} \tag{56.9} \]

with \(k = 512\) for table-to-text and \(k = 800\) for summarization. After training, the MLP is thrown away and only the computed \(P\) is stored. [Certain] So the MLP changes how the optimizer sees the problem, not what is saved. P-Tuning's LSTM prompt encoder plays the same role. Li & Liang also tested “embedding-only” (prompt tuning style: learn only input vectors) on E2E: BLEU 62.2 with 10 tokens against 69.7 for the full prefix. [Certain] Deep prompts are much more expressive at the same model size.

P-Tuning v2: deep prompts that work at every size

By 2021 the picture was confusing. Prompt tuning matched fine-tuning at 10B parameters but was much worse on models of a few hundred million, and it failed on hard sequence labeling tasks such as named-entity recognition and extractive question answering, where every token needs its own label (session 55). P-Tuning v2 (Liu et al., 2021) is not a new mechanism. It is prefix tuning (“deep prompt tuning”) with a careful recipe for language understanding tasks [Certain]:

  • Prompts at every layer, as in prefix tuning. This raises the trained fraction from about 0.01% to 0.1% to 3%, and puts the prompts close to the output where they have a direct effect.
  • Reparameterization is optional. The MLP helped on some datasets (RTE, CoNLL04) and hurt on others (CoNLL12), so it is a hyperparameter.
  • Prompt length depends on the task: simple classification prefers short prompts (under 20), hard sequence labeling prefers around 100.
  • A plain classification head on the token representations, as in BERT, instead of a “verbalizer” (making the language-model head predict label words). Verbalizers do not work for sequence labeling.
  • Optional multi-task pretraining of the prompts, which gives a better starting point.
\[ \hat{\mathbf{y}}_t = \softmax\!\big(W_c\, \mathbf{h}^{(L)}_t + \mathbf{b}_c\big) \quad \text{for each token } t, \qquad \text{trained: } \{P^{K}_\ell, P^{V}_\ell\}_{\ell=1}^{L},\ W_c,\ \mathbf{b}_c \tag{56.10} \]
Read the symbols
\(\mathbf{h}^{(L)}_t\)
the frozen model's last hidden state for token \(t\), computed with the deep prompts in every layer
\(W_c, \mathbf{b}_c\)
a new, randomly initialized linear classification head (for sentence tasks, applied to one pooled or [CLS] vector)

With this recipe the paper reports results comparable to fine-tuning from 330M to 10B parameters, with only 0.1% to 3% of the parameters tuned, on SuperGLUE and on sequence labeling. [Certain] Figure 4 shows the SuperGLUE numbers from the paper.

fine-tuning (all weights)prompt tuning / P-Tuning (input only, frozen)P-Tuning v2 (every layer, frozen)
Figure 4. SuperGLUE development-set scores from Table 2 of the P-Tuning v2 paper, for BERT-large (335M), RoBERTa-large (355M), GLM-xlarge (2B) and GLM-xxlarge (10B). “Prompt tuning” is the paper's column for Lester et al. and P-Tuning, both with frozen models. Choose a task.
How to read Figure 4

On RTE, prompt tuning on the input only is almost 28 points behind fine-tuning on RoBERTa-large (58.8 versus 86.6), while P-Tuning v2 is ahead (89.5). At 10B the shallow method has almost caught up (89.9 versus 93.1). Click through the tasks: the blue bars climb with model size, which is Lester et al.'s scale result; the orange bars stay near the grey bars at every size, which is P-Tuning v2's result. A few tasks (WiC, WSC at 10B) are slightly below fine-tuning: “comparable” does not mean “always equal”.

Comparing the family

Before the table, run all four methods through the same small block with real numbers. Figure 5 is one attention sub-layer with \(d = 4\), three frozen tokens and one head. The last token's output \(\mathbf{y}\) should become a target \(\mathbf{y}^*\) for a new task, with loss \(L = \lVert \mathbf{y} - \mathbf{y}^* \rVert^2\). Each method gets its trained numbers in a different place: BitFit in the bias rows, prompt tuning in new input rows, prefix tuning in new key and value rows, adapters in a new module after the output projection. Everything else is frozen. Step through the forward pass and watch where the orange numbers enter; then switch to “after training” to see what 300 Adam steps on those numbers alone can do.

positive valuenegative valuetrained number (orange outline)attention weight on a real tokencurrent step
  1. \(X \leftarrow [\mathbf{x}_1; \mathbf{x}_2; \mathbf{x}_3]\) // frozen embeddings
  2. prompt: \(X \leftarrow [P; X]\) // trained rows (56.5)
  3. \(\mathbf{q} \leftarrow \mathbf{x}_3 W_Q + \mathbf{b}_Q\)
  4. \(K \leftarrow X W_K + \mathbf{b}_K,\; V \leftarrow X W_V + \mathbf{b}_V\) // BitFit trains the \(\mathbf{b}\)
  5. prefix: \(K \leftarrow [P^K; K],\; V \leftarrow [P^V; V]\) // (56.6)
  6. \(s_j \leftarrow \mathbf{q} \cdot \mathbf{k}_j / \sqrt{d}\)
  7. \(\mathbf{w} \leftarrow \softmax(\mathbf{s})\) // prefix share \(= \lambda\) (56.8)
  8. \(\mathbf{a} \leftarrow \sum_j w_j \mathbf{v}_j\)
  9. \(\mathbf{o} \leftarrow \mathbf{a} W_O + \mathbf{b}_O\)
  10. adapter: \(\mathbf{o} \leftarrow \mathbf{o} + W_{\text{up}} f(W_{\text{down}} \mathbf{o} + \mathbf{b}_{\text{down}}) + \mathbf{b}_{\text{up}}\) // (56.2)
  11. \(\mathbf{y} \leftarrow \mathbf{x}_3 + \mathbf{o}\); \(L \leftarrow \lVert \mathbf{y} - \mathbf{y}^* \rVert^2\)
  12. \(\theta \leftarrow \theta - \eta\,\mathrm{Adam}(\partial L / \partial \theta)\) // \(\theta\) = the orange numbers only
Figure 5. One attention sub-layer (\(d = 4\), one head, no norm) computed live in the browser for the last of three tokens. All frozen weights are fixed numbers; the trained numbers (orange outline) start from the usual initialization: the pretrained biases for BitFit, real word embeddings for the prompt, small values for the prefix, and \(W_{\text{up}} = 0\) for the adapter. “After 300 Adam steps” trains only those numbers on \(L\) with learning rate 0.05; the gradients are computed numerically (central differences). Grey lines in the code box are not used by the chosen method.
How to read Figure 5

Press Play with “Adapter”: the forward pass runs line by line and the step box shows every formula with its numbers. The adapter's correction is exactly zero at the start, so its output equals the frozen block's output (loss 1.4890). Switch to “Prompt tuning”: two new orange rows appear in the input, they get their own keys and values through the frozen projections, and they already change the output before training (loss 2.7877), because the prompt starts from word embeddings. “Prefix tuning” adds orange rows directly to K and V; the readout shows the prefix share λ. Now choose “After 300 Adam steps”: BitFit and the adapter reach the target (loss near 0), prefix tuning with \(p = 2\) also does, and prompt tuning with \(p = 1\) (only 4 numbers, which must pass through the frozen \(W_K\) and \(W_V\)) stops at 0.0568. The last step shows the gradient: only the orange numbers have one.

This tiny example fits one output vector, so every method with enough numbers can reach it. It is a picture of where each method acts, not a benchmark: on real tasks the ranking depends on the data and the model size (Figure 4).

MethodWhat is trainedTrained sizeInference costMixed tasks in one batch
Full fine-tuningEvery weight100% per taskNone extraNo (one model per task)
Linear probeOnly a new head on frozen features\(d \times \text{classes}\)None extraYes (heads are tiny)
AdaptersBottleneck modules, 2 per layer (56.3)0.1 to 8%Extra serial layers: +20 to 30% at batch 1 (GPT-2 medium)Possible, but needs per-example routing of adapter weights
BitFitExisting bias vectors0.04 to 0.09% (BERT); 0 for LlamaNonePossible (add per-example bias)
Prompt tuning / P-Tuning\(p\) input vectors (56.5)Under 0.01% for large models\(p\) extra tokens in every layerYes, naturally
Prefix tuning / P-Tuning v2\(p\) keys and values per layer (56.6)0.01 to 3%\(p\) extra KV-cache entries per sequence; shorter usable contextYes (per-request prefix in the KV cache)

Two costs are shared by all of them. Training memory: the frozen weights still sit on the GPU, and activations are still stored for backpropagation; PEFT removes only the optimizer states and gradients of the frozen weights. For an 8B model, full fine-tuning with AdamW in mixed precision needs about 16 bytes per parameter, 128 GB before activations (session 38); with PEFT it is about 16 GB of bf16 weights plus about 16 bytes per trained parameter (0.54 GB for the 33.8M adapter above). This is what lets an 8B model be adapted on one GPU. Optimization: prompt and prefix methods are harder to tune. The LoRA paper found prefix tuning “difficult to optimize”, with quality that changes non-monotonically with the number of trained parameters, and notes that the prefix takes up part of the sequence length. [Certain]

Putting it together: algorithms and code

Algorithm 1: set up a PEFT model
  1. input: pretrained model \(\theta_0\), method, size (\(m\) or \(p\)), task head shape
  2. load \(\theta_0\); set requires_grad = False for every parameter // frozen and shared
  3. if adapters: in each block insert Adapter after attention and after MLP, before the residual add; init \(W_{\text{up}} \approx 0\) // (56.2), identity at start
  4. if BitFit: set requires_grad = True for every parameter whose name ends with “bias” // check that there are any
  5. if prompt tuning: \(P \leftarrow\) embeddings of real words (or class labels), shape \(p \times d\) // (56.5)
  6. if prefix / P-Tuning v2: for each layer create \(P^K_\ell, P^V_\ell\) (optionally through an MLP) // (56.6), (56.9)
  7. add a new task head if the task needs one (classification, tagging) // (56.10)
  8. count trained parameters and print them // catch “0 trained” and “everything trained” bugs
  9. return model, list of trained parameters
Algorithm 2: train, save and serve many tasks
  1. for each task \(n\):
  2.   \(\text{optimizer} \leftarrow \mathrm{AdamW}\) over the trained parameters only // no states for \(\theta_0\)
  3.   use a larger learning rate than full fine-tuning (about \(10^{-4}\) to \(10^{-3}\); up to 0.3 for prompt tuning with Adafactor)
  4.   train with the task loss; choose the checkpoint on the validation set
  5.   if reparameterized: compute \(P = \mathrm{MLP}(P')\) once and drop the MLP
  6.   save \(\phi_n\) only (plus head) // kilobytes to megabytes, (56.1)
  7. serve: load \(\theta_0\) once; for each request, look up its task \(n\)
  8.   prompt / prefix: prepend \(P_n\) to its input or its KV cache; batch requests of different tasks together
  9.   adapters / BitFit: apply task \(n\)'s modules or biases (group requests by task if the kernel cannot mix)

The PyTorch program below has no downloads and runs on a CPU in about 10 seconds. It “pretrains” a tiny 2-layer encoder on one task (count six token types), then adapts it to a new task (is token 7 more frequent than token 11?) with each method, always with a new trained head. It implements adapters, BitFit, prompt tuning and prefix tuning by hand, in the same block, so you can see exactly where each one enters.

import torch, torch.nn as nn, torch.nn.functional as F
torch.manual_seed(0)
V, D, H, L, T = 24, 32, 4, 2, 16          # vocab, width, heads, layers, length

class Block(nn.Module):
    def __init__(s):
        super().__init__()
        s.ln1, s.ln2 = nn.LayerNorm(D), nn.LayerNorm(D)
        s.qkv, s.o = nn.Linear(D, 3 * D), nn.Linear(D, D)
        s.up, s.down = nn.Linear(D, 4 * D), nn.Linear(4 * D, D)
        s.ad1 = s.ad2 = None                # adapters (Houlsby): one per sub-layer
        s.pk = s.pv = None                  # prefix keys / values for this layer
    def attn(s, x):
        B, N, _ = x.shape
        q, k, v = s.qkv(x).view(B, N, 3, H, D // H).transpose(1, 3).unbind(2)  # (B,H,N,dh)
        if s.pk is not None:                # prefix tuning: prepend learned K, V
            k = torch.cat([s.pk.expand(B, -1, -1, -1), k], 2)
            v = torch.cat([s.pv.expand(B, -1, -1, -1), v], 2)
        y = F.scaled_dot_product_attention(q, k, v)          # bidirectional encoder
        return s.o(y.transpose(1, 2).reshape(B, N, D))
    def forward(s, x):
        a = s.attn(s.ln1(x));  a = a + s.ad1(a) if s.ad1 else a   # adapter before the residual add
        x = x + a
        m = s.down(F.gelu(s.up(s.ln2(x))));  m = m + s.ad2(m) if s.ad2 else m
        return x + m

class Net(nn.Module):
    def __init__(s, n_out):
        super().__init__()
        s.emb, s.pos = nn.Embedding(V, D), nn.Parameter(0.02 * torch.randn(T, D))
        s.blocks = nn.ModuleList(Block() for _ in range(L))
        s.lnf, s.head = nn.LayerNorm(D), nn.Linear(D, n_out)
        s.prompt = None                     # prompt tuning: learned input vectors
    def forward(s, ids):
        x = s.emb(ids) + s.pos
        if s.prompt is not None:
            x = torch.cat([s.prompt.expand(len(ids), -1, -1), x], 1)
        for b in s.blocks: x = b(x)
        return s.head(s.lnf(x).mean(1))     # mean-pool, then classify

def adapter(m):                             # down-project, nonlinearity, up-project
    a = nn.Sequential(nn.Linear(D, m), nn.GELU(), nn.Linear(m, D))
    nn.init.normal_(a[0].weight, std=1e-2); nn.init.zeros_(a[0].bias)
    nn.init.zeros_(a[2].weight); nn.init.zeros_(a[2].bias)   # starts as the identity
    return a

def data(n, g, task):                       # task A: count of 6 tokens; task B: is count(7) > count(11)?
    ids = torch.randint(0, V, (n, T), generator=g)
    c = torch.stack([(ids == t).sum(1) for t in range(6)], 1).float()
    return ids, (c if task == "A" else ((ids == 7).sum(1) > (ids == 11).sum(1)).long())

def train(net, params, task, steps, lr, seed):
    g = torch.Generator().manual_seed(seed); opt = torch.optim.AdamW(params, lr=lr, weight_decay=0.0)
    for _ in range(steps):
        x, y = data(64, g, task); out = net(x)
        loss = F.mse_loss(out, y) if task == "A" else F.cross_entropy(out, y)
        opt.zero_grad(); loss.backward(); opt.step()

# 1) "pretrain" a backbone on task A (counting tokens 0..5)
base = Net(6); train(base, base.parameters(), "A", 600, 3e-3, seed=1)
sd = {k: v for k, v in base.state_dict().items() if not k.startswith("head")}
xt, yt = data(2000, torch.Generator().manual_seed(99), "B")      # fixed test set for task B

def run(method):
    torch.manual_seed(2); net = Net(2); net.load_state_dict(sd, strict=False)
    for p in net.parameters(): p.requires_grad = False
    if method == "adapter":
        for b in net.blocks: b.ad1, b.ad2 = adapter(4), adapter(4)
    if method == "prompt":  net.prompt = nn.Parameter(net.emb.weight[:4].clone().unsqueeze(0))  # init from real tokens
    if method == "prefix":
        for b in net.blocks:
            b.pk = nn.Parameter(0.5 * torch.randn(1, H, 4, D // H)); b.pv = nn.Parameter(0.5 * torch.randn(1, H, 4, D // H))
    for n, p in net.named_parameters():
        if method == "full" or n.startswith("head") or ".ad" in n or n in ("prompt",) or n.endswith((".pk", ".pv")) \
           or (method == "bitfit" and n.endswith("bias")):
            p.requires_grad = True
    params = [p for p in net.parameters() if p.requires_grad]
    lr = 3e-4 if method == "full" else 3e-3                      # PEFT uses a larger learning rate
    train(net, params, "B", 400, lr, seed=3)
    with torch.no_grad(): acc = (net(xt).argmax(1) == yt).float().mean().item()
    n_tr = sum(p.numel() for p in params) - sum(p.numel() for p in net.head.parameters())
    return n_tr, acc

total = sum(p.numel() for k, p in base.state_dict().items() if not k.startswith("head"))
print("backbone parameters:", total)
for m in ["probe", "bitfit", "adapter", "prompt", "prefix", "full"]:
    n, acc = run(m); print(f"{m:8s} trained (besides head): {n:6d} ({100*n/total:5.2f}%)  test acc {acc:.3f}")
# backbone parameters: 26752
# probe    trained (besides head):      0 ( 0.00%)  test acc 0.751
# bitfit   trained (besides head):    736 ( 2.75%)  test acc 0.906
# adapter  trained (besides head):   1168 ( 4.37%)  test acc 0.923
# prompt   trained (besides head):    128 ( 0.48%)  test acc 0.758
# prefix   trained (besides head):    512 ( 1.91%)  test acc 0.803
# full     trained (besides head):  26752 (100.00%)  test acc 1.000

Read the output with care: this is a 27k-parameter toy, so percentages are much larger than in real models, and full fine-tuning wins easily because the model is tiny and data is unlimited. But the ordering repeats what the papers found for small models: input-only prompt tuning barely beats the linear probe (0.758 versus 0.751), deep prefixes help more (0.803), and adapters and BitFit, which change every layer's computation directly, get close to full fine-tuning. The bias count is checkable by hand: per block 96 + 32 + 128 + 32 (linear biases) + 32 + 32 (layer-norm biases), so \(2 \times 352 + 32 = 736\). In practice you would use the peft library (PromptTuningConfig, PrefixTuningConfig, PromptEncoderConfig for P-Tuning) instead of writing these by hand. [Likely]

How to choose: prompting, linear probe, PEFT or full fine-tuning

The methods in this session sit on a ladder. Each step up trains more, costs more, and can change the model more:

  1. Prompting (zero-shot or few-shot, session 143): no training at all. Best when you have very few labeled examples, the task is close to what the model already does, and you can afford long prompts at every request.
  2. Linear probe: freeze the model, train only a head on its features (as in session 55). Very cheap and fast to serve; works when the frozen features already separate the classes. The BitFit paper's frozen baseline (62.1 GLUE average versus 82.3) shows it often does not.
  3. PEFT: thousands to a few hundred thousand examples, many tasks or customers on one base model, limited GPU memory for training. Choose prompt or prefix methods when you need mixed-task batches; adapters or LoRA when quality matters more; avoid serial adapters when batch-1 latency is critical.
  4. Full fine-tuning: a large dataset, one or few tasks, a task far from pretraining (a new language, a new modality, a big behavior change), and the compute to train and store full copies. Also the reference you compare PEFT against.

These are rules of thumb, not laws. [Likely] The only reliable answer comes from a small experiment on your own data: try the cheaper option first and move up only if the validation score demands it. Figure 6 turns the rules into a small calculator.

Figure 6. A decision helper. The bars show the storage for all tasks in bf16, on a log scale, computed from (56.1) with the sizes from this session (adapters \(m = 64\), prompt \(p = 20\), prefix \(p = 20\) with Llama-style grouped-query attention). The suggestion is a rule of thumb that follows the ladder above. [Likely]
How to read Figure 6

With 100 tasks on Llama 3.1 8B, full fine-tuning needs 1.61 TB of weights, while 100 prompt-tuned tasks add only 16 MB to one 16 GB base model. Drag the number of examples down below about 100: the suggestion becomes prompting, because there is too little data to train anything reliably. Drag it up to a million with one task and tick “far from pretraining”: full fine-tuning becomes worth its cost. Tick “batch-1 latency”: serial adapters are no longer suggested.

Where this is used

SettingTypical methodWhy
Many customers or tasks on one hosted modelPrompt tuning, prefix tuning, LoRAOne base copy in GPU memory; tiny per-task files; prompt-style methods can mix tasks in one batch (56.1)
BERT-style classifiers on small GLUE-like dataBitFit, adaptersBitFit matched full fine-tuning on BERT with 0.08 to 0.09% of the parameters
Multilingual and multi-domain modelsAdapters (one per language or domain)New languages or domains are added without touching earlier ones; adapters can be stacked or swapped [Likely]
Sequence labeling: NER, extractive QA, taggingP-Tuning v2, adaptersDeep prompts and a token-level head; input-only prompts fail here
Generation with little data (table-to-text, summaries)Prefix tuningBeat fine-tuning in low-data settings and on unseen topics in Li & Liang
VLMs: adapting to a new image domain or document typeAdapters or LoRA on the language model; sometimes learned visual promptsThe base VLM is large and the domain data is small (session 189) [Likely]
Video classification and action recognition with a frozen video encoderLinear probe first, then adapters inside the encoderVideo backbones are expensive to fine-tune fully; adapters keep one shared backbone for many label sets [Likely]
Video retrieval and fingerprintingLinear probe or small adapters on embeddingsThe index is built from frozen embeddings; a full fine-tune would force re-embedding the whole library [Likely]
Key-moment detection and Video-RAG with a Video-LLMPrefix or prompt tuning per domain; LoRA for bigger changesPer-domain behavior (sports highlights, lectures) without separate model copies (session 205) [Likely]

Common mistakes

  • Using BitFit on a model without biases. Llama-style models have none, so “BitFit” trains only the head. Print the trained parameter count before training.
  • Forgetting to freeze (or to unfreeze). A wrong requires_grad loop either trains the whole model (no savings, a huge optimizer state) or trains nothing except the head. Count trained parameters.
  • Using the full fine-tuning learning rate. PEFT parameters usually need much larger rates: \(10^{-4}\) to \(10^{-3}\) for BitFit, 0.3 with Adafactor for prompt tuning in Lester et al. With \(10^{-5}\), prompt tuning seems not to learn.
  • Initializing adapters too large. A large random \(W_{\text{up}}\) breaks the pretrained function at step 0 (Figure 2). Start at or near the identity.
  • Random soft-prompt initialization on a small model. Initialize from real word embeddings or class-label embeddings, and use enough tokens (20 or more).
  • Expecting input-only prompt tuning to work on a 300M model or on tagging tasks. Use deep prompts (P-Tuning v2), adapters or LoRA there.
  • Ignoring the context cost of prompts and prefixes. \(p\) virtual tokens use \(p\) positions of the context window and \(p\) KV-cache entries per sequence; with long inputs, check that real tokens are not cut.
  • Saving the reparameterization MLP instead of the final prefix, or forgetting the new task head in the saved file. The task file must contain everything that differs from the base model.

Interview questions

What is parameter-efficient fine-tuning and why do we need it?basic
PEFT freezes the pretrained weights \(\theta_0\) and trains a small set of task parameters \(\phi\), often under 1% of the model. Benefits: (1) storage, \(|\theta_0| + N|\phi|\) instead of \(N|\theta_0|\) for \(N\) tasks, for example 16 MB for 100 prompt-tuned tasks instead of 1.6 TB of full 8B copies; (2) training memory, since no gradients or Adam states are kept for frozen weights (about 16 GB of bf16 weights instead of about 128 GB of weights, gradients and optimizer states for 8B); (3) serving, since one base model can serve many tasks; (4) less forgetting of the base model's abilities. Costs: some quality on hard or distant tasks, and harder hyperparameter tuning for some methods.
Describe the Houlsby adapter: shape, placement, initialization and parameter count.core
A bottleneck MLP with a skip connection: \(\mathbf{h} + W_{\text{up}} f(W_{\text{down}}\mathbf{h} + \mathbf{b}_d) + \mathbf{b}_u\), with \(W_{\text{down}} \in \R^{m\times d}\), \(m \ll d\). Two per layer, after the attention sub-layer and after the feed-forward sub-layer, applied to the sub-layer output before the residual add. Initialized near zero (Gaussian, std \(10^{-2}\), truncated) so the block starts as the identity and the pretrained function is preserved. Parameters: \(2md + d + m\) per adapter; for BERT-base with \(m = 64\), 99,136 per adapter and 2.38M in total (2.17%). Layer norms and the head are trained too. On GLUE with BERT-large: 80.0 versus 80.4 for full fine-tuning with 3.6% trained per task.
Why do adapters add inference latency while LoRA does not?core
An adapter is a nonlinear module in series with the layer, so it cannot be folded into the frozen weights; every token must go through two extra small matrix multiplies and an activation per adapter, which are sequential and poorly use the GPU at small batch sizes. LoRA is linear, \(W + BA\), so after training it can be merged into \(W\) and the model has the original shape. The LoRA paper measured GPT-2 medium at batch 1, length 128: 19.8 ms base versus 23.9 to 25.8 ms with adapters (+20.7% to +30.3%); at batch 32, length 512 the overhead was only 2 to 3%.
What does BitFit train, why does it work, and would you use it on Llama 3?core
Only the bias vectors: of the Q, K, V and output projections, of both MLP layers and of the layer norms, plus the task head. On BERT-base that is 0.09% of the parameters and it matched full fine-tuning on GLUE validation (82.4 versus 82.3), while a frozen model with a head got 62.1. A bias shifts features and changes which MLP neurons fire, which is enough to expose knowledge already in the model. Training only the query bias and the first MLP bias (0.04%) still gave 81.1. On Llama 3, no: it has no biases in linear layers and RMSNorm has no bias, so there is nothing to train. Qwen2.5 has only QKV biases (0.0017% of the 7B model), which is very little capacity.
Prompt tuning versus prefix tuning: what exactly differs?core
Prompt tuning learns \(p\) vectors at the input embedding layer only (\(p\,d\) parameters); the frozen layers then process them like tokens. Prefix tuning learns \(p\) keys and \(p\) values in every attention layer (\(2Lp\,d_{kv}\) parameters); they are not computed by the model, so each layer is steered directly. Prefix tuning is more expressive at the same model size (on E2E, BLEU 69.7 versus 62.2 for an input-only version), but has more parameters and needs per-layer KV entries at inference. Prompt tuning is the most parameter-efficient (under 0.01% for models above a billion parameters) and is competitive only at large scale.
Show that a prefix in attention is a gated mix of two attention outputs.advanced
For a query \(\mathbf{q}\), let \(S_P = \sum_j e^{\mathbf{q}\cdot\mathbf{p}^K_j/\sqrt{d_h}}\) and \(S_C = \sum_i e^{\mathbf{q}\cdot\mathbf{k}_i/\sqrt{d_h}}\). The softmax over all keys has denominator \(S_P + S_C\). The output is \(\big(\sum_j e^{\cdot}\mathbf{p}^V_j + \sum_i e^{\cdot}\mathbf{v}_i\big)/(S_P + S_C)\). Multiply and divide each part by its own sum: \(\frac{S_C}{S_P+S_C}\mathrm{Attn}(\mathbf{q},K,V) + \frac{S_P}{S_P+S_C}\mathrm{Attn}(\mathbf{q},P^K,P^V)\). With \(\lambda = S_P/(S_P+S_C)\) this is (56.7). So the prefix adds a query-gated vector to each token's attention output, like an adapter's additive correction (He et al., 2021).
Why did prompt tuning work at 11B but not at 300M, and what did P-Tuning v2 change?advanced
Input-only prompts have few parameters and only an indirect effect: the vectors pass through every frozen layer before reaching the output. A large model is flexible enough that a short prompt can select the right behavior; a small one is not. Lester et al. showed the gap to full model tuning closes as size grows and is gone at T5-XXL. P-Tuning v2 put prompts in every layer (deep prompt tuning, 0.1% to 3% of parameters), made reparameterization optional, tuned prompt length per task (under 20 for simple classification, about 100 for sequence labeling), used a plain classification head instead of a verbalizer, and optionally pretrained the prompts multi-task. It matched fine-tuning from 330M to 10B, for example RTE on RoBERTa-large 89.5 versus 86.6 fine-tuned and 58.8 for shallow prompt tuning.
What is the reparameterization trick in prefix tuning and P-Tuning, and what is saved?core
Instead of optimizing the prefix matrix \(P\) directly, which Li & Liang found unstable and sensitive to learning rate and initialization, they compute \(P = \mathrm{MLP}(P')\) from a smaller matrix \(P' \in \R^{p\times k}\) (k = 512 or 800). P-Tuning uses an LSTM or MLP prompt encoder for the same purpose. The extra network couples the vectors and changes the optimization geometry. After training only \(P\) is computed once and saved; the MLP is dropped, so the stored size and inference cost do not change. P-Tuning v2 found the benefit depends on the dataset.
Does PEFT reduce activation memory in training? What does it reduce?core
Mostly no. Backpropagation must still pass gradients with respect to activations from the loss down to the lowest trained parameter. For prompt tuning, BitFit and adapters in every layer, that is the whole network, so activations of all layers are stored as in full fine-tuning (gradient checkpointing still helps). PEFT removes the weight gradients and optimizer states of frozen weights (about 14 of the roughly 16 bytes per parameter in mixed-precision AdamW; the bf16 weights stay) and the weight-gradient matrix multiplies, which is a large part of backward compute. Activation memory drops only if the trained parameters sit in the top layers.
A team wants to serve 500 customer-specific variants of an 8B model. What do you suggest?advanced
Not full fine-tuning: 500 copies at 16 GB is 8 TB and needs a separate deployment per customer. Use PEFT on one shared base. If customers' requests must share batches and changes are mostly behavior or format, prompt or prefix tuning works naturally: each request carries its own soft prompt or prefix KV entries (prompt tuning with 20 tokens is 160 KiB per customer in bf16). If quality needs more capacity, use LoRA with a multi-adapter serving system, which can also batch different adapters (session 57); serial adapters cost latency at small batch. First check whether a good system prompt or retrieval already solves it, and evaluate each customer variant against the base model.
When would you choose a linear probe, and when full fine-tuning, over PEFT?basic
Linear probe: when the frozen features already separate the classes (quick test: train the probe and look at validation accuracy), when serving must be as cheap as possible, or when the embedding index must stay fixed (retrieval, video fingerprinting). Full fine-tuning: when there is a lot of data, few tasks, and the task is far from pretraining (new language or modality), or when PEFT leaves a clear quality gap that matters. PEFT sits in between and is the default for many tasks on one large model.

Exercises

  1. BERT-large has \(d = 1024\), \(L = 24\) and 335,141,888 parameters. How many parameters do Houlsby adapters with \(m = 64\) add, and what fraction of the model is that?
  2. Prompt tuning on T5-XXL (\(d = 4096\), about 11B parameters) with the default 100 tokens: how many parameters, and what percentage of the model?
  3. Prefix tuning on Llama 3.1 8B with \(p = 20\): how many trained parameters? How much extra KV cache in bf16 for a batch of 64 sequences? (Each token's K and V across all layers take \(2 \cdot 32 \cdot 1024\) numbers.)
  4. How many parameters does BitFit train in Llama 3.1 8B? In Qwen2.5 7B (\(d = 3584\), 28 layers, 4 key-value heads of size 128, biases on Q, K and V only)?
  5. A query has scores 2, 1, 0 with three real keys. You add two prefix keys with score 1.5. What is \(\lambda\)? What happens to the ratio of the weights on the first two real keys?
  6. You must store 50 task variants of Llama 3.1 8B in bf16. Compare full fine-tuning with Houlsby adapters at \(m = 64\).
Answerscheck
1. Per adapter \(2 \cdot 64 \cdot 1024 + 1024 + 64 = 132{,}160\); 48 adapters give 6,343,680, which is 1.89% of the model (plus layer norms and head). 2. \(100 \times 4096 = 409{,}600\) parameters, about 0.0037% of 11B. 3. \(2 \cdot 32 \cdot 20 \cdot 1024 = 1{,}310{,}720\) parameters. Per sequence the prefix is 20 cached tokens of \(2 \cdot 32 \cdot 1024 \cdot 2\) bytes = 128 KiB each, so 2.5 MiB; for 64 sequences 160 MiB (or store one shared copy if the kernel supports it). 4. Llama 3.1 8B: zero, it has no biases. Qwen2.5 7B: per layer \(3584 + 512 + 512 = 4{,}608\), times 28 = 129,024, about 0.0017% of 7.62B. 5. \(\lambda = 2e^{1.5}/(2e^{1.5} + e^2 + e + 1) = 8.963/20.071 = 0.447\). The ratio stays \(e^2/e^1 = e \approx 2.72\): all real weights are multiplied by the same factor \(1 - \lambda\). 6. Full: \(50 \times 16.06\) GB = 803 GB. Adapters: 33,820,672 parameters = 67.6 MB each, 3.38 GB for 50, plus one 16.06 GB base.

Key takeaways

  • PEFT freezes the pretrained model and trains a small \(\phi\) per task, so \(N\) tasks cost \(|\theta_0| + N|\phi|\) instead of \(N|\theta_0|\) (56.1). It saves storage and optimizer memory, not activation memory.
  • Adapters (56.2) are bottleneck MLPs with a skip connection, two per block, started near the identity; \(2md + d + m\) parameters each (56.3). Good quality, but extra serial latency at small batch.
  • BitFit trains only biases (56.4): 0.09% of BERT-base and matching full fine-tuning on GLUE, but useless on bias-free models like Llama.
  • Prompt tuning learns \(p\) input vectors (56.5); it is tiny and allows mixed-task batches, and it matches fine-tuning only for very large models.
  • Prefix tuning learns keys and values in every layer (56.6); a prefix is a query-gated mix of the original attention output and a prefix output (56.7, 56.8). Reparameterization (56.9) stabilizes training and is dropped afterwards.
  • P-Tuning v2 = deep prompts + a classification head + a per-task prompt length (56.10); comparable to fine-tuning from 330M to 10B parameters.
  • Choose by data size, number of tasks, latency and distance from pretraining: prompting, then a linear probe, then PEFT, then full fine-tuning; test the cheaper option first.

Key formulas

\[ \text{storage: } |\theta_0| + N|\phi| \ \text{ vs } \ N|\theta_0|; \qquad \mathrm{Adapter}(\mathbf{h}) = \mathbf{h} + W_{\text{up}} f(W_{\text{down}}\mathbf{h} + \mathbf{b}_d) + \mathbf{b}_u; \qquad |\phi| = 2L(2md + d + m) \]
\[ \text{prompt: } |\phi| = p\,d; \qquad \text{prefix: } |\phi| = 2Lp\,d_{kv}; \qquad \mathrm{Attn}(\mathbf{q}, [P^K;K], [P^V;V]) = (1-\lambda)\,\mathrm{Attn}(\mathbf{q},K,V) + \lambda\,\mathrm{Attn}(\mathbf{q},P^K,P^V) \]

References for this session

  1. Houlsby et al. (2019). Parameter-Efficient Transfer Learning for NLP
  2. Ben Zaken et al. (2021). BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
  3. Lester et al. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning
  4. Li & Liang (2021). Prefix-Tuning: Optimizing Continuous Prompts for Generation
  5. Liu et al. (2021). GPT Understands, Too (P-Tuning)
  6. Liu et al. (2021). P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
  7. Houlsby et al. (2019). Parameter-Efficient Transfer Learning for NLP
  8. Ben Zaken et al. (2021). BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
  9. Lester et al. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning
  10. Li & Liang (2021). Prefix-Tuning: Optimizing Continuous Prompts for Generation
  11. Liu et al. (2021). GPT Understands, Too (P-Tuning)
  12. Liu et al. (2021). P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
  13. He et al. (2021). Towards a Unified View of Parameter-Efficient Transfer Learning
  14. Hu et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models
  15. Hugging Face (code). PEFT library
  16. Noyan, Farré, Marafioti, Zohar, Vision Language Models (O’Reilly, 2026), chapter 5: Post-Training Vision Language Models