LLM & VLM courseBlog
55Track 4: Post-training and RL · session 3 of 25

Fine-tuning for classic NLP tasks

Many real NLP jobs are not chat: decide if a review is positive, find the names in a contract, or write a short summary of a long text. This session shows how to fine-tune a pretrained model for the three classic task shapes (one label per text, one label per token, text in and text out), how to choose an encoder or a decoder for each, and how to measure the result honestly when the classes are imbalanced (F1, entity F1, ROUGE).

Time: 2.5 to 3 hours (reading 2 hours, exercises 45 minutes)Level: BeginnerBefore this: Sessions 23, 24, 33 and 54

What you will learn

  • The three task shapes: sequence classification (sentiment, topic, intent), token classification (named entities, tagging) and sequence-to-sequence (summarization, translation)
  • How a classification head works: pool the hidden states ([CLS], mean, or last token), one linear layer, cross-entropy; and why it adds only about a thousand parameters
  • Token classification with BIO tags, how to align word labels with subword tokens, and why entity-level F1 is the right metric
  • Sequence-to-sequence fine-tuning with an encoder–decoder or a decoder-only model, and how ROUGE scores a summary
  • Class imbalance: why accuracy lies, class weights, focal loss and threshold tuning, and the exact link between inverse-frequency weights and the decision threshold
  • How to choose between a fine-tuned encoder, an encoder–decoder and a prompted or fine-tuned LLM for each task

Why this matters

In session 54 we taught a base model to answer like an assistant. But many real NLP jobs are not conversations. A bank wants to know if a complaint is about fraud. A search engine wants the names of people and places in a query. A newsroom wants a three-line summary of a long article. These are the classic NLP tasks, and fine-tuning a pretrained model is still the standard way to solve them well and cheaply.

All classic tasks come in three shapes:

  • Sequence classification: one label for the whole text. Sentiment (is this review positive or negative?), topic (sports, politics, tech), intent (does the user want to cancel, pay or talk to a person?).
  • Token classification: one label for every token. Named-entity recognition (NER: which words are a person, a place, a company?), part-of-speech tagging, finding personal data to hide.
  • Sequence-to-sequence: text in, new text out. Summarization and translation.

For each shape, this session answers four questions: what to put on top of the pretrained model, which loss to train, which model family to use (encoder, decoder, or encoder–decoder), and which number to report. The last question is where most projects go wrong. With 1% positive examples, a model that always says “no” is 99% accurate and completely useless.

Real numbers set the scene. BERT-base (110M parameters) fine-tuned on the SST-2 sentiment set (67k training sentences) reached 93.5% test accuracy, and BERT-large (340M) reached 94.9%. On CoNLL-2003 NER, fine-tuned BERT-base and BERT-large reached 92.4 and 92.8 test F1. [Certain] Fine-tuning took only 2 to 4 epochs. These small models are still used in production because they are fast: a 110M encoder costs about 60 times less compute per token than a 7B LLM.

Three task shapes on one pretrained model

A pretrained transformer turns \(T\) input tokens into \(T\) hidden vectors \(H = (\mathbf{h}_1, \dots, \mathbf{h}_T)\), each of width \(d\) (768 for BERT-base, 4096 for Llama 2 7B). These vectors already encode a lot of meaning (session 12). Fine-tuning adds a small head on top and then trains the head and (usually) the whole body on labeled examples. The three shapes differ only in which hidden vectors the head reads and what it outputs.

hidden state read by the headhead output (trained with the loss)ignored by the loss
Figure 1. The same pretrained body, four ways to read it. Choose a task shape, a model width \(d\) and the number of labels \(K\). The readout counts the new parameters the head adds.
How to read Figure 1

Tokens are at the bottom, the transformer body in the middle, the hidden states above it, and the head on top. Blue dots are the hidden states the head reads. In sequence classification only one vector is read (the [CLS] token); in token classification every token is read; in a decoder classifier the last real token is read, because only it has seen the whole text; in sequence-to-sequence the decoder’s normal language-model head predicts the next output word at every position. Notice how tiny the head is: for sentiment on BERT-base it is 1,538 numbers next to 110 million.

Sequence classification: one label per text

The head

We need one vector for the whole text, then a score per class. Three ways to get the vector (this is called pooling):

  • [CLS] pooling (BERT): use the hidden state of the special first token [CLS]. Through self-attention it can read every other token. The BERT paper does exactly this. [Certain]
  • Mean pooling: average the hidden states of all real (non-padding) tokens. Often as good or better for encoders (session 33).
  • Last-token pooling (decoders): in a causal model, token \(t\) sees only tokens \(1..t\). Only the last real token has seen the whole text, so the head reads that one. Hugging Face’s LlamaForSequenceClassification does this and finds the last token from the padding mask, and Raschka’s GPT-2 spam classifier (RAS1 chapter 6) also reads the last token. [Likely]

The head is one linear layer, then softmax:

\[ \mathbf{h} = \operatorname{pool}(\mathbf{h}_1, \dots, \mathbf{h}_T), \qquad \mathbf{z} = W\mathbf{h} + \mathbf{b}, \qquad \mathbf{p} = \softmax(\mathbf{z}) \tag{55.1} \]
Read the symbols
\(\mathbf{h}_t \in \R^{d}\)
the last-layer hidden state of token \(t\)
\(\operatorname{pool}\)
pick [CLS], or average the real tokens, or pick the last real token
\(W \in \R^{K \times d}, \mathbf{b} \in \R^{K}\)
the new head weights; \(K\) is the number of classes. These start random; everything else starts from the pretrained model
\(\mathbf{z}, \mathbf{p} \in \R^{K}\)
the class scores (logits) and the class probabilities

The loss is cross-entropy with the true class \(y\) (session 3). Its gradient with respect to the logits is simple, and it is what flows back into the whole network:

\[ \mathcal{L} = -\log p_y, \qquad \frac{\partial \mathcal{L}}{\partial \mathbf{z}} = \mathbf{p} - \mathbf{e}_y \tag{55.2} \]

Here \(\mathbf{e}_y\) is the one-hot vector of the true class. The gradient pushes the true class up by \(1 - p_y\) and every other class down by its own probability.

Worked example: one sentiment step

BERT-base, \(K = 2\) classes (positive, negative). The head has \(2 \times 768 = 1{,}536\) weights plus 2 biases: 1,538 new parameters, about 0.0014% of the 110M body.

For “the film was great” the head gives logits \(\mathbf{z} = (2.0,\ 0.5)\). Softmax: \(\mathbf{p} = (0.818,\ 0.182)\). The true class is positive, so \(\mathcal{L} = -\log 0.818 = 0.201\). The gradient on the logits is \(\mathbf{p} - \mathbf{e}_y = (-0.182,\ +0.182)\): raise the positive score, lower the negative one. Backpropagation carries this signal through \(W\) into the [CLS] vector and then into all 12 layers.

Multi-label: several labels per text

Topic tags often overlap: a news story can be about both sports and business. Then the classes are not exclusive and softmax is wrong, because it forces the probabilities to sum to 1. Use one independent sigmoid per label and binary cross-entropy:

\[ \mathcal{L} = -\sum_{k=1}^{K} \Big[ y_k \log \sigma(z_k) + (1 - y_k) \log\big(1 - \sigma(z_k)\big) \Big], \qquad y_k \in \{0, 1\} \tag{55.3} \]

At inference, each label is on if \(\sigma(z_k) \gt \tau_k\). The thresholds \(\tau_k\) are tuned per label on the validation set, not fixed at 0.5. In PyTorch this is BCEWithLogitsLoss.

The fine-tuning recipe

The BERT paper fine-tuned all weights, with dropout 0.1, and found these values work well across tasks: batch size 16 or 32, Adam learning rate \(5\times10^{-5}\), \(3\times10^{-5}\) or \(2\times10^{-5}\), and 2, 3 or 4 epochs. They note that large datasets (100k+ examples) were much less sensitive to these choices than small ones, and that BERT-large was sometimes unstable on small sets, so they ran several random restarts and kept the best on the dev set. [Certain] The learning rate is about 10 times lower than in pretraining: the body is already good, and big steps destroy what it knows (catastrophic forgetting). A short linear warmup and linear decay are standard (session 4).

Token classification: one label per token

BIO tags

In NER we want spans: “Ada Lovelace” is a person, “London” is a place. We turn spans into one label per word with the BIO scheme: B-X marks the beginning of an entity of type X, I-X a word inside it, and O a word outside any entity. With \(E\) entity types there are \(2E + 1\) tags. CoNLL-2003 has 4 types (PER, LOC, ORG, MISC), so 9 tags.

WordAdaLovelacemetCharlesBabbageinLondon
TagB-PERI-PEROB-PERI-PEROB-LOC

Why B at all? Without it, “gave Ada Charles’s book” with two people next to each other would look like one long name. B marks where a new entity starts.

Subwords: whose label is it?

Labels are given per word, but the model sees subword tokens (session 8). Suppose the tokenizer splits “Lovelace” into Love + ##lace (WordPiece marks a continuation piece with ##). Which token gets the label? The common answer, used in the BERT paper, is: put the word’s label on its first subword and ignore the rest. [Certain] “Ignore” means the label value −100, which PyTorch’s cross-entropy skips by default (ignore_index=-100). Special tokens ([CLS], [SEP]) and padding also get −100. Fast Hugging Face tokenizers give a word_ids() list that maps each token back to its word, which makes this alignment easy.

\[ \mathcal{L} = -\frac{1}{\sum_{t} m_t} \sum_{t=1}^{T} m_t \log p_{t,\,y_t}, \qquad m_t = \begin{cases} 1 & \text{if } y_t \ne -100 \\ 0 & \text{otherwise} \end{cases} \tag{55.4} \]
Read the symbols
\(p_{t,\,y_t}\)
the probability the head gives to the true tag of token \(t\): the same head as (55.1), applied to every \(\mathbf{h}_t\) separately, with no pooling
\(m_t\)
the mask: 1 for tokens with a real label, 0 for continuation pieces, special tokens and padding

This is the same loss-mask idea as in SFT, where only assistant tokens count (session 54). At inference, predict a tag for every token, keep the tag of each word’s first subword, and convert the BIO tags back into spans.

Measure entities, not tokens

Most tokens are O, so token accuracy looks high even when the model finds few entities. The standard NER metric (the CoNLL conlleval script, and the seqeval library) counts entities: a predicted entity is correct only if its type and both its start and end match a true entity exactly. Then precision, recall and F1 are computed over entities (formula 55.8 below). Try it in Figure 2.

Figure 2. Token accuracy against entity-level F1 on one sentence. The small grey line under each word is the true tag. Click a predicted tag to cycle it through O, B-PER, I-PER, B-LOC, I-LOC, B-ORG, I-ORG. Red rings mark tags that differ from the truth.
How to read Figure 2

Start with “A weak model”: 7 of 9 tags are right (77.8% token accuracy), but only 1 of the 3 entities is found exactly, so entity F1 is 0.333. One wrong tag (“Babbage” as O) cuts the second name short, and one wrong type (London as ORG) loses the place. Now try “Split one name”: a single wrong B-PER turns one true entity into two wrong ones, so it costs a false negative and two false positives. Finally, set “Charles” to I-PER instead of B-PER: CoNLL scoring treats an I- after O as the start of an entity, so the score stays perfect even though one tag is “wrong”.

Why encoders are the default for tagging

The tag of a word often depends on the words after it. In “Washington said”, Washington is a person; in “Washington is cold”, a place. An encoder’s hidden state \(\mathbf{h}_t\) has read the whole sentence, left and right. A decoder’s hidden state at position \(t\) has read only tokens \(1..t\), so it must tag “Washington” before seeing the next word. The code later in this session shows the effect on a toy task where a name is a person only if “said” follows: a bidirectional tagger gets entity F1 0.999, the same network with a causal mask gets 0.268. [Certain: our run] Older systems added a CRF layer on top to enforce valid tag sequences (no I-LOC right after B-PER); BERT did well without one. [Certain]

Sequence-to-sequence: summarization and translation

Here the output is new text \(y = (y_1, \dots, y_m)\), so we train the model to generate it, one token at a time, with teacher forcing: at each step the decoder sees the true previous output tokens.

\[ \mathcal{L} = -\frac{1}{m} \sum_{t=1}^{m} \log p_\theta\big(y_t \mid y_{\lt t},\ x\big) \tag{55.5} \]
Read the symbols
\(x\)
the source: the article to summarize, or the sentence to translate
\(y_{\lt t}\)
the true target tokens before position \(t\)
\(p_\theta\)
the model’s next-token distribution over the whole vocabulary; no new head is needed, the language-model head is reused

Two families can do this:

  • Encoder–decoder (BART, T5, and translation models like mBART): the encoder reads \(x\) bidirectionally once; the decoder writes \(y\) and reads the encoder through cross-attention (session 24). The loss covers only \(y\) by construction. BART was reported to work especially well when fine-tuned for generation tasks such as summarization. [Likely]
  • Decoder-only (GPT, Llama, Qwen): concatenate the source, a separator or prompt, and the target into one sequence, and mask the loss so only target tokens count. This is exactly SFT with a fixed prompt. It is the natural choice when you already serve an LLM, or when the task needs world knowledge.

At inference we decode with beam search or greedy decoding, usually with a length limit; summarizers also often block repeated n-grams. [Likely]

Scoring a summary: ROUGE

We cannot check a summary with exact match: many different summaries are good. ROUGE (Lin, 2004) compares the words of the candidate summary \(c\) with a human reference summary \(r\). ROUGE-N counts shared n-grams (sequences of \(n\) words):

\[ R_N = \frac{\sum_{g \in \text{ngrams}_N(r)} \min\big(\text{count}_c(g),\ \text{count}_r(g)\big)}{\sum_{g \in \text{ngrams}_N(r)} \text{count}_r(g)}, \qquad P_N = \text{same overlap} \div \text{number of n-grams in } c \tag{55.6} \]

\(R_N\) is a recall: what share of the reference’s n-grams appear in the candidate. The \(\min\) stops a candidate from scoring by repeating a word many times. ROUGE-L uses the longest common subsequence (LCS): the longest list of words that appears in both texts in the same order, not necessarily next to each other:

\[ P_L = \frac{\operatorname{LCS}(c, r)}{|c|}, \qquad R_L = \frac{\operatorname{LCS}(c, r)}{|r|}, \qquad F_L = \frac{2 P_L R_L}{P_L + R_L} \tag{55.7} \]

Lin’s paper defines ROUGE-N as a recall and ROUGE-L with a weighted F-measure; common libraries (such as Google’s rouge-score) report precision, recall and F1 for each, and most papers quote the F1. [Likely] Check which one a paper reports before you compare numbers. Figure 3 computes all three.

Reference:

Figure 3. ROUGE-1, ROUGE-2 and ROUGE-L for a candidate summary against one reference. Text is lower-cased and split on anything that is not a letter or digit, as the rouge-score package does by default. Green words also appear in the reference; underlined words form the longest common subsequence.
How to read Figure 3

“Good” drops one word (“new”) and scores high everywhere. “Same words, wrong order” keeps every word, so ROUGE-1 is unchanged at 0.933, but ROUGE-2 and ROUGE-L fall (0.462 and 0.400): word order only shows up in the longer measures. Its meaning is nonsense (“the budget approved the council”) yet ROUGE-1 cannot tell. “Paraphrase” says the same thing in other words and scores 0 on all three. Type your own candidate and watch the scores move.

For translation the classic metric is BLEU (n-gram precision with a brevity penalty), with chrF (character n-grams) and learned metrics such as COMET used more and more because they agree better with human judgments. [Likely] For both tasks, a small human or LLM-judge evaluation is worth more than a 0.5-point ROUGE gain.

Class imbalance and the right metric

Why accuracy lies

Real labels are rarely balanced. Fraud, toxic posts and rare intents may be 1–10% of the data. With 900 negatives and 100 positives, a model that always says “negative” has 90% accuracy. It finds no positives at all. We need numbers that look at each class. For one class, count true positives (TP), false positives (FP) and false negatives (FN):

\[ P = \frac{TP}{TP + FP}, \qquad R = \frac{TP}{TP + FN}, \qquad F_1 = \frac{2PR}{P + R} = \frac{2\,TP}{2\,TP + FP + FN} \tag{55.8} \]
Read the symbols
\(P\) (precision)
of the items the model called positive, the share that really are
\(R\) (recall)
of the truly positive items, the share the model found
\(F_1\)
the harmonic mean of the two; it is low if either one is low

With \(K\) classes, there are two ways to combine per-class scores:

\[ F_1^{\text{macro}} = \frac{1}{K} \sum_{k=1}^{K} F_{1,k}, \qquad F_1^{\text{micro}} = \frac{2 \sum_k TP_k}{\sum_k \big(2\,TP_k + FP_k + FN_k\big)} \tag{55.9} \]

Macro F1 gives every class the same weight, so rare classes count as much as common ones. Micro F1 pools all counts, so common classes dominate; for single-label multi-class problems micro F1 equals accuracy. Report macro F1 (and per-class numbers) when the rare classes matter. NER entity F1 is a micro F1 over entities.

Worked example: the “always negative” model

900 negatives, 100 positives, the model always predicts negative. \(\text{Accuracy} = 900/1000 = \mathbf{90\%}\). For the positive class: \(TP = 0\), so \(F_1 = 0\). For the negative class: \(P = 0.9\), \(R = 1\), \(F_1 = 2(0.9)(1)/1.9 = 0.947\). \(\text{Macro } F_1 = (0.947 + 0)/2 = \mathbf{0.474}\). Accuracy says 90%; macro F1 says the model is worse than a coin for the class we care about.

Fixing the training: class weights and focal loss

The model sees 9 negatives for every positive, so the loss is dominated by negatives. Three common fixes:

  1. Class weights. Multiply each example’s loss by a weight for its class. The usual choice is inverse frequency:
    \[ w_c = \frac{N}{K\, N_c}, \qquad \mathcal{L} = -\frac{\sum_i w_{y_i} \log p_{i,\,y_i}}{\sum_i w_{y_i}} \tag{55.10} \]
    \(N\) is the number of training examples, \(N_c\) the number in class \(c\). With 900/100: \(w_{\text{neg}} = 1000/1800 = 0.556\), \(w_{\text{pos}} = 1000/200 = 5\). Each positive now counts 9 times as much as a negative. In PyTorch: F.cross_entropy(logits, y, weight=w); the mean divides by the sum of the weights, as in (55.10).
  2. Resampling. Show positives more often (oversampling) or negatives less often (undersampling). Similar effect to weights, but changes the batches the model sees.
  3. Focal loss (Lin et al., 2017, from object detection): down-weight examples the model already gets right.
    \[ \mathcal{L}_{\text{focal}} = -(1 - p_y)^{\gamma} \log p_y \tag{55.11} \]
    With \(\gamma = 2\) (the value used in the paper [Certain]), an easy example with \(p_y = 0.9\) has its loss multiplied by \(0.1^2 = 0.01\): from 0.105 to 0.00105. A hard one with \(p_y = 0.1\) keeps 81% of its loss. The many easy negatives stop drowning out the few hard cases.

The hidden link: weights move the threshold

Class weights do not make the model “smarter”; mostly they move its decision boundary. Suppose the model is flexible enough to output the best possible probabilities. For an input whose true chance of being positive is \(p\), plain cross-entropy is minimized by predicting \(q = p\). With weights \(w_0, w_1\), the expected loss \(-w_1 p \log q - w_0 (1-p) \log(1-q)\) is minimized at

\[ q^\ast = \frac{w_1 p}{w_1 p + w_0 (1 - p)}, \qquad q^\ast \gt \tfrac12 \iff p \gt \frac{w_0}{w_0 + w_1} \tag{55.12} \]

With inverse-frequency weights (55.10), \(w_0/(w_0 + w_1) = \pi\), the share of positives. So a weighted model that uses the usual 0.5 cut behaves like an unweighted model that says “positive” whenever \(p \gt \pi\). With 10% positives, that is \(p \gt 0.1\). This explains why weights raise recall and lower precision, and why they can hurt F1: in the code below, plain CE reached F1 0.737 (P 1.000, R 0.584), weighted CE only 0.539 (P 0.447, R 0.680). [Certain: our run] The robust practice is: train (with or without weights), then tune the decision threshold on the validation set for the metric you report. Figure 4 lets you do this.

negativespositivespositives scaled to the same size (dashed)threshold: predict positive to the right
Figure 4. A fixed classifier on 10,000 test texts. Its score \(s\) (a logit) follows a normal distribution with mean +1.25 for positives and −1.25 for negatives, standard deviation 1. The curve heights are scaled by class size, so rare positives look small; the dashed curve shows their shape at full size. Counts are expected values, computed exactly.
How to read Figure 4

At \(\pi = 50\%\), threshold 0 is best and accuracy and F1 agree (0.894). Now set \(\pi = 5\%\) and keep the threshold at 0 (this is what inverse-frequency weights do, formula 55.12): recall stays 0.894, but there are now 1,004 false positives for 447 true ones, so precision is 0.308 and F1 0.458. “Plain CE” moves the threshold to \(s = 1.178\), the point where the true probability is 0.5: F1 rises to 0.632. “Best F1” finds \(s = 0.89\) with F1 0.658. The classifier never changed; only the threshold did. Also watch accuracy: always predicting negative already gives 95%.

Encoder or decoder for each task

All three families can do all three tasks. The question is which one gives the best quality per dollar and per millisecond for your data. A practical guide:

TaskGood defaultWhyWhen to choose something else
Sentiment, topic, intent (labeled data available)Fine-tuned encoder (BERT, RoBERTa, DeBERTa, ModernBERT) + [CLS] or mean headBidirectional reading, small and fast, easy to batch; one forward pass, no generationVery few labels or labels change often: prompt an LLM (zero- or few-shot), or fine-tune an LLM with a classification head or LoRA
Many labels, multi-label topicsEncoder + sigmoid head (55.3)Independent per-label thresholdsLabels are long descriptions: embed text and labels and compare
NER, tagging, personal-data detectionFine-tuned encoder + token headEach token sees both sides; exact spans; very fastOpen-ended extraction (“list every obligation in this contract”): an LLM generating JSON
SummarizationEncoder–decoder (BART, T5) or an instruction-tuned LLM with SFTEncoder reads the long input once, bidirectionally; LLMs bring more world knowledge and fluencyVery long inputs: a long-context LLM, or split and merge
TranslationEncoder–decoder trained on parallel text, or a large multilingual LLMEncoder–decoders are strong and efficient for a fixed language pairMany languages, style control, or document-level context: an LLM

Some numbers to reason with. Per token, the forward pass costs about \(2N\) FLOPs (session 25): \(2.2\times10^{8}\) for a 110M encoder, \(1.4\times10^{10}\) for a 7B LLM, about 60 times more. For a sentence classifier the encoder also needs one pass and no generation. BERT-style encoders were trained with at most 512 positions [Certain]; ModernBERT supports 8,192 tokens [Certain], which removes much of the old “split long documents” work. On narrow tasks with thousands of labels, fine-tuned small encoders often match or beat prompted LLMs; with only a handful of labels, a prompted LLM usually wins. [Likely] session 144 goes deeper into zero-shot classification with label words and calibration.

Putting it together

Algorithm 1: fine-tune a pretrained model for sequence or token classification
  1. input: pretrained body, labeled train / validation / test sets, task shape (sequence or token), number of labels \(K\)
  2. tokenize; for token tasks, map word labels to the first subword and set every other label to −100 // formula (55.4)
  3. add a head \(W \in \R^{K \times d}\), \(\mathbf{b}\), small random init; choose pooling: [CLS], mean, or last real token // formula (55.1)
  4. optionally compute class weights \(w_c = N/(K N_c)\) from the train set only // formula (55.10)
  5. AdamW, peak learning rate \(2\times10^{-5}\) to \(5\times10^{-5}\), short warmup, linear decay, 2 to 4 epochs, dropout 0.1
  6. for each batch: logits \(\leftarrow\) head(pool(body(tokens))); loss \(\leftarrow\) (weighted) cross-entropy; backward; clip; step
  7. after each epoch, evaluate on validation with the task metric (macro F1, entity F1); keep the best checkpoint
  8. tune the decision threshold(s) on validation for that metric // formula (55.12), Figure 4
  9. report the metric once on the test set, with the threshold fixed
Algorithm 2: entity-level precision, recall and F1 (CoNLL style)
  1. input: gold and predicted BIO tag lists for each sentence (one tag per word)
  2. function spans(tags): walk left to right; close the open entity when the tag is O, a B-, or an I- of another type; open a new entity at every B-X, and at an I-X that does not continue an open X
  3. \(G \leftarrow\) spans(gold) as (type, start, end) triples; \(\hat G \leftarrow\) spans(pred)
  4. \(TP \mathrel{+}= |G \cap \hat G|\), \(n_{\text{pred}} \mathrel{+}= |\hat G|\), \(n_{\text{gold}} \mathrel{+}= |G|\) // exact match of type and both ends
  5. return \(P = TP/n_{\text{pred}}\), \(R = TP/n_{\text{gold}}\), \(F_1 = 2TP/(n_{\text{pred}} + n_{\text{gold}})\) // formula (55.8), micro over entities

The PyTorch version below is deterministic and runs on a CPU in under a minute. It builds a tiny 2-layer encoder, fine-tunes it for (1) imbalanced sentiment with plain and weighted cross-entropy, and (2) NER where a name is a person only if the word “said” follows, once with full attention and once with a causal mask. The data are synthetic, so the numbers only illustrate the ideas.

import warnings; warnings.filterwarnings("ignore")
import torch, torch.nn as nn, torch.nn.functional as F
torch.manual_seed(0)
V, D, T = 40, 64, 16                       # vocab size, width, max length
PAD, CLS, SAID = 0, 1, 16                  # special ids; 16 plays the word "said"
POS, NEG = [2, 3, 4, 5], [6, 7, 8, 9]      # "good" words and "bad" words
IGNORE = -100                              # label value the loss skips

class Encoder(nn.Module):
    def __init__(self, n_labels, causal=False):
        super().__init__()
        self.causal = causal
        self.emb, self.pos = nn.Embedding(V, D), nn.Embedding(T, D)
        layer = nn.TransformerEncoderLayer(D, 4, 4 * D, dropout=0.1, batch_first=True, norm_first=True)
        self.body = nn.TransformerEncoder(layer, 2)
        self.head = nn.Linear(D, n_labels)               # the only new layer: W and b
    def forward(self, ids):                              # returns logits for every position
        mask = nn.Transformer.generate_square_subsequent_mask(ids.size(1)) if self.causal else None
        h = self.body(self.emb(ids) + self.pos.weight[: ids.size(1)], mask=mask,
                      src_key_padding_mask=ids.eq(PAD))
        return self.head(h)                              # (B, T, n_labels)

def train(model, X, Y, steps=400, seq=True, weight=None):
    opt = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=0.01)
    g = torch.Generator().manual_seed(2)
    for _ in range(steps):
        i = torch.randint(0, len(X), (64,), generator=g)
        out = model(X[i])
        if seq:   # sequence classification: read the [CLS] position, formula (55.2) with (55.6)
            loss = F.cross_entropy(out[:, 0], Y[i], weight=weight)
        else:     # token classification: every position, skip IGNORE, formula (55.4)
            loss = F.cross_entropy(out.reshape(-1, out.size(-1)), Y[i].reshape(-1), ignore_index=IGNORE)
        opt.zero_grad(); loss.backward(); opt.step()
    return model.eval()

def prf(pred, y):                                        # formula (55.8), positive class
    tp = ((pred == 1) & (y == 1)).sum().item()
    n_pred, n_true = (pred == 1).sum().item(), (y == 1).sum().item()
    return tp / max(1, n_pred), tp / max(1, n_true), 2 * tp / max(1, n_pred + n_true)

# ---- 1) sentiment with about 10% positive documents ----
def sentiment_data(n, g):
    X = torch.randint(17, V, (n, T), generator=g); X[:, 0] = CLS
    y = (torch.rand(n, generator=g) < 0.1).long()
    for k in range(n):
        good = int(torch.randint(0, 3, (1,), generator=g)) + int(y[k])
        bad = int(torch.randint(0, 3, (1,), generator=g)) + 1 - int(y[k])
        spots = torch.randperm(T - 1, generator=g)[: good + bad] + 1
        X[k, spots] = torch.tensor([POS[j % 4] for j in range(good)] + [NEG[j % 4] for j in range(bad)], dtype=torch.long)
    return X, y

g = torch.Generator().manual_seed(1)
Xtr, ytr = sentiment_data(3000, g); Xte, yte = sentiment_data(2000, g)
cnt = torch.bincount(ytr, minlength=2).float()
w = cnt.sum() / (2 * cnt)                                # formula (55.6)
print("class weights", [round(v, 2) for v in w.tolist()])
print("always negative : acc %.3f  P %.3f R %.3f F1 %.3f" % ((yte == 0).float().mean(), *prf(torch.zeros_like(yte), yte)))
for name, wt in [("plain CE", None), ("weighted CE", w)]:
    torch.manual_seed(0)
    m = train(Encoder(2), Xtr, ytr, weight=wt)
    with torch.no_grad(): p = m(Xte)[:, 0].argmax(-1)
    print("%-15s: acc %.3f  P %.3f R %.3f F1 %.3f" % (name, (p == yte).float().mean(), *prf(p, yte)))

# ---- 2) NER: a run of name words is an entity only if "said" comes right after it ----
TAGS = ["O", "B-PER", "I-PER"]
def ner_data(n, g):
    X = torch.randint(17, V, (n, T), generator=g); X[:, 0] = CLS
    Y = torch.zeros(n, T, dtype=torch.long)
    for k in range(n):
        t = 1 + int(torch.randint(0, 3, (1,), generator=g))
        while t < T - 4:
            L = 1 + int(torch.randint(0, 3, (1,), generator=g))
            X[k, t:t + L] = torch.randint(10, 16, (L,), generator=g)   # name words
            if torch.rand(1, generator=g) < 0.5:                       # "... said": a person
                X[k, t + L] = SAID; Y[k, t] = 1; Y[k, t + 1:t + L] = 2
            t += L + 2 + int(torch.randint(0, 3, (1,), generator=g))
    Y[:, 0] = IGNORE                                     # no label for [CLS]
    return X, Y

def spans(tags):                                         # BIO tags -> set of (start, end)
    out, start = set(), None
    for i, t in enumerate(tags + ["O"]):
        if start is not None and t != "I-PER":
            out.add((start, i - 1)); start = None
        if t == "B-PER" or (t == "I-PER" and start is None):
            start = i
    return out

Xtr, Ytr = ner_data(3000, g); Xte, Yte = ner_data(1000, g)
for name, causal in [("bidirectional", False), ("causal (decoder)", True)]:
    torch.manual_seed(0)
    m = train(Encoder(3, causal), Xtr, Ytr, seq=False)
    with torch.no_grad(): P = m(Xte).argmax(-1)
    tp = n_pred = n_gold = 0
    for k in range(len(Xte)):
        gold = spans([TAGS[v] for v in Yte[k, 1:].tolist()])
        pred = spans([TAGS[v] for v in P[k, 1:].tolist()])
        tp += len(gold & pred); n_pred += len(pred); n_gold += len(gold)
    keep = Yte != IGNORE
    acc = (P[keep] == Yte[keep]).float().mean().item()
    print("%-17s: token acc %.3f  entity P %.3f R %.3f F1 %.3f" % (
        name, acc, tp / max(1, n_pred), tp / n_gold, 2 * tp / (n_pred + n_gold)))

Output (PyTorch 2.8, CPU):

class weights [0.55, 5.23]
always negative : acc 0.901  P 0.000 R 0.000 F1 0.000
plain CE       : acc 0.959  P 1.000 R 0.584 F1 0.737
weighted CE    : acc 0.886  P 0.447 R 0.680 F1 0.539
bidirectional    : token acc 1.000  entity P 0.998 R 0.999 F1 0.999
causal (decoder) : token acc 0.842  entity P 0.237 R 0.309 F1 0.268

Read it line by line. “Always negative” is 90.1% accurate with F1 0. Plain CE is careful: every positive it predicts is right (P 1.000) but it finds only 58% of them. Weighted CE finds more (R 0.680) at the cost of many false alarms, as formula (55.12) predicts, and here its F1 is lower; tuning the threshold would let you pick any point between. The causal tagger cannot see “said” when it tags a name, so it can only guess; token accuracy (0.842) still looks fine while entity F1 collapses to 0.268.

Where this is used

ApplicationTask shapeTypical model and metric
Review sentiment, support-ticket routing, intent detection in voice assistantsSequence classificationFine-tuned encoder; macro F1, per-intent recall; strict latency budget
Spam, toxicity and policy moderationMulti-label sequence classification, heavy imbalanceEncoder + sigmoid heads, per-label thresholds; precision at a fixed recall
NER for search, personal-data redaction, medical and legal entity extractionToken classificationEncoder + BIO head; entity-level F1 (seqeval)
Invoice and form field extractionToken classification over text plus layoutLayout-aware encoders or VLMs (VLM chapter 8); field-level F1
News, meeting and call summariesSequence-to-sequenceBART / T5 or an SFT-tuned LLM; ROUGE plus human or LLM-judge checks
Machine translationSequence-to-sequenceEncoder–decoder or multilingual LLM; BLEU, chrF, COMET
Video classification and action recognitionSequence classification over frame or patch tokensVideo encoder + [CLS] or mean-pooled head; top-1 accuracy, macro F1 for rare actions
Key-moment detection and temporal action segmentation (goals, fouls, scene cuts)Token classification over time: one BIO-style tag per frame or clipBidirectional temporal encoder for offline video (it sees what comes after); event-level F1 with a time-overlap rule, just like entity F1
Video captioning and video summarizationSequence-to-sequence (video in, text out)Video encoder + text decoder or Video-LLM; ROUGE, CIDEr, human checks
Query routing in Video-RAG and search (“find a clip” vs “answer a question”)Sequence (intent) classificationSmall encoder in front of the expensive model; must be fast

Common mistakes

  • Reporting accuracy on imbalanced data. Always report the majority-class baseline next to it, plus per-class precision, recall and macro F1.
  • Misaligned NER labels. After subword tokenization the token list is longer than the word list. Map labels with word_ids(); put −100 on continuation pieces, special tokens and padding. Off-by-one shifts here give silently bad models.
  • Token-level F1 for NER. It rewards getting the many O tags right. Use entity-level F1 with exact span matching (seqeval or conlleval).
  • Decoder classifier reading the wrong position. With right padding, position −1 is a pad token. Read the last real token, and make sure the tokenizer has a pad token set (many LLM tokenizers do not by default).
  • Tuning thresholds or picking checkpoints on the test set. Use the validation set; touch the test set once.
  • Too high a learning rate. \(10^{-3}\) on a pretrained body often collapses it to one class. Start at \(2\times10^{-5}\) to \(5\times10^{-5}\) for full fine-tuning.
  • Silent truncation. A 512-token encoder cuts long documents; if the label depends on the end (for example the verdict in a long review), split into chunks and combine, or use a long-context model.
  • Softmax for multi-label data, or loss on the source in decoder-only seq2seq. Use sigmoid + BCE for overlapping labels, and mask the prompt so only target tokens are trained.

Interview questions

How do you fine-tune BERT for sentiment classification?basic
Tokenize each text with [CLS] at the start, take the final hidden state of [CLS] (or the mean of the real tokens), add a linear layer \(W \in \R^{K\times d}\) with softmax, and train all weights with cross-entropy (formulas 55.1, 55.2). Use AdamW with a small learning rate (\(2\times10^{-5}\) to \(5\times10^{-5}\)), warmup and linear decay, batch 16 or 32, 2 to 4 epochs, and pick the best checkpoint on validation. The head adds only \(Kd + K\) parameters: 1,538 for two classes on BERT-base.
[CLS] pooling, mean pooling, last-token pooling: when do you use which?core
In an encoder every position sees the whole text, so [CLS] or mean pooling both work; mean pooling is often a bit more robust, [CLS] is what BERT was pretrained to use (through next-sentence prediction). In a decoder, position \(t\) only sees tokens \(1..t\), so the only vector that has read the whole input is the last real token; with right padding you must find that position from the attention mask, not use index −1.
How do you assign NER labels when a word is split into several subword tokens?core
Put the word’s label on its first subword and give the other pieces the label −100 so the loss ignores them (formula 55.4); this is what the BERT paper did. Special tokens and padding also get −100. At inference, read the prediction of each word’s first subword. An alternative is to label every piece (B on the first, I on the rest), but then you must merge them back consistently before scoring at the word level.
Why is token accuracy a poor metric for NER, and what do you report instead?core
Most tokens are O, so a model that finds few entities still has high token accuracy. Also one wrong tag can break an entity, which token accuracy barely notices. Report entity-level precision, recall and F1 with exact matching of type and span (CoNLL scoring, seqeval). Example: 7 of 9 tags right (77.8%) can mean only 1 of 3 entities right (F1 0.333).
Why are encoders preferred to decoders for token classification?core
A token’s tag often depends on the words after it (“Washington said” vs “Washington is cold”). An encoder’s hidden state has seen both sides. A causal decoder must decide at position \(t\) using only tokens \(1..t\). On a toy task where a name is a person only if “said” follows, the same small network got entity F1 0.999 bidirectional and 0.268 causal. Decoders can still do extraction well by generating the entities after reading the whole input, but that costs generation time and needs output parsing.
Your positive class is 1% of the data. What do you do?core
First, the metric: stop using accuracy (99% for the trivial model); use precision, recall, F1 or PR-AUC for the positive class, or precision at a recall target set by the business. Then training: class weights or oversampling, or focal loss; make sure every batch has some positives. Then the decision: tune the threshold on validation for the chosen metric. Also look at the data: more labeled positives (active learning, LLM-assisted labeling) often help more than any loss trick.
What do inverse-frequency class weights do to the decision threshold?advanced
For a flexible model, weighted cross-entropy learns \(q = w_1 p / (w_1 p + w_0(1-p))\) instead of the true probability \(p\). Predicting \(q \gt 0.5\) is the same as \(p \gt w_0/(w_0 + w_1)\). With \(w_c = N/(K N_c)\) this cut equals the positive share \(\pi\). So weights mainly lower the threshold from 0.5 to \(\pi\): more recall, less precision, and probabilities that are no longer calibrated. You can get the same effect, more flexibly, by training unweighted and choosing the threshold on validation.
Macro F1 vs micro F1?basic
Macro F1 averages the per-class F1 scores, so each class counts the same; rare classes matter. Micro F1 pools TP, FP and FN over classes, so frequent classes dominate; for single-label multi-class problems it equals accuracy. With 90/10 data, the always-majority model has accuracy and micro F1 0.90 but macro F1 0.474.
What does ROUGE measure, and what does it miss?core
ROUGE-N counts n-gram overlap with a reference summary (recall-oriented; libraries also give precision and F1); ROUGE-L uses the longest common subsequence, so it rewards correct order. It misses paraphrases (a correct summary in other words can score 0), factual errors (swap two names and ROUGE-1 does not change), and fluency. With one reference it also punishes valid different choices. Use it to compare close systems, and add human or LLM-judge checks for faithfulness.
Encoder–decoder or decoder-only for summarization?core
An encoder–decoder (BART, T5) reads the source once with bidirectional attention and the decoder cross-attends to it; it is efficient and was the standard for supervised summarization and translation. A decoder-only LLM treats source and summary as one sequence (mask the loss on the source); it brings more world knowledge and instruction following and is the default if you already serve one. For a fixed task with lots of pairs and tight cost limits, a fine-tuned encoder–decoder is still a strong choice.
Fine-tuning BERT-large on a small dataset is unstable: some runs fail completely. Why and what helps?advanced
With few examples, a few noisy updates early on can push the model far from the pretrained weights, and the randomly initialized head sends large random gradients into the body. The BERT authors simply ran several random restarts and kept the best on dev. Other common fixes: lower learning rate with warmup, more epochs on small data, bias-corrected Adam, freezing the body for the first steps (train the head first), or parameter-efficient methods like LoRA that change fewer weights. [Likely]
In production, would you prompt an LLM or fine-tune an encoder for intent classification?advanced
It depends on labels, latency and change rate. With thousands of labeled examples, a fixed label set and a latency budget of tens of milliseconds, a fine-tuned encoder is cheaper (about 60 times fewer FLOPs per token than a 7B model, one forward pass, no generation) and usually as accurate. With few labels or labels that change weekly, prompt an LLM. A common hybrid: label data with an LLM, train an encoder, and route low-confidence cases to the LLM.

Exercises

  1. How many new parameters does a classification head add for 20 intents on a model with \(d = 4096\)? Check with Figure 1.
  2. A training set has 950 negatives and 50 positives. Compute the inverse-frequency weights (55.10). By formula (55.12), what probability threshold does the weighted model effectively use?
  3. Gold tags: B-PER I-PER O B-PER I-PER O B-LOC O O. Predicted: B-PER B-PER O B-PER I-PER O B-LOC O O (one tag wrong). Compute token accuracy and entity P, R, F1. Check with Figure 2.
  4. Reference “the cat sat on the mat”, candidate “the cat was on the mat”. Compute ROUGE-1, ROUGE-2 and ROUGE-L F1.
  5. With 90% negatives, what are accuracy, micro F1 and macro F1 of the model that always predicts negative?
  6. With focal loss and \(\gamma = 2\), what is the loss of an example with \(p_y = 0.9\)? With \(p_y = 0.5\)? Compare with plain cross-entropy.
Answerscheck
1. \(20 \times 4096 + 20 = 81{,}940\). 2. \(w_{\text{neg}} = 1000/(2 \cdot 950) = 0.526\), \(w_{\text{pos}} = 1000/(2 \cdot 50) = 10\). Threshold \(w_0/(w_0 + w_1) = 0.526/10.526 = 0.05\), the positive share. 3. Token accuracy \(8/9 = 88.9\%\). Predicted entities: PER(0,0), PER(1,1), PER(3,4), LOC(6); gold: PER(0,1), PER(3,4), LOC(6). \(TP = 2\), so \(P = 2/4 = 0.5\), \(R = 2/3 = 0.667\), \(F_1 = 4/7 = 0.571\). 4. Unigram overlap 5 of 6 on both sides: \(\text{ROUGE-1} = 0.833\). Bigrams: 3 of 5 shared (the cat, on the, the mat): \(\text{ROUGE-2} = 0.6\). LCS = “the cat on the mat” = 5 words: \(\text{ROUGE-L} = 0.833\). 5. Accuracy 0.90; micro F1 0.90 (equal to accuracy); \(\text{macro } F_1 = (0.947 + 0)/2 = 0.474\). 6. \(p_y = 0.9\): CE 0.105, focal \(0.01 \times 0.105 = 0.00105\). \(p_y = 0.5\): CE 0.693, focal \(0.25 \times 0.693 = 0.173\). Easy examples lose 99% of their weight, medium ones 75%.

Key takeaways

  • Classic tasks have three shapes: one label per text, one label per token, or text in and text out. The pretrained body is the same; only the head and the loss change (55.1, 55.4, 55.5).
  • A classification head is one linear layer (\(Kd + K\) parameters). Pool with [CLS] or the mean in an encoder, with the last real token in a decoder. Fine-tune everything with a small learning rate for a few epochs.
  • NER: BIO tags, label the first subword, −100 for the rest, and score with entity-level F1, not token accuracy. Encoders win because each token sees its right context.
  • Seq2seq: teacher-forced cross-entropy on target tokens only, with an encoder–decoder or a decoder-only LLM. ROUGE (55.6, 55.7) measures word overlap, not meaning or facts.
  • With imbalance, accuracy lies; use precision, recall, F1 and macro F1 (55.8, 55.9). Class weights (55.10) and focal loss (55.11) help training, but weights mainly move the threshold to \(p \gt \pi\) (55.12). Tune the threshold on validation.

Key formulas

\[ \mathbf{p} = \softmax\big(W \operatorname{pool}(H) + \mathbf{b}\big), \quad \frac{\partial \mathcal{L}}{\partial \mathbf{z}} = \mathbf{p} - \mathbf{e}_y, \quad F_1 = \frac{2\,TP}{2\,TP + FP + FN} \]
\[ w_c = \frac{N}{K N_c}, \quad \mathcal{L}_{\text{focal}} = -(1-p_y)^{\gamma}\log p_y, \quad F_L = \frac{2 P_L R_L}{P_L + R_L},\ P_L = \frac{\operatorname{LCS}}{|c|},\ R_L = \frac{\operatorname{LCS}}{|r|} \]

References for this session

  1. Devlin et al. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  2. Raschka, Build a Large Language Model (From Scratch), chapter 6: Fine-tuning for classification
  3. Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, chapter 4: Logistic Regression and Text Classification
  4. Liu et al. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach
  5. Raffel et al. (2019). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  6. Lewis et al. (2019). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  7. Lin (2004). ROUGE: A Package for Automatic Evaluation of Summaries
  8. Papineni et al. (2002). BLEU: a Method for Automatic Evaluation of Machine Translation
  9. Lin et al. (2017). Focal Loss for Dense Object Detection
  10. Tjong Kim Sang & De Meulder (2003). Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition
  11. Warner et al. (2024). Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
  12. Rei et al. (2020). COMET: A Neural Framework for MT Evaluation
  13. Touvron et al. (2023). Llama 2: Open Foundation and Fine-Tuned Chat Models