Text classification: sentiment and beyond
Putting a label on a text is the most common job for language models in real products: sentiment, topic, intent, spam, safety. This session builds the three ways to do it (an encoder with a head, a decoder that compares label words, and embeddings with logistic regression), shows how to remove a prompt's bias with calibration, how to measure a classifier with F1, ROC-AUC and a well-chosen threshold, and how to classify documents longer than the model's window.
What you will learn
- Three ways to build a text classifier: encoder + [CLS] head, decoder + label words, frozen embeddings + logistic regression, and what each one trains and costs
- Zero-shot classification by scoring label words instead of generating, including label tokenization and multi-token labels
- Why prompts bias label scores (majority label, recency, common token) and how contextual calibration removes the bias, with real numbers from two small models on IMDb
- Probability calibration with temperature scaling and the expected calibration error
- Accuracy, F1, ROC-AUC and PR curves: what each measures, why ROC-AUC ignores class imbalance and precision does not
- Choosing a decision threshold by metric, by constraint, or by cost
- Long documents: head+tail truncation, overlapping windows, and the right rule to combine window scores
Why this matters
Most LLM work in companies is not chat. It is putting a label on a piece of text: is this review positive, is this message spam, which team should get this ticket, is this post safe to show? This is text classification.
A classifier takes a text \(x\) and returns one label \(y\) from a fixed set (for example {negative, positive}), usually together with a score. Sentiment is the classic first example, but the same tools give you topic labels, intent detection for a chatbot, content moderation, routing of support tickets, and the safety filters that sit in front of almost every LLM product.
Today you have three very different ways to build one. You can fine-tune a small encoder with a classification head. You can ask a decoder LLM and read the probabilities of the label words. Or you can turn texts into embedding vectors and train a tiny logistic regression on top. They differ in cost by orders of magnitude, in how much labeled data they need, and in how they fail. This session builds all three, shows how to make the zero-shot one reliable (calibration), how to measure a classifier honestly (F1, ROC-AUC, thresholds), and what to do when the text is longer than the model’s window.
This session builds on session 33 (the [CLS] token), session 55 (heads, class weights, F1) and session 143 (prompts and few-shot examples).
Three ways to build a classifier
Every text classifier has the same three parts: a model that reads the text, a place where we read out a summary of it, and a small rule that turns that summary into a label. The three families differ in which model and which rule.
How to read Figure 1
Follow one row from left to right. Notice the third column: the encoder gives one vector for the [CLS] position, the decoder gives a probability for every word in its vocabulary at the next position, and the embedding model gives one vector for the whole text. Then compare the trainable numbers in the readout: about 1.5 thousand (plus the encoder if you fine-tune it), zero, and under a thousand.
1. Encoder + classification head
BERT-style encoders put a special token [CLS] at the start of every input. Because attention in an encoder is bidirectional, the final hidden state of that position can see the whole text. We add one linear layer (the head) and a softmax:
- Read the symbols
- \(\mathbf{h}_{\texttt{[CLS]}}(x)\)
- the final hidden state at the [CLS] position, a vector of size \(d\) (768 for BERT-base)
- \(W, \mathbf{b}\)
- the new head: a \(K \times d\) matrix and \(K\) biases, for \(K\) classes
- \(y^\ast\)
- the correct label of the training example; \(\mathcal{L}\) is the usual cross-entropy
For BERT-base and two classes the head has \(768 \times 2 + 2 = 1{,}538\) numbers. Usually we fine-tune the whole encoder (about 110 M weights) together with the head, with a small learning rate. The input is limited to 512 tokens. [Certain] With a decoder model the same idea works if you read the hidden state of the last token instead, because only the last token has seen the whole text (session 55).
2. Decoder + label words
A decoder LLM already knows what “positive” means. Write the text into a prompt that ends where the label should come, for example Review: … Sentiment (positive or negative):, and look at the probability the model gives to each label word as the next token. Nothing is trained. The next section explains this in detail.
3. Embeddings + logistic regression
An embedding model turns a whole text into one vector \(\mathbf{e}(x) \in \R^d\) (session 60). Freeze it, compute vectors for your labeled examples once, and train a logistic regression: a single linear layer with a sigmoid, the oldest classifier in NLP (SLP chapter 4):
- Read the symbols
- \(\sigma(z)\)
- the sigmoid \(1/(1 + e^{-z})\), which turns any number into a probability
- \(\mathbf{w}, b\)
- \(d\) weights and one bias: the only trainable numbers
- \(\lambda\)
- the strength of L2 regularization; it stops the weights from growing huge when \(d\) is close to \(n\)
With \(K\) classes, replace the sigmoid by a softmax over \(K\) scores. Training takes seconds on a CPU, and the same stored vectors can feed ten different classifiers and a search index. SetFit (Tunstall et al., 2022) improves this recipe by first fine-tuning the embedding model contrastively on pairs built from the few labeled examples. [Certain]
Beyond sentiment: many classes and many labels
Nothing above is special to two classes. Topic and intent tasks have tens or hundreds of classes: the head simply has more rows. Some tasks are multi-label: a post can be both “spam” and “offensive”. Then each label gets its own sigmoid and its own threshold, and the loss is a sum of binary cross-entropies, one per label. Do not use a softmax for multi-label tasks: it forces the labels to compete.
Zero-shot: compare label likelihoods
The decoder approach turns a classification task into a fill-in-the-blank. A template wraps the input; a verbalizer maps every class \(y\) to a word or short phrase \(v_y\) (the name comes from PET, Schick and Schütze, 2020). We do not let the model generate; we score every label phrase:
- Read the symbols
- \(v_y\)
- the label phrase for class \(y\), for example the token “ positive” (with its leading space)
- \(v_{y,j}\)
- its \(j\)-th token, if the phrase has more than one token
- \(s(y)\)
- the total log-probability of the phrase; a softmax over the \(s(y)\) of all classes gives \(p(y \mid x)\)
Why score instead of generate? Generation can answer “Positive.”, “The review is positive”, or something off-topic, and you then need a parser. Scoring always returns one of your classes, gives a probability for each, and costs one forward pass when every label is a single token: all label probabilities come from the same next-token distribution. This is how GPT-3 was evaluated on classification tasks. [Likely]
Labels with several tokens
If one label is “positive” (1 token) and another is “mixed feelings” (2 tokens), the sum in (144.3) punishes the longer label: every extra token adds a negative log-probability. Say the model gives \(\log P(\text{“ positive”}) = -1.6\), and \(-2.0\) and \(-0.5\) to the two tokens of “ mixed feelings”. The sum gives \(-1.6\) against \(-2.5\): “positive” wins. The average per token gives \(-1.6\) against \(-1.25\): “mixed feelings” wins. There is no rule that is always right. The simplest fix is to choose single-token labels. Another is to compare against how likely each label is with no input at all, which leads to calibration.
Calibration: removing the model’s bias
The word “calibration” has two meanings in this session. Both matter.
Meaning 1: bias in zero-shot and few-shot prompts
Zhao et al. (2021) showed that few-shot accuracy of GPT-3 changes a lot with the choice and the order of examples, and traced it to a bias of the model toward some answers. They name three: majority label bias (the label seen most often in the prompt), recency bias (the label of the last example), and common token bias (labels that are frequent words in pretraining data). [Certain] The model’s score for a label is “evidence from the text” plus “a prior for that label in this prompt”. We want only the first part.
Contextual calibration estimates the prior with a content-free input: the same prompt with the text replaced by “N/A”, an empty string, or “[MASK]”. Such an input carries no evidence, so whatever the model prefers there is pure bias. Let \(\mathbf{p}_{\text{cf}}\) be the label probabilities for the content-free input (averaged over the three). Then divide it out:
The paper writes this as \(\hat{\mathbf{q}} = \softmax(W\hat{\mathbf{p}} + \mathbf{b})\) with \(W = \operatorname{diag}(\mathbf{p}_{\text{cf}})^{-1}\) and \(\mathbf{b} = 0\), and takes the argmax, which is the same decision. [Certain] For two classes it becomes very simple. Take the log of the odds:
So, for a binary task, contextual calibration subtracts one constant from every log-odds. It moves the decision threshold from 0 to the content-free log-odds. It cannot change the order of the examples, so the ROC-AUC (Section 5) stays exactly the same. With many classes it can change which label wins in more complex ways. A related idea, domain-conditional PMI (Holtzman et al., 2021), scores each answer by \(\log P(v_y \mid \text{prompt}(x)) - \log P(v_y \mid \text{prompt without } x)\): again “how much more likely did the text make this label”. [Certain]
For our zero-shot prompt (below), Qwen2.5-0.5B gives the content-free inputs on average \(p_{\text{cf}}(\text{neg}) = 0.723\) and \(p_{\text{cf}}(\text{pos}) = 0.277\). [Certain: our run] The model leans to “negative” before it reads anything.
A review gets raw \(p(\text{pos}) = 0.40\), \(p(\text{neg}) = 0.60\): the raw answer is “negative”. Formula (144.4): \(0.40 / 0.277 = 1.444\) and \(0.60 / 0.723 = 0.830\), so \(q(\text{pos}) = 1.444 / (1.444 + 0.830) = 0.635\): now “positive”.
Same with (144.5): raw log-odds \(\ln(0.40/0.60) = -0.405\), content-free log-odds \(\ln(0.277/0.723) = -0.957\), calibrated log-odds \(-0.405 + 0.957 = 0.552\), and \(\sigma(0.552) = 0.635\).
Figure 2 simulates this. Each review has some evidence for “positive”; the model adds the same bias \(\beta\) to all of them. Move the bias and watch accuracy fall while the ranking (AUC) stays fixed. Then turn on calibration.
How to read Figure 2
The default \(\beta = -1.0\) is close to the real bias above (\(-0.957\)). With calibration off, many positive reviews (orange) sit left of the line and are called negative; the model predicts “positive” far less than 50% of the time. Turn calibration on: the line jumps to the bias and accuracy recovers. Now move \(\delta\): the content-free input is only an estimate of the bias, and a wrong estimate costs accuracy again. The AUC never changes, because every review moves by the same amount.
What happened on real data
We ran this on 500 IMDb test reviews (250 positive, 250 negative, the first 200 tokens of each) with two small base models, Qwen2.5-0.5B and SmolLM2-135M, the prompt Review: {text}\nSentiment (positive or negative):, and the single tokens “ positive” and “ negative”. Each cell shows accuracy, the share predicted “positive” in brackets (the truth is 50%), and ROC-AUC. [Certain: our run]
| Method | Qwen2.5-0.5B | SmolLM2-135M |
|---|---|---|
| Zero-shot, raw argmax | 85.6% (39.6%), AUC 0.968 | 58.8% (87.6%), AUC 0.793 |
| Zero-shot, contextual calibration (144.4) | 91.2% (51.2%), AUC 0.968 | 65.0% (77.4%), AUC 0.793 |
| 2-shot, both examples positive, raw | 84.8% (60.4%), AUC 0.932 | 50.8% (99.2%), AUC 0.806 |
| 2-shot, both examples positive, calibrated | 85.6% (53.2%), AUC 0.932 | 52.6% (96.6%), AUC 0.806 |
| Mean-pooled last hidden state + logistic regression (500 labeled training reviews) | 83.4%, AUC 0.924 | 81.6%, AUC 0.900 |
Five lessons. (1) The zero-shot bias is real and its direction depends on the model: Qwen called only 39.6% of a balanced set positive, SmolLM2 called 87.6% positive. (2) Calibration helped both (+5.6 and +6.2 points) with the AUC unchanged, exactly as (144.5) says. (3) For SmolLM2 the content-free estimate was not good enough: after calibration it still said “positive” 77.4% of the time. This is the \(\delta\) slider of Figure 2. (4) Two positive demonstrations pushed both models toward “positive” (majority label bias: 60.4% and 99.2%). (5) When the zero-shot ranking is weak (AUC 0.793), 500 labels and a linear classifier on hidden states win easily (81.6%); when it is strong, calibrated zero-shot can beat them. Averaged decoder hidden states are not a trained embedding; a real embedding model (session 60) is the right input for approach 3. These are numbers from two small models on one dataset; do not generalize them to other models.
Meaning 2: do the probabilities mean what they say?
A classifier is calibrated in the probability sense if, among all examples where it says 0.9, about 90% are really positive. Modern networks are often over-confident. Guo et al. (2017) showed that temperature scaling, dividing all logits by one number \(T\) fitted on a validation set, is “surprisingly effective”. [Certain] The usual measure is the expected calibration error:
- Read the symbols
- \(\mathbf{z}(x)\), \(T\)
- the logits, and the temperature (\(T \gt 1\) makes the model less confident); \(T\) is chosen to minimize the validation loss
- \(M\), \(n_b\)
- the confidence range is cut into \(M\) bins (often 10 or 15); \(n_b\) examples fall in bin \(b\)
- \(\operatorname{acc}(b)\), \(\operatorname{conf}(b)\)
- the real accuracy and the average predicted confidence inside bin \(b\)
Example: a bin holds 100 predictions with average confidence 0.90 but only 75 are right: that bin contributes a gap of 0.15. Temperature scaling does not change the argmax, so accuracy stays the same; it only makes the probabilities honest. That matters as soon as you use a threshold or combine scores, which is the next two sections.
Measuring a classifier: accuracy, F1, ROC-AUC
Every evaluation starts from the confusion matrix: true positives (TP), false positives (FP), true negatives (TN), false negatives (FN). Accuracy is \((TP + TN)/n\). Precision \(TP/(TP + FP)\), recall \(TP/(TP + FN)\) and their harmonic mean F1 were built in session 55, together with the “always negative” model that has 90% accuracy and F1 = 0. Here we add the tools that look at all thresholds at once.
The ROC curve and ROC-AUC
A classifier gives a score \(s(x)\); we say “positive” when \(s(x) \gt t\). Each threshold \(t\) gives two rates:
Sweep \(t\) from high to low and plot TPR against FPR: that is the ROC curve. A random scorer gives the diagonal; a perfect one goes straight up to (0, 1). The area under it has a beautiful meaning:
- Read the symbols
- \(x^+, x^-\)
- a random positive and a random negative example
- \(n_+, n_-\)
- the number of positives and negatives
- \(\mathbb{1}[\cdot]\)
- 1 if the condition is true, else 0; ties count one half
AUC is the chance that the model ranks a random positive above a random negative. It does not depend on any threshold, and, because TPR and FPR are each computed inside one class, it does not depend on the share of positives. [Certain] For two Gaussian score distributions with equal variance and means \(d\) standard deviations apart, \(\mathrm{AUC} = \Phi(d/\sqrt{2})\): 0.760 for \(d = 1\), 0.921 for \(d = 2\), 0.983 for \(d = 3\).
Positives score 0.9, 0.8, 0.55, 0.4; negatives score 0.7, 0.35, 0.2, 0.1. There are \(4 \times 4 = 16\) pairs. 0.9 and 0.8 beat all four negatives (8 pairs). 0.55 and 0.4 beat 0.35, 0.2 and 0.1 but lose to 0.7 (6 pairs). AUC = 14/16 = 0.875. No threshold was needed.
Why AUC can look great while precision is terrible
Precision mixes the two classes, so it depends on the share of positives \(\pi\) (the prevalence). From (144.7):
Take a good classifier with TPR = 0.90 and FPR = 0.05. On a balanced test set (\(\pi = 0.5\)) precision is 0.947. In production, where only 1% of messages are spam (\(\pi = 0.01\)), the same classifier has precision 0.154: six out of seven alarms are false, although accuracy is 95%. The ROC curve did not change at all. That is why, for rare positives, we also look at the precision-recall (PR) curve and its area, which do change with \(\pi\). Play with this in Figure 3.
How to read Figure 3
Move the share of positives from 10% down to 0.1%. The left curve and its AUC do not move at all. The right curve sinks, and so do precision and F1 at your threshold. Press “Best F1 threshold”: with rare positives, the best threshold moves far to the right, because each false alarm now costs more precision. Increase \(d\) to see what a better model (not a better threshold) buys you.
Choosing the threshold
The model gives scores; a product needs yes or no. Three honest ways to pick the threshold, all on a validation set, never on the test set:
- Maximize the metric you report (for example F1), as the button in Figure 3 does.
- Fix one rate: “recall at least 95%” for a safety filter, or “at most 1% false positive rate” for a spam filter, and take the threshold that meets it.
- Minimize the expected cost, if the probabilities are calibrated (meaning 2). Say “positive” when the expected cost of saying it is lower: \((1 - p)\, C_{FP} \lt p\, C_{FN}\). That gives
If missing a fraudulent payment costs 10 times more than checking a good one (\(C_{FN} = 10\), \(C_{FP} = 1\)), then \(t^\ast = 1/11 = 0.091\). A payment with \(p = 0.2\) is flagged: flagging costs \(0.8 \times 1 = 0.8\) on average, not flagging costs \(0.2 \times 10 = 2.0\). The 0.5 threshold is only right when the two mistakes cost the same and the probabilities are calibrated.
Long documents: split, classify, combine
A BERT-style encoder reads at most 512 tokens; a contract, a support thread or a book chapter is much longer. Even a long-context LLM pays for every token, and it may use the middle of a long input poorly (session 28). Three options:
- Truncate. Keep only part of the text. Sun et al. (2019) compared on IMDb with BERT-base: keep the first 510 tokens (test error 5.63%), the last 510 (5.44%), or the first 128 plus the last 382 (5.42%, the best of all their methods). [Certain] Reviews put their verdict at the start and the end.
- Split, classify, combine. Cut the text into windows of \(w\) tokens that overlap by \(o\) tokens, classify each window, and combine the window scores with a fixed rule.
- Hierarchical model. Encode each window, then learn how to combine the window vectors (mean, max, or a small attention layer on top). In the same study, mean pooling (5.89%), max pooling (5.71%) and self-attention (5.49%) did not beat head+tail on IMDb. [Certain] On tasks where the evidence is in the middle, they can.
With step \(w - o\), a document of \(L\) tokens needs
windows. For \(L = 5{,}000\), \(w = 510\) (512 minus [CLS] and [SEP]) and \(o = 128\): \(n = \lceil 4490/382 \rceil + 1 = 13\) windows, so the model reads 6,630 tokens, 1.33 times the document. The overlap makes sure a sentence cut at a border is seen whole in at least one window.
How to combine the window probabilities \(p_1, \dots, p_n\) depends on what the label means:
- Read the symbols
- \(\bar p\)
- the mean: right when the label is about the whole text (overall sentiment, topic)
- \(p_{\max}\)
- the max: right when one window is enough (“contains a threat”, “has a non-compete clause”)
- \(p_{\text{OR}}\)
- noisy-OR: the chance that at least one window is positive, if windows were independent; many weak signals add up
- \(p_{\text{logit}}\)
- the mean of log-odds: like a vote where very confident windows count more
12 windows; 11 have \(p_i = 0.05\) and one has \(p_i = 0.95\). Mean: \(\bar p = 0.125\), the post passes. Mean of logits: 0.079, passes. Max: 0.950, blocked. Noisy-OR: \(1 - 0.95^{11} \times 0.05 = 0.972\), blocked. For a moderation label, max or noisy-OR is right; the mean hides the bad paragraph. But with 12 windows all at 0.15, noisy-OR gives \(1 - 0.85^{12} = 0.858\) and blocks a harmless post, while max gives 0.15.
How to read Figure 4
Start with “One bad paragraph” and compare the bars: mean and mean logit stay low, max and noisy-OR go high, and “first chunk” misses chunk 8 completely. Switch to “Many weak signals”: noisy-OR now says positive although no chunk is above 0.15. Click chunks to make your own document. Below, set \(L = 5{,}000\) and \(o = 128\) to check the worked example (13 windows).
Putting it together
- input: model \(P_\theta\), template \(\text{prompt}(\cdot)\), label phrases \(v_1, \dots, v_K\), texts \(x_1, \dots, x_n\)
- check with the tokenizer that each \(v_k\) is one token (with the leading space); if not, decide sum or average for (144.3)
- for \(c\) in {“N/A”, “”, “[MASK]”}: \(\mathbf{p}_c \leftarrow \softmax_k\, s(k)\) for \(\text{prompt}(c)\) // content-free inputs
- \(\mathbf{p}_{\text{cf}} \leftarrow\) mean of the \(\mathbf{p}_c\)
- for each text \(x_i\): one forward pass on \(\text{prompt}(x_i)\); \(\mathbf{p}_i \leftarrow \softmax_k\, s(k)\) // formula (144.3)
- \(\mathbf{q}_i \leftarrow (\mathbf{p}_i / \mathbf{p}_{\text{cf}})\) normalized to sum 1 // formula (144.4)
- return \(\argmax_k q_{i,k}\), and \(\mathbf{q}_i\) as scores for thresholds and AUC
- input: token ids of length \(L\), window \(w\), overlap \(o\), window classifier \(f\), combine rule \(g\), threshold \(t\)
- if \(L \le w\): return \(f(\text{ids}) \gt t\)
- starts \(\leftarrow 0, (w - o), 2(w - o), \dots\) while the window still adds new tokens // \(n\) windows, formula (144.11)
- for each start \(a\): \(p_a \leftarrow f(\text{ids}[a : a + w])\) // batch the windows in one forward pass
- \(p_{\text{doc}} \leftarrow g(p_1, \dots, p_n)\) // mean for whole-text labels, max for “contains X”, formula (144.12)
- return \(p_{\text{doc}} \gt t\), with \(t\) tuned on validation documents (not on windows)
The code we ran for the table above (it needs transformers and downloads the 0.5B model the first time; greedy scoring, so it is deterministic):
import torch from transformers import AutoTokenizer, AutoModelForCausalLM torch.manual_seed(0) name = "Qwen/Qwen2.5-0.5B" tok = AutoTokenizer.from_pretrained(name) model = AutoModelForCausalLM.from_pretrained(name, dtype=torch.float32).eval() labels = [" negative", " positive"] # note the leading space ids = [tok(l, add_special_tokens=False).input_ids for l in labels] assert all(len(i) == 1 for i in ids) # one token each, so one forward pass ids = [i[0] for i in ids] template = "Review: {x}\nSentiment (positive or negative):" @torch.no_grad() def label_probs(text): # formula (144.3), renormalized over the labels x = tok(template.format(x=text), return_tensors="pt").input_ids logp = model(x).logits[0, -1].log_softmax(-1) return logp[ids].softmax(-1) # contextual calibration, formula (144.4) p_cf = torch.stack([label_probs(c) for c in ["N/A", "", "[MASK]"]]).mean(0) def classify(text): p = label_probs(text) q = p / p_cf; q = q / q.sum() return p, q print("content-free:", p_cf) # our run: tensor([0.7225, 0.2775]) for r in ["I did not expect much, but it was a lovely film.", "Two hours of my life I will never get back."]: p, q = classify(r) print(labels[p.argmax()], labels[q.argmax()], q)
And the evaluation side, in plain PyTorch with no downloads: ROC-AUC by ranks (144.8), the best-F1 threshold, the cost threshold (144.10), windows (144.11) and combine rules (144.12).
import math, torch torch.manual_seed(0) # 1) a validation set: 2,000 scores, 10% positives, separation d = 2 n, pi, d = 2000, 0.1, 2.0 y = (torch.rand(n) < pi).long() s = torch.randn(n) + d * y # higher score = more likely positive # 2) ROC-AUC = P(random positive scores above random negative), via ranks def roc_auc(s, y): r = s.argsort().argsort().double() + 1 # ranks 1..n (continuous scores: no ties) n1 = int(y.sum()); n0 = len(y) - n1 return ((r[y == 1].sum() - n1 * (n1 + 1) / 2) / (n1 * n0)).item() # 3) precision, recall, F1 at a threshold, and the best-F1 threshold def prf(s, y, t): p = (s > t).long() tp = int(((p == 1) & (y == 1)).sum()); fp = int(((p == 1) & (y == 0)).sum()); fn = int(((p == 0) & (y == 1)).sum()) P = tp / max(1, tp + fp); R = tp / max(1, tp + fn) return P, R, 2 * P * R / max(1e-12, P + R) auc = roc_auc(s, y) print(f"positives {int(y.sum())}, ROC-AUC {auc:.3f} (theory {0.5*(1+math.erf(d/2)):.3f})") print("at t=0: P %.3f R %.3f F1 %.3f" % prf(s, y, 0.0)) ts = torch.linspace(-2, 5, 701) best = max(ts.tolist(), key=lambda t: prf(s, y, t)[2]) print("best t=%.2f: P %.3f R %.3f F1 %.3f" % ((best,) + prf(s, y, best))) # 4) cost-based threshold for calibrated probabilities: a miss costs 10x a false alarm c_fp, c_fn = 1.0, 10.0 print("cost threshold", round(c_fp / (c_fp + c_fn), 3)) # 5) long documents: overlapping windows, classify each, combine def windows(ids, w=510, overlap=128): step = w - overlap starts = list(range(0, max(1, len(ids) - overlap), step)) return [ids[i:i + w] for i in starts] doc = list(range(5000)) # stand-in for 5,000 token ids ch = windows(doc) print("chunks", len(ch), "last chunk covers", ch[-1][0], "to", ch[-1][-1]) p = torch.tensor([0.05] * 7 + [0.95] + [0.05] * 4) # per-chunk P(positive), 12 chunks logit = torch.log(p / (1 - p)) print("mean %.3f mean-logit %.3f max %.3f noisy-OR %.3f" % ( p.mean(), torch.sigmoid(logit.mean()), p.max(), 1 - torch.prod(1 - p))) # Output (PyTorch 2.x, CPU): # positives 202, ROC-AUC 0.931 (theory 0.921) # at t=0: P 0.188 R 0.995 F1 0.317 # best t=1.46: P 0.571 R 0.698 F1 0.628 # cost threshold 0.091 # chunks 13 last chunk covers 4584 to 4999 # mean 0.125 mean-logit 0.079 max 0.950 noisy-OR 0.972
Note the second line of output: with 10% positives, the threshold 0 finds almost every positive (recall 0.995) but precision is only 0.188. Tuning the threshold doubles F1, with the same model.
Where this is used
| Application | Typical approach | What to watch |
|---|---|---|
| Product-review and social-media sentiment | Fine-tuned encoder; zero-shot LLM to bootstrap labels | Sarcasm, mixed reviews; head+tail truncation for long reviews |
| Content moderation and LLM safety filters | Multi-label classifier, one sigmoid and threshold per policy | Rare positives: report PR curves and recall at a fixed FPR; max over chunks |
| Intent detection and ticket routing | Embeddings + logistic regression, or a small encoder with 10–500 classes | New intents appear: an “other” class, or a confidence threshold to send to a human |
| Routing between LLMs (cheap vs strong model) | Small classifier that predicts if the cheap model will be good enough | The cost threshold (144.10): a wrong “cheap” costs quality, a wrong “strong” costs money |
| Contract and document review | Split, classify, combine with max (“does any clause say X?”) | Overlap at window borders; threshold set on documents, not windows |
| Video classification and action recognition | Same split–classify–combine over clips: score each clip, average for the video label | Mean for “what is this video about”, max for “does it contain X” |
| Key-moment detection and video moderation | Per-clip classifier on frames or on the speech transcript; max or a run of positive clips | One-second violations are rare: thresholds from PR curves, not ROC-AUC |
| Zero-shot video and image classification (CLIP-style) | Compare the input with label prompts such as “a video of {label}”, the vision version of label likelihoods | Prompt wording and label frequency bias, the same as in (144.4) |
Common mistakes
- Label tokens that are not what you think. “positive” vs “ positive”, or a label that splits into three tokens. Print the token ids of every label before running anything.
- Generating and parsing instead of scoring. Free generation gives answers outside your label set and no probabilities. Score the label words (144.3), or use constrained decoding.
- Trusting raw zero-shot probabilities. The prior of the prompt can shift a balanced dataset to 40/60 predictions. Check the share of each predicted label, and calibrate (144.4).
- Reporting accuracy or ROC-AUC alone with rare positives. AUC ignores prevalence (144.9). Report precision, recall and F1 at the chosen threshold, and the PR curve.
- Choosing the threshold on the test set. That is training on the test set. Tune on validation, then report test once.
- Using a 0.5 threshold on uncalibrated or class-weighted outputs. Weighted training already moves the threshold (session 55); fit temperature (144.6) or tune the threshold directly.
- Silent truncation of long documents. Tokenizers cut at 512 tokens without an error. Log how many inputs were cut, and pick head+tail or windows on purpose.
- Mean-pooling a decoder’s hidden states and calling it an embedding. It works somewhat (81.6–83.4% in our run), but a model trained to produce embeddings is the right tool for approach 3.
Interview questions
Name three ways to build a sentiment classifier with modern models. When would you pick each?basic
How do you do zero-shot classification with a decoder LLM without generating any text?basic
Two labels have different numbers of tokens. What goes wrong and how do you fix it?core
What is contextual calibration, and why does it work?core
Why can contextual calibration not change ROC-AUC for a binary task?advanced
Accuracy, F1, ROC-AUC: which do you report for a fraud model with 0.5% positives?core
Your classifier has ROC-AUC 0.95 offline, but in production most alarms are false. Explain.core
How do you choose the decision threshold?core
How would you classify a 20,000-token document with a 512-token encoder?core
Your fine-tuned classifier says 0.99 on wrong examples. How do you fix its probabilities?advanced
A product team wants to use GPT-class LLM calls for a 50M-messages-per-day spam filter. What do you suggest?advanced
Exercises
- The content-free probabilities are \(p_{\text{cf}}(\text{pos}) = 0.6\), \(p_{\text{cf}}(\text{neg}) = 0.4\). A review gets raw \(p(\text{pos}) = 0.55\). What is the calibrated \(q(\text{pos})\), and does the label change?
- Positives score 0.8, 0.6, 0.3; negatives score 0.7, 0.2. Compute ROC-AUC by counting pairs.
- A classifier has TPR 0.8 and FPR 0.02. What is its precision at \(\pi = 0.2\) and at \(\pi = 0.002\)?
- A false negative costs 4 and a false positive costs 1. What threshold on calibrated probabilities minimizes expected cost?
- How many windows of 510 tokens with overlap 64 does a 3,000-token document need?
- Eight windows each have \(p_i = 0.2\). Compute the mean, max and noisy-OR document scores. Which ones say “positive” at threshold 0.5?
Answerscheck
Key takeaways
- Three families: encoder + head on [CLS] (144.1), decoder + label words (144.3), frozen embeddings + logistic regression (144.2). They trade labeled data, cost and robustness.
- Zero-shot with an LLM means scoring label words, not generating. Check label tokenization; multi-token labels need a length rule.
- Prompts add a prior to the label scores. Contextual calibration divides it out with a content-free input (144.4); for two classes it is a pure threshold shift (144.5), so it fixes accuracy but not AUC. In our run: 85.6% → 91.2% on IMDb for Qwen2.5-0.5B.
- ROC-AUC is the chance a positive outranks a negative (144.8) and ignores prevalence; precision does not (144.9). Choose thresholds on validation data, by metric, by constraint, or by cost (144.10). Make probabilities honest with temperature scaling (144.6).
- Long documents: truncate with head+tail, or split into overlapping windows (144.11) and combine with the rule that matches the label: mean for whole-text labels, max or noisy-OR for “contains X” (144.12).
Key formulas
References for this session
- Zhao et al. (2021). Calibrate Before Use: Improving Few-Shot Performance of Language Models
- Devlin et al. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, chapter 4: Logistic Regression and Text Classification
- Zhao et al. (2021). Calibrate Before Use: Improving Few-Shot Performance of Language Models
- Devlin et al. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, chapter 4: Logistic Regression and Text Classification
- Holtzman et al. (2021). Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
- Guo et al. (2017). On Calibration of Modern Neural Networks
- Sun et al. (2019). How to Fine-Tune BERT for Text Classification?
- Schick & Schütze (2020). Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
- Tunstall et al. (2022). Efficient Few-Shot Learning Without Prompts
- Brown et al. (2020). Language Models are Few-Shot Learners
- Liu et al. (2023). Lost in the Middle: How Language Models Use Long Contexts