Home

GenAI

GenAI

Natural Language Inference (NLI)

Natural Language Inference (NLI), breaking down the concepts, architecture, and processes in greater detail.

Natural Language Inference is a fundamental reasoning task in AI. Think of the model as a “logic detective” tasked with examining two statements: a piece of evidence (the Premise) and a claim (the Hypothesis). The model’s job isn’t to determine if the hypothesis is true in the real world, but only if it’s true, false, or unrelated based on the evidence provided in the premise.

There are three possible verdicts:

  • Entailment: The evidence proves the claim. The hypothesis logically follows from the premise.
  • Contradiction: The evidence refutes the claim. The hypothesis logically contradicts the premise.
  • Neutral: The evidence is irrelevant to the claim. The premise neither proves nor refutes the hypothesis.
Example Data
  • Premise: "A man in a blue shirt is riding a horse through a field."
  • Hypothesis: "A person is outdoors."
  • Target: Entailment (Riding a horse through a field implies being outdoors).
Use Case Scenarios

NLI is a critical component for building more reliable and logical AI systems.

  • Validating AI-Generated Content: If you ask an AI to summarize a legal document, you can use an NLI model to check if the summary’s claims (hypothesis) are logically supported by the original text (premise). This helps prevent factual errors or “hallucinations.”
  • Improving Search Engines: NLI can help a search engine understand if a web page (premise) truly answers a user’s query (hypothesis), going beyond simple keyword matching.

  • Fact-Checking Systems: In a news article, if the premise is "The company's profits soared to a record high in Q4", NLI can flag a user comment saying "The company lost money in Q4" as a Contradiction.

Deep Dive: The Encoder-Only Architecture (The Gold Standard for NLI)

While other architectures can perform NLI, Encoder-Only models like BERT, RoBERTa, and DeBERTa are purpose-built for this kind of task and consistently achieve the best performance.

Why Encoders Dominate NLI

The key is bi-directional context.

Imagine you’re solving a logic puzzle with two pieces of text. You wouldn’t just read the first one and then the second one. You’d read both, then jump back and forth, comparing specific words and phrases. This is what an encoder does.

  • Bi-directional Attention: Every token in the input can “see” and “attend to” every other token, regardless of position. This allows the model to create direct connections, for example, between the word "horse" in the premise and the word "animal" in the hypothesis.

  • Holistic Understanding: An encoder doesn’t process text left-to-right. It builds a holistic representation of the entire input sequence at once. For a comparison task like NLI, this is a massive advantage over decoder models that are optimized for sequential, next-token prediction.

The Training Phase in Detail

Goal: To fine-tune a pre-trained encoder model to specialize in comparing sentence pairs and predicting their logical relationship.

Step 1: Data Preparation & Tokenization

The input data is meticulously structured to be fed into the model.

  1. Start with the Pair:
    • Premise: “The event is scheduled for Tuesday.”
    • Hypothesis: “The event is on the weekend.”
  2. Add Special Tokens: The sentences are concatenated into a single string with special tokens that act as structural signposts.
    • [CLS] (Classification): A token added to the very beginning. Its purpose is to act as a “gathering point” for the entire sequence’s meaning, specifically for classification tasks.
    • [SEP] (Separator): A token used to mark the boundary between the two sentences.
  3. Final Formatted Input: [CLS] The event is scheduled for Tuesday. [SEP] The event is on the weekend. [SEP]
  4. Tokenization: This string is converted into a sequence of numerical IDs by the model’s tokenizer.
Step 2: The Forward Pass Through the Encoder
  1. Input Embeddings: The numerical IDs are converted into vectors. Positional embeddings are added to give the model a sense of word order.
  2. Processing through Encoder Layers: The sequence of vectors is passed through a stack of Transformer Encoder layers. In each layer, the embedding for every token is updated based on its relationship with all other tokens via self-attention.
  3. The [CLS] Token’s Journey: The [CLS] token is treated like any other token initially. However, because it can attend to every word in both the premise and the hypothesis, by the time it exits the final layer, its output vector (e.g., a 768-dimensional vector) has become a rich, aggregated representation of the relationship between the two sentences.
Step 3: The Classification Head & Loss Function
  1. The “Head”: We take the single output vector of the [CLS] token from the final encoder layer. This vector is fed into a “classification head,” which is typically a very simple neural network (often just one linear layer) that is added on top of the pre-trained model.
  2. Output Logits: The head’s job is to project the 768-dimensional [CLS] vector down to a vector with a size equal to the number of labels. For NLI, this is a vector of size 3. These raw output scores are called logits.
    • [CLS] vector (size 768) -> Classification Head -> Logits (size 3)
  3. Loss Calculation: To train the model, we use Cross-Entropy Loss. This function compares the logits (after a softmax is applied) to the correct “one-hot” label. If the correct label is Contradiction (e.g., index 1), the target is [0, 1, 0]. The loss function calculates a penalty based on how far the model’s prediction is from this target. This penalty is then used to update the weights of the classification head and, typically, to fine-tune the weights of the entire encoder model as well.

The Inference Phase in Detail 💡

Goal: To use our fine-tuned NLI model to predict the relationship for a new, unseen pair of sentences.

  1. Format the Input: A new premise-hypothesis pair is formatted in the exact same [CLS]...[SEP]...[SEP] structure.
  2. Perform Forward Pass: The formatted input is fed through the fine-tuned model.
  3. Extract the [CLS] Vector: We take the final output vector corresponding to the [CLS] token.
  4. Predict with the Head: This vector is passed through the trained classification head, producing a logit vector of size 3.
  5. Apply Softmax for Probabilities: The softmax function is applied to the logits to convert them into a human-readable probability distribution.
    • Logits: [-4.6, 6.2, -3.1] -> Softmax -> Probabilities: [0.001, 0.998, 0.001]
  6. Interpret the Output: The final prediction is the label corresponding to the highest probability.
    • Result: { "label": "Contradiction", "score": 0.998 }

#

Alternative Architectures: The Decoder-Only Approach

While encoders are the specialized tool, a powerful decoder-only model like GPT-4 can also perform NLI by framing it as a question-answering or completion task.

  • Prompt Engineering: You would need to create a detailed prompt that instructs the model on what to do. 
      Based on the Premise, determine if the Hypothesis is an entailment, contradiction, or neutral.
    
  Premise: A man in a blue shirt is riding a horse through a field.
  Hypothesis: A person is outdoors.
    
  The logical relationship is:
  • Inference: The model would then generate the next token. A well-instructed model would produce the word “Entailment”.

Pros and Cons for NLI:

  • Pros: Extremely flexible, requires no special architecture, can use few-shot prompting to learn the task on the fly.
  • Cons: The left-to-right nature is fundamentally less suited for direct comparison than an encoder’s bi-directional view. It can be less efficient and potentially less robust than a fine-tuned encoder specifically optimized for the NLI task.