Hallucinations: Why and How to Reduce Them

Why models confidently make things up, the different kinds of hallucination, and a toolbox of fixes.

• Beginner
• 20 min read
• 3 objectives

Before this lessonLesson 3: Prompts, Temperature and Sampling

Ask a model for a legal case that supports your argument and it may hand you a beautifully formatted citation for a case that does not exist. This is not a bug in the usual sense. It is the direct consequence of the fact you learned in lesson one: an LLM produces the most plausible continuation, not the most true one. This lesson explains why hallucinations happen and, more usefully, the practical toolbox for reducing them.

What is a hallucination?

A hallucination is output that is fluent and confident but false, made up, or not supported by the source it should rely on. It is worth splitting into two kinds, because they have different fixes:

• Factual hallucination: the answer contradicts the real world. "The Eiffel Tower was completed in 1912." (It was 1889.)
• Faithfulness (grounding) hallucination: you gave the model a document and it says something the document does not support. This is the kind that hurts RAG chatbots, and the kind we can measure.

Why do models hallucinate?

• They are trained to continue, not to verify. The training goal is "predict plausible text", and plausible and true are different.
• Gaps and rare facts. Common facts are learned well. Obscure ones (a small company's refund policy, a local law, last week's news) were seen rarely or never, so the model fills the gap with something that sounds right.
• They are rewarded for answering. Models are tuned to be helpful, and "I do not know" can feel unhelpful, so guessing is an easy habit.
• Knowledge cut-off. The model knows nothing after its training date and will not always admit it.
• Long or messy context. Give a model a contradictory or huge prompt and it may blend or misremember.
• Sampling randomness. High temperature makes unlikely (wrong) tokens more probable.

The hallucination toolbox

There is no single cure. You stack defences, roughly from cheapest to most involved:

• 1. Ground the answer in documents (RAG). Retrieve the relevant text and tell the model to answer only from it. The single most effective fix for company- and domain-specific questions.
• 2. Permission to say "I do not know". "If the context does not contain the answer, reply exactly: I do not have that information."
• 3. Require citations. "Quote the sentence you used." Forced quoting makes fabrication harder and lets you verify.
• 4. Lower the temperature for factual tasks.
• 5. Use tools for facts the model is bad at: a calculator for maths, a database query for numbers, a search API for fresh news.
• 6. Verify with a second pass. Ask another model call to check each claim against the source ("LLM as judge"), or check with code (does that URL exist? does that function compile?).
• 7. Constrain the output with schemas and enums so it cannot invent categories.
• 8. Keep humans in the loop for high-stakes answers (medical, legal, financial).

Measuring faithfulness with a tiny checker

You cannot fix what you cannot measure. Real systems use an LLM-as-judge or NLI models, but the idea is simple: split the answer into claims, and check whether each claim is supported by the retrieved context. Here is a deliberately simple version using word overlap, enough to see the shape of the problem:

import re

context = ("Acme laptops can be returned within 30 days of delivery for a full refund. "
           "Opened software is not refundable. Shipping is free on orders over 50 dollars.")

def sentences(text):
    return [s.strip() for s in re.split(r"(?<=[.!?])\s+", text) if s.strip()]

def words(text):
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def support(claim, context, threshold=0.6):
    """Fraction of the claim's content words that appear in the context."""
    stop = {"the", "a", "an", "is", "are", "can", "be", "of", "for", "on", "in", "to", "and", "you"}
    claim_words = words(claim) - stop
    if not claim_words:
        return 1.0, True
    overlap = len(claim_words & words(context)) / len(claim_words)
    return overlap, overlap >= threshold

answer = ("Laptops can be returned within 30 days for a full refund. "
          "You also get a free case with every laptop. "
          "Shipping is free on orders over 50 dollars.")

for claim in sentences(answer):
    score, ok = support(claim, context)
    print(f"{'SUPPORTED  ' if ok else 'UNSUPPORTED'} {score:4.0%}  {claim}")

SUPPORTED   100%  Laptops can be returned within 30 days for a full refund.
UNSUPPORTED  14%  You also get a free case with every laptop.
SUPPORTED   100%  Shipping is free on orders over 50 dollars.

It caught the invented "free case". A word-overlap checker is crude (it would miss a claim that reuses the right words with the wrong meaning), which is exactly why production systems use a second LLM or a trained model as the judge. But the workflow is the same: claim by claim, supported or not.

A prompt that reduces hallucination

You are a support assistant. Answer using ONLY the context below.

Rules:
1. If the answer is not in the context, reply exactly: "I don't have that information."
2. After your answer, quote the sentence from the context that supports it.
3. Do not use outside knowledge, and do not guess.

<context>
{retrieved_chunks}
</context>

Question: {question}

Hallucination in the wild: a quick field guide

• Fake citations and URLs: plausible-looking papers or links that do not exist. Always click through.
• Invented API functions: code that calls a library method that was never there. Compile and test everything.
• Confident wrong numbers: statistics with no source. Demand sources or compute with tools.
• Sycophancy: agreeing with a wrong premise in your question. Ask neutral questions and ask for counter-arguments.
• Stale facts: answers that were true at the training cut-off.

Evaluating at scale

Build a small evaluation set: 50–200 real questions with known-good answers (and some with no answer in your documents). After every change to prompts, retrieval or model, rerun the set and track: correctness, faithfulness (is every claim supported), and refusal accuracy (does it decline the unanswerable ones?). This habit is the difference between a demo and a product, and we return to it in the RAG evaluation and production lessons.

# Write your solution here