The Hallucination Problem in Production
Hallucination — LLMs generating confident, fluent text that is factually wrong — is the most discussed quality problem in LLM deployment and, in practice, one of the most manageable once you understand its causes and patterns. The framing of hallucination as a random, unpredictable failure mode is misleading. Hallucination follows predictable patterns, clusters around specific conditions, and can be significantly reduced through systematic interventions. This guide covers how to measure hallucination rates in your specific application, what causes the hallucinations you are actually seeing, and the interventions that reliably reduce them.
Types of Hallucination and Their Causes
Fabricated citations and references. LLMs generate plausible-looking but non-existent paper titles, author names, URLs, and publication details. The cause is that citation patterns are strongly represented in training data but specific citation content is not reliably stored — the model generates what a citation looks like without having reliable access to what specific citations actually say. This is one of the most dangerous hallucination types because citations appear verifiable and users often do not check them. The mitigation is structural: require the model to retrieve citations from a verified source rather than recall them.
Entity attribute confusion. Correct entity name, wrong attributes — confusing two companies with similar names, assigning a product feature to the wrong product, attributing a statement to the wrong person. Caused by associative similarity in the model’s representations where related entities bleed into each other. Mitigation: provide explicit, specific context about the entity in question, and instruct the model to use only the provided context for entity-specific claims.
Temporal confabulation. Presenting outdated information as current, or inventing details about events after the training cutoff. Particularly dangerous for time-sensitive applications. Mitigation: RAG with current sources, explicit training cutoff disclosure, and instructions to express uncertainty on time-sensitive claims.
Confident extrapolation. The model knows a general pattern and extrapolates to a specific case it has not seen, producing a plausible but wrong answer. “What is the return policy for product X?” answered with the general return policy pattern when product X has an exception. Mitigation: retrieval of the specific policy document, with instructions to use only retrieved content.
Context-answer mismatch in RAG. The model generates from training knowledge rather than the retrieved context — often occurring when the retrieved context does not contain the answer, and the model fills the gap rather than saying it does not know. Mitigation: explicit grounding instructions, faithfulness evaluation, and “I don’t know” reinforcement.
Measuring Your Hallucination Rate
Before investing in mitigation, establish a baseline measurement of your actual hallucination rate on a representative sample. The measurement approach depends on your task type. For fact-based Q&A with verifiable answers, compare model answers against ground truth answers for a test set and measure exact match and factual accuracy. For document-grounded tasks (RAG, document summarisation), use faithfulness evaluation: ask an LLM judge to evaluate whether each claim in the response is supported by the provided source documents, or use tools like RAGAS that automate faithfulness scoring. For open-domain generation without clear ground truth, use NLI-based factual consistency scoring that assesses whether the response is consistent with known facts. Run your measurement on 100–200 representative examples before changing anything. This baseline is your reference for whether mitigations actually work — it is surprisingly common to deploy an anti-hallucination intervention that feels like it should help but does not meaningfully move the measured rate.
Figure 1 — Hallucination Mitigation by Type
The Grounding Prompt: Your First Line of Defence
For document-grounded tasks, the system prompt is the most impactful single intervention for reducing hallucination. A grounding prompt explicitly instructs the model to use only the provided context, to say clearly when it cannot find information in the context rather than filling gaps from training knowledge, and to express uncertainty appropriately. A strong grounding prompt: “Answer the user’s question using only the information in the provided documents. If the documents do not contain the information needed to answer fully, say explicitly what information is missing rather than inferring or guessing. Do not add information from your general knowledge that is not present in the documents.” This prompt, added to a RAG system with a weak or absent grounding instruction, typically reduces context-answer mismatch hallucination by 30–50%.
The additional instruction “before answering, identify which document section contains the relevant information, then answer based on that section” adds a reasoning step that further reduces hallucination by forcing the model to locate evidence before generating the answer. This chain-of-thought grounding technique is particularly effective for multi-hop questions that require synthesising information from multiple document sections.
Retrieval Quality as a Hallucination Driver
For RAG systems, retrieval quality is the most underappreciated hallucination driver. When retrieval fails to surface the relevant context, the model must choose between saying “I don’t know” and hallucinating — and many models default to the latter, especially when they have relevant training knowledge that is close but not exactly right. Measuring and improving retrieval quality is often a higher-ROI intervention than any generation-side prompt change. Improve retrieval through better chunking, hybrid search, and reranking. Then measure how much hallucination decreases as a result. In most RAG systems, fixing retrieval reduces hallucination more than any prompt engineering change.
Automated Faithfulness Checking in Production
For high-stakes applications where hallucination has serious consequences, automated faithfulness checking as a post-generation validation layer catches hallucinations before they reach users. The implementation: after generating a response, pass the response, the query, and the source documents to a faithfulness evaluator (a separate LLM call or a dedicated faithfulness model). If the evaluator flags the response as unfaithful — containing claims not supported by the source documents — either regenerate with a stronger grounding prompt, return a fallback response indicating the answer could not be determined from available sources, or route to human review. This adds latency and cost for the validation step but prevents the user-facing quality failures that damage trust. For medical, legal, and financial applications where wrong information has serious consequences, the cost of validation is almost always justified by the cost of hallucinated outputs reaching users.
Training Your Model to Say “I Don’t Know”
One of the most effective long-term interventions for hallucination reduction is improving a model’s calibration — its ability to express appropriate uncertainty rather than generating confident wrong answers. For fine-tuned models, include training examples where the correct response is to acknowledge missing information: inputs where the context does not contain the answer, paired with outputs that clearly say “the provided documents do not contain enough information to answer this question” rather than guessing. Models trained on these examples learn that expressing uncertainty is an acceptable and valued response, reducing the pressure to hallucinate when uncertain. For models used via API without fine-tuning, include examples of appropriate uncertainty expressions in your few-shot examples, and explicitly reward uncertainty expressions in your LLM-as-judge quality evaluation.
Hallucination vs. Wrong Answers: An Important Distinction
Hallucination and incorrect answers are not the same thing, and conflating them leads to the wrong mitigations. A hallucination is a confident, fabricated claim — the model invents something that has no basis in its context or training. A wrong answer is a claim that is incorrect but based on something real — a misunderstanding, a outdated piece of training data, or a logical error. Wrong answers are reduced by better information access (RAG) and better reasoning (prompting, model upgrade). Hallucinations specifically are reduced by grounding constraints, retrieval forcing, and calibration training. Diagnosing which type you are actually seeing — by reviewing a sample of failures and categorising them — determines which intervention to apply. Applying hallucination mitigations to wrong-answer problems, or vice versa, produces disappointing results and the false conclusion that the interventions do not work.
Communicating Hallucination Risk to Stakeholders
Every LLM application has some residual hallucination rate that cannot be reduced to zero with current technology. Communicating this honestly to stakeholders and users is both an ethical obligation and a practical necessity. Users who understand that the AI may occasionally produce incorrect information and who know to verify important claims are more resilient to hallucinations than users who have been implicitly or explicitly told that the AI is always accurate. Product designs that include uncertainty signals (confidence indicators, “please verify” prompts for high-stakes outputs, clear sourcing for cited information) manage hallucination risk more effectively than those that present AI outputs as authoritative. And stakeholders who understand the current state of hallucination reduction — where it is now, what interventions are in place, and what the roadmap is — are better partners in managing quality than those who expect zero hallucination from day one. Honesty about current limitations, paired with systematic improvement, is the right posture for any team deploying LLMs in production.
The teams that achieve the lowest hallucination rates in production are not those with the most sophisticated prompts — they are those who have measured their hallucination rate honestly, identified which specific types they are dealing with, and applied targeted mitigations with rigorous before-and-after measurement to confirm what actually works.