Office Hours — What are practical techniques for reducing hallucinations and improving reliability in production LLM systems?

What are practical techniques for reducing hallucinations and improving reliability in production LLM systems?

Hallucinations aren’t a single problem you solve once. They’re a class of failure modes that compound differently depending on your architecture, model, and use case. Here’s what actually works in production.

The hallucination taxonomy matters

Not all wrong outputs are equal. A model inventing a citation is different from confidently stating “Python lists are ordered” when they’re insertion-ordered only in 3.7+. The first is a retrieval failure; the second is a training data artifact. The third is a reasoning error. Your fix changes based on which you have.

Start by categorizing failures in your production logs. Build a small labeling dashboard. Spend a week tagging outputs as: missing information, invented information, misapplied knowledge, reasoning error, or semantic drift (technically correct but not what was asked). You’ll spot patterns. Most teams find one or two dominant failure modes. Fix those first.

Structured outputs + validation layers

Structured outputs alone give false confidence. A model can return valid JSON that’s factually wrong. The move is adding a validation layer that sits between the model and your user.

Here’s the pattern:

from pydantic import BaseModel, Field
from typing import Optional

class Citation(BaseModel):
    quote: str = Field(..., description="Exact quote from source")
    page: int
    source_id: str

class Answer(BaseModel):
    answer: str
    citations: list[Citation]
    confidence: float = Field(ge=0.0, le=1.0)

# After getting the response:
answer = Answer.model_validate(response_json)

# Validation layer
for citation in answer.citations:
    if not verify_quote_in_source(citation.quote, citation.source_id):
        # Log, flag for review, or reject
        raise ValueError(f"Citation {citation.source_id} does not contain: {citation.quote}")

The validator doesn’t need to be sophisticated. It can be a simple substring search, a BM25 lookup, or a semantic similarity check. The point is: don’t let invented citations reach users. Reject or escalate instead.

Constrain the output space

Models hallucinate less when the output space is small. Instead of “tell me about this company,” use:

class CompanyLookup(BaseModel):
    company_name: str
    status: Literal["found", "not_found", "ambiguous"]
    employee_count: Optional[int] = None  # None if not_found
    headquarters_city: Optional[str] = None

Now the model must choose: does the company exist in your knowledge base or not? If it doesn’t know, it picks “not_found” rather than inventing. Pair this with a retrieval check. If the model says “found” but your database has no match, log it as a failure.

Temperature and sampling matter less than you think

Setting temperature to 0 doesn’t prevent hallucinations. It prevents variance. A confident hallucination is still a hallucination. What helps: increasing the log probability threshold. Claude and GPT-5.5 both expose top_logprob in the response. If the model’s confidence on a fact is below a threshold (like -2 log probability), flag it for retrieval or human review.

# Example: only trust outputs with high confidence
response = client.messages.create(
    model="claude-opus-4.8",
    messages=[...],
    top_logprobs=2,  # Include top-2 alternatives per token
)

# Check confidence
for block in response.content:
    if hasattr(block, 'logprobs'):
        for token_info in block.logprobs:
            if token_info['logprob'] < -2.0:  # Low confidence token
                # Flag for review
                pass

RAG + validation is table stakes for fact-heavy tasks

If you’re building a system that answers questions about documents, PDFs, or a knowledge base, don’t use naked LLM generation. Use retrieval-augmented generation with a structured validation step.

The Daily Signal from July 4 covered this: return typed schemas instead of free text, validate evidence spans before returning, and accept “not found” as a valid answer. This means your prompt needs to explicitly teach the model when not to answer.

# Good prompt pattern for RAG
system_prompt = """
You are answering questions based on retrieved documents.

Rules:
1. Only use information from the documents provided.
2. If the documents don't contain the answer, respond with:
   {"status": "not_found", "reason": "specific reason"}
3. Always cite the exact span from the document.
4. If the documents are unclear or contradictory, say so.
"""

Defense in depth: use multiple validation layers

Don’t rely on a single check. Stack them:

Structural validation (is the JSON valid?).
Semantic validation (does the citation actually appear in the source?).
Consistency check (does this answer contradict something we already know?).
Plausibility check (is the confidence score reasonable given the data quality?).

If any layer fails, escalate or reject. This is tedious but it works. From the Daily Signal (July 6), enterprises doing RAG at scale validate answers before users see them using span quotes and feedback loops.

Fine-tuning only if you have a specific, repeatable failure mode

Most teams try fine-tuning to reduce hallucinations. It doesn’t work as a general fix. It works when you have hundreds of examples of a specific mistake your model makes repeatably. “The model always hallucinates the same incorrect detail for this type of query” is fine-tuning material. “The model sometimes hallucinates” is not.

Use lower-capability models with constraints over capable models without them

GPT-5.5 will hallucinate more creatively than GPT-4.1 Nano. But Nano with a validation layer and constrained output schema beats GPT-5.5 unsupervised. You get faster inference, lower cost, and better reliability. The tradeoff is you can’t ask Nano to reason over ambiguous input. You can ask it to classify, extract, or fill a constrained template.

Monitor what you can’t see

Set up logging for:

Confidence scores on critical facts.
Cases where the model says “I don’t know” (you want this to be common).
Cases where retrieved context contradicts the model’s answer.
User corrections or negative feedback.

Build a dashboard. You’ll spot failure modes before they become incidents.

The hard part: knowing when you’re hallucinating

The worst hallucinations are the confident ones. The model asserts something false with high probability. There’s no bulletproof fix. But there’s a pragmatic one: make hallucinations expensive. Require a citation. Require the model to explain its reasoning. Require a confidence score. Then enforce consequences when those fail validation. The model learns (not through fine-tuning, but through your system design) that making things up costs more than admitting uncertainty.

Bottom line: Structure your outputs, validate before serving, use retrieval for facts, and escalate uncertainty instead of guessing. No single technique wins, but layered validation plus constrained schemas catches 80% of hallucinations in production.

Question via Hacker News