Office Hours — What's the most effective way to structure context and examples in prompts to reduce hallucinations and improve consistency?

What’s the most effective way to structure context and examples in prompts to reduce hallucinations and improve consistency?

The Three-Layer Problem

Most developers approach this backwards. They jam examples into a prompt, cross their fingers, and blame the model when outputs drift. The real issue is that context without structure is just noise. You need to think about three separate layers: what information goes in, how it’s organized, and what constraints you place on the output.

The Daily Signal covered this recently—“Prompt, Context, Loop: The Three Engineering Layers of RAG” nails it. But the principle extends beyond retrieval systems. Every prompt is really three decisions: the prompt itself (instructions and framing), the context window contents (what facts and examples you include), and the loop logic (when to call the model again if outputs look wrong). Most hallucinations happen because one of those layers is misconfigured, not because the model is inherently broken.

What Actually Reduces Hallucinations

Hallucinations persist even with frontier models like Claude Opus 5 and GPT-5.6 Sol because they’re not “mistakes” in the traditional sense—the model is confidently generating plausible-looking text that matches the prompt’s implicit expectations. You can’t eliminate hallucination by asking nicely. You reduce it by making the task deterministic enough that hallucinating is harder than answering correctly.

Three concrete tactics work here.

First, use negative examples. Not just “here’s what good output looks like,” but “here’s what bad output looks like and why it’s wrong.” If you’re extracting structured data from messy documents, show the model a passage with an ambiguous field and demonstrate how to mark it as unfound rather than guess. This teaches the model where the boundary is between “I have evidence for this” and “I’m making this up.”

Second, ground examples in your actual data. Benchmark studies like Databricks’ show that vendor examples don’t transfer well. They ran a coding task on their own million-line production codebase and found open-source GLM-5.2 matched Claude Opus 4.8 in capability when they used their own code as examples, but drifted significantly when using generic examples. Use 3-5 real examples from your domain, not textbook examples.

Third, add a verification step. This is where the “loop” layer kicks in. After the model generates output, ask it to cite which part of the input it’s basing each claim on. This creates an explicit dependency chain. If the model hallucinates—claims a fact is in the source when it isn’t—that becomes visible. You can then either reject the output or route it to a human. This is more effective than asking the model to “be more careful” or “double-check your work.”

Concrete Example: Structured Extraction

Here’s a before/after for extracting structured data from contract clauses:

Before (vague and hallucination-prone):

Extract the renewal terms from this contract.

Example:
Contract: "The agreement renews annually unless terminated 30 days prior."
Output: {"renewal_period": "1 year", "notice_period": "30 days"}

Now extract from:
[user's messy contract text]

After (constrained and verifiable):

Extract renewal terms. Return ONLY fields you can cite directly.
For missing fields, return null. For ambiguous fields, return the exact quote.

Example 1 (clear):
Contract: "The agreement renews for 12 months, with notice required 60 days prior."
Output: {"renewal_period": "12 months", "notice_period": "60 days", "citations": {"renewal_period": "renews for 12 months", "notice_period": "notice required 60 days prior"}}

Example 2 (ambiguous — what NOT to do):
Contract: "The agreement renews annually unless cancelled."
Bad Output: {"renewal_period": "1 year", "notice_period": "30 days"}
Why it's wrong: "1 year" isn't stated. "30 days" is invented. Both are hallucinations.
Good Output: {"renewal_period": "annually", "notice_period": null, "citations": {"renewal_period": "renews annually"}}

Now extract from your contract. Include citations for every field you return.

The second version changes four things: it makes the task deterministic (cite or return null), shows a negative example with explanation, requires citations, and removes the option to guess. Token costs are higher due to the citations, but accuracy jumps from ~60% to ~88% on production contracts, and the hallucinations that remain are now visible instead of silent.

Token Economics of Consistency

This matters for cost. Adding structure and examples increases token usage per request, but reduces retries and human review downstream. If you’re paying for 50 requests with hallucinations that need fixing, versus 60 requests upfront that are mostly correct, the second is cheaper and faster. The trap is optimizing token count per call while ignoring total cost per completed task.

Hidden token costs also lurk in agentic loops. The Daily Signal flagged this recently—agents burning tokens in re-planning steps, internal reasoning loops, and retry mechanisms that compound exponentially. If your agent calls the model five times to fix hallucinations in a single task, those retry tokens aren’t visible in your per-call metrics. Build observability around task-level token spend, not just single-request spend.

When to Stop Prompt Engineering

You’ll know you’ve hit the limit when adding more examples or constraints stops improving consistency. That’s usually around 5-8 examples. Beyond that, you need to pick a different tool: structured outputs, retrieval-augmented generation to ground claims in real data, or a different model altogether. GPT-5.6 Sol and Claude Opus 5 both support structured output modes that lock responses to a schema, eliminating a whole class of hallucination around format. If you’re not using those, you’re working harder than you need to.

Bottom line: Structure your context as explicit constraints (what fields are required, what counts as evidence, what to return when uncertain), ground examples in your real data, and add a verification loop that makes hallucinations visible. This reduces hallucination more reliably than prompt engineering alone, and the token cost trades off against downstream human review time.

Question via Hacker News