Office Hours — How do you prevent hallucinated tool calls from wasting tokens and breaking production LLM workflows?

How do you prevent hallucinated tool calls from wasting tokens and breaking production LLM workflows?

This is a real problem. Your agent calls a tool that doesn’t exist, or calls an existing tool with invalid parameters, and you’ve just burned tokens on a failed operation. Scale that across thousands of requests and you’re hemorrhaging cost and reliability. The issue gets worse when hallucinated calls cascade: a broken tool call triggers error handling, which consumes more tokens, which can trigger more hallucinated calls downstream.

The Core Problem

LLMs don’t actually “know” which tools exist or what parameters they accept. They’re pattern-matching on training data that included tool definitions. When you define a tool, you’re encoding a contract, but the model is just predicting the next token. If your tool schema is ambiguous, if error messages are vague, or if the model is uncertain, it will confidently generate invalid calls.

A few months back, someone posted a case study about a production RAG agent that was calling a search function with parameters that didn’t match the schema. The function rejected it silently, the agent retried with slightly different parameters, and over 48 hours they’d burned through thousands of dollars on token waste from a hallucinated parameter name that was off by one character.

Strict Schema Validation and Rejection

Start with schema enforcement at the boundary. Don’t let invalid calls through to your actual tool layer. Use a schema validator (JSON Schema is fine) before executing anything.

Here’s the pattern:

from pydantic import BaseModel, ValidationError
from typing import Literal

class SearchToolInput(BaseModel):
    query: str
    max_results: int = 10
    
class EmailToolInput(BaseModel):
    recipient: str
    subject: str
    body: str

TOOL_DEFINITIONS = {
    "search": {
        "schema": SearchToolInput,
        "description": "Search documents"
    },
    "send_email": {
        "schema": EmailToolInput,
        "description": "Send email"
    }
}

def execute_tool_call(tool_name: str, raw_params: dict):
    """
    Validate before executing. Reject invalid calls instead of trying to fix them.
    """
    if tool_name not in TOOL_DEFINITIONS:
        return {
            "error": f"Unknown tool: {tool_name}. Available: {list(TOOL_DEFINITIONS.keys())}"
        }
    
    try:
        schema = TOOL_DEFINITIONS[tool_name]["schema"]
        validated_params = schema(**raw_params)
        # Execute the actual tool here
        return execute_real_tool(tool_name, validated_params)
    except ValidationError as e:
        return {
            "error": f"Invalid parameters for {tool_name}: {e.json()}"
        }

This catches parameter mismatches immediately. But it doesn’t prevent hallucinated calls. The error goes back to the agent, which might retry or hallucinate a different invalid call. You’ve still wasted tokens.

Grounding: Make the Tool List Explicit and Specific

Claude Opus 4.8 and GPT-5.5 both support structured tool definitions now. The key insight is specificity. Vague tool descriptions lead to vague calls.

Instead of:

"search": "Search for information"

Write:

"search": "Search the internal document repository. Only accepts 'query' (required, string, max 100 chars) and 'max_results' (optional, integer, 1-20). Returns a list of documents with title, excerpt, relevance_score."

That specificity reduces hallucination significantly. The model now understands boundaries.

Also: list tools once, upfront. Don’t dynamically regenerate the tool list on every call. The more times you change the tool definitions, the more the model drifts from what it learned about the schema.

Control the Error Feedback Loop

When a tool call fails, your error message matters more than you think. If you return a vague error like {"error": "Invalid input"}, the agent will hallucinate a fix. If you return {"error": "max_results must be an integer between 1 and 20, got: 'unlimited'"}, the agent can correct itself.

But here’s the tradeoff: detailed error messages give the model more information to retry on, which can lead to longer chains and more token waste if the agent keeps failing. You need to break the retry loop.

Implement a retry limit per tool call, and per conversation. After 3 failed attempts on the same tool, stop trying. Return a structured “tool unavailable” response instead of letting the agent keep hallucinating.

def call_tool_with_retry_limit(tool_name: str, params: dict, max_retries: int = 3):
    for attempt in range(max_retries):
        result = execute_tool_call(tool_name, params)
        if "error" not in result:
            return result
        if attempt == max_retries - 1:
            return {
                "error": f"Tool {tool_name} failed after {max_retries} attempts. Stopping.",
                "final_error": result["error"]
            }
    return result

Separate Tool Definition Versions

When you update a tool schema (add a parameter, change constraints), version it. Don’t silently change the definition. Create a new tool name or explicitly tell the agent that the schema changed.

This prevents the agent from operating on stale assumptions about what tools exist or how they work.

Use Smaller Models with Tool Calling Where Possible

Counterintuitively, Claude Sonnet 4.6 hallucinates fewer tool calls than Opus 4.8 in some domains because it’s more conservative about calling tools it’s uncertain about. For high-volume, low-reasoning tasks (dispatch, simple lookups), Sonnet can be cheaper and more reliable. Only use Opus when you genuinely need the reasoning.

Test both models on your tool-calling workflows. Token waste from hallucinations can overwhelm the savings from a cheaper model.

Monitor Tool Call Failure Rates

Set up alerting on the percentage of tool calls that fail validation or execution. If that rate exceeds 5%, something is wrong: either your tool definitions are drifting, or your error messages are confusing the agent. Fix it before it scales.

Log which tools hallucinate most often. If one tool consistently gets invalid calls, either simplify its schema or remove it and replace it with two simpler tools.

Bottom line: Validate all tool calls against strict schemas before executing them, provide explicit and specific tool definitions upfront, implement retry limits to break feedback loops, and monitor failure rates to catch drift early. The goal isn’t to prevent all hallucinations, it’s to contain their cost.

Question via Hacker News