Office Hours — What's the best way to structure agent prompts and tool definitions to maximize skill utilization and accuracy?
A daily developer question about AI/LLMs, answered with a direct, opinionated take.
What’s the best way to structure agent prompts and tool definitions to maximize skill utilization and accuracy?
You need to separate concerns: what the model thinks it’s trying to do versus what it actually can do. Most agent failures don’t come from the model reasoning poorly. They come from ambiguous tool definitions, vague success criteria, and prompts that assume the model will magically discover the right tool ordering.
Define Tools Like APIs, Not Like Wishes
Tool definitions should read like strict API documentation, not aspirational descriptions. Compare these:
# Bad
{
"name": "search",
"description": "Search for information"
}
# Good
{
"name": "search_docs",
"description": "Full-text search across internal documentation. Returns up to 10 results ranked by relevance. Searches titles, body text, and metadata.",
"parameters": {
"query": {
"type": "string",
"description": "Search terms. Use AND/OR operators. Partial matching enabled.",
"maxLength": 200
},
"filters": {
"type": "object",
"description": "Optional: { 'doc_type': 'API|Guide|Tutorial', 'updated_after': 'YYYY-MM-DD' }",
"optional": true
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 10,
"default": 5
}
},
"returns": {
"type": "array",
"items": {
"title": "string",
"url": "string",
"excerpt": "string (first 300 chars)",
"relevance_score": "float 0-1"
}
},
"error_cases": [
"No results found (empty array)",
"Query too long (error)",
"Invalid filter format (error)"
]
}
The second one tells the model exactly what it’s getting back, what constraints exist, and what failure modes look like. That matters for tool selection.
Prompt Structure: Separate Capability Declaration from Task Instructions
Frontier models like GPT-6 Astra or Claude Opus 5.5 work better when you explicitly compartmentalize what the agent can do, what it’s trying to do, and how to handle failure. This isn’t overthinking it. Production data shows quality drift accounts for 38% of agent failures, and drift starts with ambiguous prompts.
SYSTEM PROMPT STRUCTURE:
1. CAPABILITY DECLARATION
"You have access to these tools: [list names only, one per line]"
"You cannot access: [databases, files, real-time data, etc.]"
2. SUCCESS CRITERIA
"A task is complete when: [measurable condition]"
"You have failed if: [specific failure modes]"
3. TOOL USAGE RULES
"Before calling a tool, explain which tool and why."
"If a tool returns empty results, do NOT invent data."
"If you're unsure which tool to use, ask for clarification."
4. TASK INSTRUCTION
[Actual task goes here, brief and specific]
5. OUTPUT FORMAT
"Return: [structured format: JSON, markdown list, etc.]"
This forces separation. The model understands its constraints before it sees the task. Anthropic’s recent guidance warns that old “think hard” prompts are now drains on Opus 5.5 — the model’s default Medium effort handles most tasks better when prompts are clear rather than demanding. Same principle applies to tool definitions.
Avoid Over-Specification on Tool Ordering
Don’t hardcode the sequence tools should be called. Instead, provide the model with:
- What information it needs to gather first (prerequisites)
- What the natural dependencies are between tools
- What constitutes a “complete answer” so it knows when to stop
If you’re building an agent that needs to fetch user data, then check permissions, then modify records, say that explicitly:
"Complete the task in this order:
1. Fetch user record (tool: get_user)
2. Check if you have permission to modify (tool: check_permission)
3. If permitted, apply changes (tool: update_user)
If any step returns an error, stop and report the error. Do NOT skip steps."
Models respect this. They don’t respect vague suggestions like “you might want to validate first.”
Real Example: Document Classification Agent
This is a task where tool ambiguity kills accuracy. You have tools to fetch documents, classify them, and log results.
TOOLS:
1. fetch_document(doc_id: str) -> { content: str, metadata: {...} }
2. classify_document(content: str, category_set: ["A", "B", "C"]) -> { category: str, confidence: 0.0-1.0, reasoning: str }
3. log_classification(doc_id: str, category: str, confidence: float, classifier_version: str)
PROMPT:
"You are classifying documents into one of three categories: Product (A), Legal (B), or Support (C).
Success = correctly classify all 10 provided documents and log each result.
Failure = any classification without calling log_classification, or any confidence < 0.7 that you don't flag for review.
For each document:
1. Fetch it
2. Classify it
3. If confidence >= 0.9, log it
4. If 0.7 <= confidence < 0.9, log it but add a note: 'Review recommended'
5. If confidence < 0.7, do NOT log. Instead, report: 'Could not classify [doc_id] with confidence.'
Document IDs to process: [list]"
Notice: success is unambiguous (10 results logged), constraints are clear (confidence thresholds matter), and tool dependencies are explicit. The model doesn’t have to guess whether it should log low-confidence results.
Monitor Tool Selection, Not Just Tool Calls
Track which tools the agent picked and why it picked them. If your agent consistently chooses the wrong tool (fetching when it should search, or vice versa), that’s a prompt problem, not a model problem. BM25 keyword search actually beats 8B embedding models at tool retrieval on CPU while matching accuracy, so sometimes the issue is your tool definitions are too similar or the prompt isn’t distinguishing them clearly enough.
A simple heuristic: add a “reasoning” field to your agent’s internal thoughts and log it. If the reasoning is vague (“I’ll try the search tool”), your prompt is vague. If it’s specific (“I need exact titles, not content matching, so I’ll use search_docs with the doc_type filter set to ‘API’”), your prompt is working.
Avoid the Hallucination Trap on Tool Availability
Agents hallucinate tool calls that don’t exist. Prevent this by:
- Explicitly listing only the tools available, not tools that “might” exist
- Making error messages from tool calls informative enough that the agent can recover (“Tool not found” vs. “search_legacy is deprecated; use search_docs instead”)
- Testing your tool schema parsing before production. JSON schema validation should happen on your end, not theirs.
Structured output validation matters here. If your agent is supposed to output tool calls as JSON, validate the schema yourself. Don’t assume the model’s output is well-formed.
Bottom line: Structure prompts like a spec, not like a suggestion. Define tools with the precision you’d use for an API. Explicit success criteria and tool dependencies beat clever reasoning every time.
Question via Hacker News