Office Hours — What do you do when your AI agents are working?

What do you do when your AI agents are working?

You’re staring at a terminal. The agent is running. It’s been 45 minutes. It’s made 2,847 tool calls. You have no idea what it’s actually doing.

This is the real problem nobody talks about. Not whether agents work—they do, increasingly reliably. The problem is what happens once they do work and you’ve handed them actual autonomy. Now you need to know what they’re doing, catch them before they break something, and explain their behavior to someone who will ask hard questions.

The Visibility Problem Is Real

Autonomous agents generate decision trees that are humanly impossible to trace in real time. An agent calling APIs, reading responses, making branching decisions, and looping back on failures creates a narrative that moves faster than you can follow. Claude Code Auto Mode catches 89% of dangerous commands versus 13.6% for humans, which is great—until you realize humans were approving things anyway. The shift from “human reviews every step” to “AI is the actor and humans supervise approval workflows” inverts the control problem. You’re no longer preventing failures; you’re catching them after the AI has already decided to act.

The Qwen3.7-Max incident is instructive. Alibaba reported 35 hours of autonomous chip-kernel optimization with 1,158 tool calls. That’s not a feature. That’s a warning. If something goes wrong at hour 23, you have no real way to rewind or understand what premise broke the agent’s reasoning.

Logging Beats Explanations

You need logs that capture three layers:

Agent reasoning checkpoints. What did the agent decide at each branching point? This is different from token-by-token LLM generation—you need the structured decision that caused an API call or file write. When an agent fails, you’re asking “which decision was wrong” not “what did the model output.” Use structured logging: decision, reasoning excerpt (not full token stream), consequence, next action.

Tool execution records. Every API call, database query, file operation, or code execution needs a record with inputs, outputs, and latency. Not for auditing later; for real-time pattern matching. If an agent makes the same API call 47 times in a loop, you want to catch that in the 10th iteration, not after token spend doubles.

Outcome signals. Did the agent succeed? Did it get stuck? Did it reach a cost limit? Did it diverge from the task? You need objective measures, not vibe checks. The Remote Labor Index (AISI, July 2026) found agents complete roughly 16% of real freelance work at professional quality—but that number is invisible if you’re not measuring completion and quality against rubrics built upfront.

Here’s a minimal logging structure that actually matters:

import json
from datetime import datetime
from typing import Any

class AgentLog:
    def __init__(self, agent_id: str, task_id: str):
        self.agent_id = agent_id
        self.task_id = task_id
        self.events = []
    
    def log_decision(self, decision_type: str, context: dict, action: str, reasoning: str):
        self.events.append({
            "timestamp": datetime.utcnow().isoformat(),
            "type": "decision",
            "decision_type": decision_type,
            "action": action,
            "reasoning_excerpt": reasoning[:200],  # Don't log full model output
            "context_keys": list(context.keys()),
            "cost_estimate": context.get("estimated_tokens", 0)
        })
    
    def log_tool_call(self, tool_name: str, inputs: dict, result: Any, latency_ms: float, error: str = None):
        self.events.append({
            "timestamp": datetime.utcnow().isoformat(),
            "type": "tool_call",
            "tool": tool_name,
            "input_keys": list(inputs.keys()),
            "success": error is None,
            "error": error,
            "latency_ms": latency_ms,
            "result_preview": str(result)[:100] if result else None
        })
    
    def log_outcome(self, status: str, metrics: dict):
        self.events.append({
            "timestamp": datetime.utcnow().isoformat(),
            "type": "outcome",
            "status": status,  # "completed", "stuck", "cost_exceeded", "diverged"
            "total_tool_calls": len([e for e in self.events if e["type"] == "tool_call"]),
            "metrics": metrics
        })
    
    def write(self, path: str):
        with open(path, 'w') as f:
            json.dump(self.events, f, indent=2)

That’s not comprehensive instrumentation. It’s the minimum that lets you answer: “Did the agent do what I asked? Where did it fail? How much did it cost?”

Real-Time Circuit Breakers

Logging is post-mortem. You also need live gates that stop agents before they burn through your budget or get stuck in loops.

Token budgets matter, but they’re blunt. Better: measure agent decision velocity. If an agent makes 10 decisions per minute but completes zero subtasks, it’s stuck. If it’s calling the same tool repeatedly without consuming the output, it’s looping. Set a “decision ceiling” per time window. The UK AI Security Institute found that test-time token budgets matter enormously (10x increase in token budget raised success ~25%)—but that doesn’t mean you should let agents run unbounded. It means you should tune the limit to your task, measure empirically, and enforce it.

Cost limits are tablestakes. Most teams skip this. Set a hard spend cap per agent run, then map that to a token budget specific to your model and workload. If a task consistently exceeds the budget, it’s telling you something: the task is harder than you thought, or the agent architecture is inefficient. Both are actionable signals, not failures.

The Hardest Part: Success Metrics That Actually Work

You can log everything and still not know if the agent succeeded. This is where most teams fail.

For coding agents, you have a fast objective: tests pass or they don’t. For agentic RAG or document processing, it’s murkier. Did the agent retrieve the right information? Did it synthesize coherently? Did it hallucinate? You need evals, not vibes. Towards AI (August 18) emphasized that evals are becoming the testing standard for non-deterministic agents—traditional metrics fail. Build a rubric for what “done” looks like, score agent outputs against it, and use that signal to improve prompts or routing.

Stripe acquiring OpenRouter is instructive. Model routing—sending simple tasks to cheaper models, hard tasks to frontier models—saved Databricks 34% per task (from $1.94 to $1.28 per task on its own codebase). That only works if you have ground truth about task difficulty and outcome quality. Routing without evals is guessing.

The Monitoring Gap Nobody Is Solving

Agent interpretability remains unsolved. Towards AI (August 20) flagged this starkly: your agent works, but you can’t explain why it made a specific decision or took a specific action. This matters for compliance, for debugging, and for trust. Claude Opus 5 with Auto Mode achieves 0% prompt injection success across 129 test scenarios—which is meaningful, but you still can’t see the reasoning that made the agent reject a malicious instruction.

The practical answer: don’t bet on interpretability yet. Instead, bet on bounded autonomy. Give agents clear success criteria, hard limits (cost, time, action count), and human-in-the-loop gates for risky decisions. Approval workflows where humans see agent intent before execution aren’t bottlenecks if they’re structured right. They’re your safety layer until the field figures out explainability.

Bottom line: Real-time structured logging of decisions and tool calls, combined with hardened cost/decision rate limits and rubric-based outcome measurement, is the operational baseline for autonomous agents in production. Skip interpretability for now; focus on observability, guardrails, and objective success metrics that let you catch failures before they compound.

Question via Hacker News