Office Hours — How do you safely let AI agents run unattended to build features, and what safeguards do you put in place?

How do you safely let AI agents run unattended to build features, and what safeguards do you put in place?

The honest answer: most teams aren’t doing this safely yet, and the ones that are have built a lot of custom infrastructure. Autonomous agents in production work well when success is objective and fast to verify (tests pass, linter clears, CI gates open). They fail silently when the signal is subjective or delayed (is this refactor actually safe? Did we introduce a subtle security hole?). If you’re shipping agents unattended, you need to architect around what they’re provably good at and keep humans in the loop for everything else.

The Real Failure Mode

The mistake most teams make is thinking safeguards are primarily about preventing malicious behavior. They’re not. The real risk is silent degradation. An agent that confidently commits broken code, introduces a dependency with a known CVE, or ships a feature that technically works but violates your architectural principles is worse than one that fails loudly. You notice the loud failure. The silent one ships to prod.

This matters because agent capability is skewed. Coding agents like Claude Code and GitHub Copilot (with GPT-5.5, Claude Sonnet 5, or Gemini 3.5 Flash) excel at individual tasks with clear pass/fail criteria. They’re mediocre at multi-step tasks where intermediate decisions require taste or judgment. They’re terrible at preventing themselves from doing the wrong thing confidently.

Safeguard Layer 1: Constraint the Action Space

The simplest safeguard is to strictly limit what the agent can touch. Don’t give it blanket repository access. Give it a sandbox environment—a branch, a separate service, a test database—where actions are reversible. This is table stakes.

# Bad: Agent can push to main
agent.run(
    task="Fix the bug in payment processing",
    repo_access="full",
    can_push=True
)

# Better: Agent writes to isolated branch, human reviews before merge
agent.run(
    task="Fix the bug in payment processing",
    repo_access="read_main_write_feature_branch",
    can_push=False,  # Requires human approval for merge
    isolated_test_env=True
)

The second approach costs more human time (code review), but you’re trading time for safety. If you’re building a feature without unattended agents, you’re doing code review anyway. The agent just makes it faster to generate the initial candidate.

Safeguard Layer 2: Separate the Executor From the Decider

Don’t let the agent decide when to ship. Separate the execution of tasks (run tests, write code, generate artifacts) from the decision to deploy or merge. The agent generates candidates; a human or a deterministic policy decides whether to accept them.

This is critical because agents have no real understanding of risk. They optimize for task completion, not for business impact or safety. An agent that sees “deploy this new database migration” as a subtask will happily deploy it if the immediate tests pass, without understanding the blast radius or rollback strategy.

# Agent runs autonomously but can't cross the deployment boundary
task_result = agent.run(
    task="Implement feature X",
    boundaries={
        "can_write_code": True,
        "can_run_tests": True,
        "can_open_pr": True,
        "can_merge": False,  # Hard stop
        "can_deploy": False  # Hard stop
    },
    notifications={
        "on_pr_open": "slack_channel",
        "on_test_failure": "slack_channel"
    }
)

# Human reviews the PR, runs manual tests, makes deploy decision
if review_approved(task_result):
    deploy(task_result)

Safeguard Layer 3: Monitor the Cost and Token Trail

Agents fail by getting stuck in loops, retrying failed operations obsessively, or exploring dead-end solutions for hours. Set hard cost limits, token budgets, and timeout windows. Make these visible and alertable.

agent_run = agent.run(
    task="Fix the flaky test",
    max_cost=10.00,  # USD
    max_tokens=500_000,
    timeout_minutes=30,
    cost_alert_threshold=7.50,  # Notify at 75% of budget
    on_cost_exceeded="terminate_immediately"
)

# Log every API call and tool invocation
for call in agent_run.trace:
    log.info({
        "model": call.model,
        "tokens_in": call.tokens_in,
        "tokens_out": call.tokens_out,
        "cost": call.cost,
        "tool": call.tool_called,
        "timestamp": call.timestamp
    })

Grok 4.3 and Claude Opus 4.8 have better instruction-following and lower hallucination rates than older models, which means fewer loops. But loops still happen. A $10 budget limit on a single task forces the agent to either succeed or fail fast. A $1,000 budget with no visibility lets it spin for hours on a problem that needs human input.

Safeguard Layer 4: Verify Against a Checklist Before Acceptance

Before the agent’s output moves forward, run it through deterministic checks that don’t require taste. This isn’t a replacement for code review; it’s a gate that catches obvious issues before they reach a human.

def verify_agent_output(code_changes, manifest):
    """Checklist-based verification of agent-generated code."""
    checks = {
        "no_hardcoded_secrets": not re.search(r'(api_key|password|secret)', code_changes),
        "linter_passes": lint(code_changes) == [],
        "type_checking_passes": mypy(code_changes) == [],
        "test_suite_passes": pytest(code_changes) == 0,
        "no_new_vulnerable_deps": check_dependencies(code_changes),
        "matches_style_guide": code_matches_formatting_standard(code_changes),
        "architecture_layers_respected": no_circular_imports(code_changes),
    }
    
    failed_checks = [k for k, v in checks.items() if not v]
    if failed_checks:
        raise AssertionError(f"Output failed checks: {failed_checks}")
    
    return True

This sounds basic, but most teams skip it. The agent generates a PR, a human glances at it and merges it, and six hours later you’re rolling back because the linter wasn’t enforced. Deterministic gates are cheap; rollbacks are expensive.

Safeguard Layer 5: Audit the Decision Chain

When something goes wrong, you need to know what the agent was thinking. Maintain a detailed trace of every decision, tool call, and reasoning step. This isn’t just for debugging; it’s for understanding whether the agent is doing something you don’t like but is technically within bounds.

# Example trace structure
agent_trace = {
    "task": "Add caching layer to API endpoint",
    "model": "claude-opus-4.8",
    "steps": [
        {
            "step": 1,
            "reasoning": "Endpoint is being hit 10K times/min, need to reduce load",
            "tool": "grep_codebase",
            "result": "Found endpoint in api/routes.py, 50 LOC",
        },
        {
            "step": 2,
            "reasoning": "Use Redis for caching to avoid database queries",
            "tool": "run_command",
            "command": "pip list | grep redis",
            "result": "redis-py already installed",
        },
        {
            "step": 3,
            "reasoning": "Add cache decorator above endpoint function",
            "tool": "edit_file",
            "file": "api/routes.py",
            "diff": "... decorator added ...",
        },
    ],
    "final_state": "success",
    "cost": 2.34,
    "tokens_used": 12_456,
    "time_elapsed_seconds": 87
}

When you review a PR generated by an agent, you’re not just reading the code; you’re reading the trace. Did the agent skip an obvious security concern? Did it avoid a simpler solution for a convoluted one? Did it introduce a dependency without checking if it’s already in the tree? The trace tells you

Question via Hacker News