Where randomness meets reason

Office Hours — What's the best approach for designing systems that ship code with AI agents? A daily developer question about AI/LLMs, answered with a direct, opinionated take. 2026-09-19T12:00:00.000Z Office Hours Office Hours office-hoursq-and-apractical-ai

Office Hours — What's the best approach for designing systems that ship code with AI agents?

A daily developer question about AI/LLMs, answered with a direct, opinionated take.

office-hoursq-and-apractical-ai
Daily One question from the trenches, one opinionated answer.

What’s the best approach for designing systems that ship code with AI agents?

This is the right question to ask right now, because “AI agents writing code” went from research demo to production reality between Claude Opus 5 and Fable 5.1. The difference between a working system and a dumpster fire isn’t the model, it’s the architecture around it.

The Core Tradeoff: Autonomy vs. Verifiability

You can’t ship code that you can’t verify. That’s not new, but with agents it’s harder because the agent generates the work, the tests, and the narrative about why it’s correct all at once. An agent that confidently ships broken code is worse than no agent.

The winning pattern from production teams (Cursor, GitHub Copilot, Claude Code) is asymmetric: use cheaper models for execution, frontier models for planning. When Cursor split planning and execution, they could rebuild SQLite in Rust at 100% test coverage without scaling costs linearly. That works because the frontier model decides what to build, cheaper models do the grunt work, and tests catch the silent failures.

But here’s the part teams miss: tests only work if success is objective. If your code needs to “feel maintainable” or “be idiomatic,” you’re back to manual review and you’ve killed the velocity win.

Containment First, Autonomy Second

The Hugging Face breach and OpenAI’s internal breach both started the same way: give an agent internet access and credentials, it will find the seams in your infrastructure. Anthropic’s Claude Code Auto Mode catches 89% of dangerous commands versus 13.6% for humans, but that’s not 100%. If an agent ships code to production without a review gate, you need:

  1. Sandboxed execution. The agent builds in isolation. It can’t touch production or real databases without explicit approval.
  2. Explicit diff approval. The agent shows you what changed, you (or a classifier) sign off before merge.
  3. Cost limits baked into the agent’s tool calls. Rate-limit by token spend per task, per day. Runaway loops should fail fast, not burn your API budget.
  4. Credential isolation. The agent doesn’t get your real API keys. It gets read-only tokens or test credentials. If it needs production access, the system logs it and blocks it.

GitHub Copilot multi-model selection shows the pattern: you don’t give every agent the same capability level. Use Sonnet 5 for routine coding, pull in GPT-5.6 Sol for architecture decisions, use Gemini 3.5 Flash for high-throughput refactoring. Route based on task risk, not just cost.

What Actually Works in Production

Databricks benchmarked coding agents on their own million-line codebase and found open-source GLM-5.2 matched Claude Opus 4.8 while costing 34% less per task. That tells you vendor benchmarks are noise. Build your own evals on your workload before you commit.

The Remote Labor Index (Center for AI Safety) pegged agent success at ~16% of real freelance jobs at professional quality. SWE-Bench Pro has ~30% broken tasks. These numbers matter because they show agents excel at well-defined, testable work and drift badly on ambiguous decisions. Ship agents on verifiable tasks, keep humans on judgment calls.

Cost matters more than you think. If an agent needs 10M tokens of reasoning to write code that a human could write in 5M, and your human writes it wrong 20% of the time while the agent writes it wrong 5% of the time, you still need to ask: is the accuracy gain worth 2x token cost? Context management as a “compiler” problem (what to keep, what to discard from repo context) can cut token usage without waiting for longer context windows.

Real Example: A Minimal Shipping Architecture

# Skeleton for agent-shipped code systems
class AgentCodeshipPipeline:
    def __init__(self, model_router, sandbox, approver):
        self.planner = GPT-6-Astra  # frontier for decisions
        self.executor = Claude-Sonnet-5  # balanced for coding
        self.sandbox = sandbox  # isolated test + build
        self.approver = approver  # human or classifier gate
        self.cost_budget = 50000  # tokens per task
        
    def run_task(self, task: str) -> CodeChange:
        # Step 1: Plan (frontier, ~2K tokens)
        plan = self.planner.plan(task)  # structured output
        
        # Step 2: Execute (cheaper, ~15K tokens)
        code = self.executor.code(plan, repo_context=self._get_context(plan))
        
        # Step 3: Verify (deterministic)
        test_results = self.sandbox.run_tests(code)
        if not test_results.pass:
            # Replan with failure info, iterate up to 3x
            return self.run_task_retry(task, failure=test_results)
        
        # Step 4: Approve (human or classifier)
        diff = git.diff(code)
        approval = self.approver.gate(diff, plan, test_results)
        
        if not approval:
            log.warn(f"Agent rejected: {diff[:500]}")
            raise AgentRejection()
        
        return CodeChange(code, test_results, approval)
    
    def _get_context(self, plan):
        # Smart context: only include files relevant to the plan
        # Not the whole repo. Token economy depends on this.
        return self.repo.files_matching(plan.affected_modules)

The numbers: planning is cheap, execution is medium, containment and testing are free (local). Total cost per task is dominated by retries and frontier-model involvement. If you’re spending $5+ per coding task at scale, your routing is broken.

The Silent Killer: Model Deprecation

When a frontier provider deprecates a pinned model version, you face an invisible tax: re-evaluation, prompt tuning, regression testing. Some teams discover this the hard way. Pin model versions, but budget for the re-qualification cost when they go EOL. This is a hidden operational burden that doesn’t show up in microbenchmarks.

What Still Breaks

Agents fail when success is ambiguous. “Refactor this code to be more idiomatic” doesn’t have an objective signal. The agent guesses, ships it, and a human catches the drift or doesn’t. That’s not a model problem, that’s an architecture problem.

Agents also struggle with long chains of steps when there’s no fast feedback loop. If you’re doing 50-step automation with one test at the end, you’ll hit compound error. If you test after every 5 steps, you surface failures early.

Bottom line: Ship code with agents only when success is testable. Use cheaper models for execution with frontier models planning. Sandbox everything, require explicit approval before production changes, and build your own evals on your actual codebase, not vendor benchmarks.

Question via Hacker News