Office Hours — How do you decide what parts of your production codebase are safe for LLMs to write, and what requires human review?

How do you decide what parts of your production codebase are safe for LLMs to write, and what requires human review?

This is the wrong mental model. You’re not carving up your codebase into “LLM-safe” and “human-only” zones. That’s security theater. What you’re actually doing is evaluating two separate things: the blast radius if the LLM gets it wrong, and whether you have a fast objective signal to catch mistakes.

The Real Constraint Is Verifiability

LLMs are reliable at tasks where you can immediately know if they’re wrong. A test suite passes or it doesn’t. A linter catches style violations. A type checker validates contracts. A database query either returns the right data or doesn’t.

They’re unreliable at tasks where correctness is subjective or delayed. Is this refactor safe? Does this architecture scale? Is this code maintainable? These questions take days or weeks to answer, and by then the LLM-generated code is already in your deploy pipeline.

Claude Opus 5 combined with Auto Mode achieves 0% prompt injection attack success across 129 browser agent test scenarios. But that’s because there’s a classifier catching risky patterns. For your code, there’s no classifier. You’re the classifier, and you won’t catch everything in code review.

Bucketing by Signal Quality

Split your codebase like this instead:

Fast feedback, high confidence: Unit tests, integration tests, API handlers that return structured responses. CRUD operations with clear schemas. Data validation and transformation pipelines. Anything that fails loudly and immediately. The LLM can own this. Your test suite is the safety net. If Claude Code or Cursor Agent generates a buggy handler and the test catches it, you’re not worse off than having written it yourself—you’re just faster. Databricks benchmarked its own codebase and found open-source GLM-5.2 matched Claude Opus 4.8 on task completion while cutting costs 34%. That’s because their codebase had robust tests.

Slow feedback, high complexity: Distributed systems logic, concurrency primitives, anything touching authentication or secrets. Event-driven orchestration. Long-running state machines. These have failure modes that show up under load or after hours of operation. The LLM can draft these, but a human who understands your infrastructure needs to review every line. Not because the LLM can’t write correct code, but because you won’t know if it’s correct for months. Banks are already locking down AI agents that access customer credentials because the security properties of autonomously-generated code touching that layer aren’t fully understood yet.

Subjective correctness: Architecture decisions. API surface design. Performance-critical paths where the algorithm choice matters more than implementation. Database schema changes. The LLM should not be the primary decision-maker here. It can generate options and explain tradeoffs, but a human makes the call.

Regulatory or compliance-sensitive: Anything that touches PII, payment processing, healthcare data, or audit trails. If a regulator is going to ask how a decision was made, you need a human fingerprint on that code. Full stop. Not because LLMs always get it wrong, but because you need to be able to explain it and defend it in a way that “Claude generated it” doesn’t satisfy.

What Actually Breaks in Production

Agentic AI in production has a silent failure mode: when a tool breaks or an API changes, the system often keeps generating confident answers from hallucinations while your dashboards stay green. If Claude Code pushes a commit that looks syntactically correct but calls a deprecated API, your tests will catch it if you test against real systems. They won’t if you only have unit tests.

Here’s a concrete example. You let an agent auto-generate database migrations. The migration passes local tests (single-node SQLite). It ships to production (PostgreSQL cluster), and some edge case around concurrent writes or constraint ordering breaks. The agent has no way to know. The problem isn’t that the agent hallucinated—it’s that you didn’t have the right signal. The agent needs to run migrations against a staging database that mimics production, or the migration doesn’t ship.

The Human Review Question

Code review from a human doesn’t catch AI-generated errors better than tests do. A study on junior patent attorneys found that AI tools made them instantly better at mechanical tasks, but gains vanished once the tool was removed. They weren’t building judgment. Similarly, if you’re reviewing every line of LLM output by hand, you’re not getting the speed benefit. You’re just doing work twice.

Better pattern: code review for architectural decisions and security-sensitive code. Automated checks for everything else. Linters, type checkers, tests, static analysis tools. If the LLM passes all of those and changes pass review at the design level, merge it.

If you’re reviewing LLM-generated code line by line because you don’t trust it, you’ve already lost the efficiency game. That means either the code is too complex to safely generate (which signals you should simplify it or split it up), or your testing and linting are too loose.

Where Rate Limits and Compute Matter

Anthropic’s IPO filing exposed that Claude’s rate limits aren’t really about customer demand—they’re about raw compute scarcity. This means you can’t just let your agents thrash. If an agent is generating code that fails tests, regenerating, failing again, and burning through your token budget, you need backpressure. Claude Fable 5.1 allows you to identify software vulnerabilities in source code; use that. Have the agent run a vulnerability check before submitting PRs. Have it run tests locally before pushing. Cheaper models can verify structure and lint compliance. Reserve the expensive reasoning for the parts where it actually matters.

Cursor’s redesigned agent architecture separates planning (frontier models like Claude Opus or GPT-6 Astra) from execution (cheaper models). That’s the pattern that works: expensive model plans, cheap model executes, tests verify.

Bottom line: Stop asking “is this code safe for an LLM to write?” and start asking “can I immediately detect if this is wrong?” LLMs own anything with fast, objective feedback. Humans own anything where correctness takes time or judgment to establish. Use tests and linters as your actual safety boundary, not code review.

Question via Hacker News