Where randomness meets reason

Office Hours — How should code review processes change when most code is AI-generated? A daily developer question about AI/LLMs, answered with a direct, opinionated take. 2026-08-16T12:00:00.000Z Office Hours Office Hours office-hoursq-and-apractical-ai

Office Hours — How should code review processes change when most code is AI-generated?

A daily developer question about AI/LLMs, answered with a direct, opinionated take.

office-hoursq-and-apractical-ai
Daily One question from the trenches, one opinionated answer.

How should code review processes change when most code is AI-generated?

The short answer: traditional code review is broken for AI-generated code, but not in the way people think. The real problem isn’t spotting bugs—it’s spotting confidence without competence, architectural debt baked in by default, and the silent accumulation of technical decisions that humans never explicitly made.

The actual failure mode

Code review works when humans read code written by humans who had to think through tradeoffs. With AI-generated code, the review process often becomes “does this look reasonable?” instead of “is this the right design?” An AI can produce syntactically correct, well-structured code that solves the immediate problem while introducing subtle brittleness or making architectural choices that create dependencies you’ll regret in six months.

Claude Code is now merging 46% of PRs into Anthropic’s own codebase after human review. That’s not a sign that the model is perfect; it’s a sign that the review process has adapted to trust the output incrementally. The problem surfaces later when you realize the model made three architectural assumptions you never questioned because the code “looked fine.”

What actually needs to change

Shift from line-level review to decision-level review. Traditional code review looks for bugs, style violations, and logic errors. With AI-generated code, you need to ask: What architectural choice is this making? Is this choice reversible? Does this couple systems in ways we didn’t intend? These aren’t questions you ask by reading the code—they’re questions you ask by understanding what problem the AI was solving and why it chose this particular solution.

Require explicit context propagation. Make AI systems attach reasoning artifacts to generated code. If Claude Code generates a refactor, it should emit a structured summary: “I changed the error handling pattern from try-catch to Result types because the codebase already uses Result in three other modules, and this aligns with that pattern.” The review process then validates the reasoning, not just the code. Tools like Cursor Agent’s planning layer already separate high-level reasoning from execution code; formalize this in your PRs.

Add determinism checkpoints. AI-generated code often works “well enough” in common cases but has edge-case brittleness. Require tests that explicitly probe boundary conditions, error paths, and assumptions the AI made. If the model assumed a certain input format, your test should verify that assumption breaks gracefully. Code review then becomes “are these tests actually catching what the AI might have missed?” rather than “does the code look right?”

Validate architectural consistency before merging. Before reviewing individual code changes, ask: Does this pattern conflict with patterns elsewhere in the codebase? Is this using a different library than similar functionality three modules over? AI systems are blind to codebase-wide consistency. A human reviewer should run a static check or ask the AI explicitly: “Why are you using HTTP client X instead of the standard client we use elsewhere?” If the answer is “I didn’t know you had a standard,” the review process caught something important.

Concrete example: the difference

Old code review (doesn’t work for AI):

  • Reviewer reads PR, checks syntax, runs tests, looks for obvious bugs.
  • “Looks good to me. Tests pass.”
  • Code merges. Six months later, the error handling pattern in this module is different from the rest of the codebase, and when you need to add observability, you discover it’s not instrumented the same way.

New code review (catches what matters):

AI generates refactor of auth module. PR includes:
1. Code changes
2. Structured reasoning: "I refactored error handling from 
   if-then-else to match-case pattern because it makes error 
   types explicit and aligns with patterns in security.rs and 
   crypto.rs."
3. Tests for: null credentials, expired tokens, malformed headers, 
   concurrent auth attempts
4. Dependency audit: "No new dependencies added"
5. Architectural impact: "Changes internal API of AuthContext; 
   callers updated in 4 places"

Reviewer checks:
- Is the reasoning sound? (Yes, match-case is better here.)
- Do the tests actually catch edge cases? (Tests are weak on 
  concurrent scenarios; ask for improvement.)
- Are all callers actually updated? (Run codebase-wide grep to verify.)
- Does this conflict with ongoing auth work? (Check Jira tickets.)

The second approach catches both bugs and architectural drift. The first catches neither.

Practical implementation

Use your code review tool to enforce a template. If the PR is AI-generated (flag this in commit metadata or PR title), require:

  1. Decision summary: What problem does this solve, and why did the model choose this approach?
  2. Test coverage report: Did the tests actually stress-test the generated code or just verify happy paths?
  3. Codebase impact scan: Does this introduce a new pattern, library, or architectural choice that conflicts with existing code?
  4. Fallback: human implementation. If the generated code solves the problem but doesn’t feel right, require the reviewer to either ask the AI to regenerate with specific constraints or write it manually.

Don’t just scan for bugs and merge. Bugs are the least of your problems with AI-generated code.

What you’re really worried about

The deepest risk isn’t hallucination or syntax errors; it’s invisible architectural debt. An AI can generate correct code that accumulates small decisions—slightly different error handling, a new library for HTTP calls when you already have one, a different concurrency model—that individually are fine but collectively fragment your codebase. By the time you notice, you’ve got six months of code built on misaligned patterns.

Code review on AI-generated code needs to catch these patterns before they compound. That means making the AI’s reasoning visible and auditing it as aggressively as you’d audit an architect’s design doc.

Bottom line: Stop reviewing AI-generated code for correctness and start reviewing it for architectural alignment. Require reasoning artifacts, test coverage audits, and explicit conflict checking against your codebase norms before you merge anything substantial.

Question via Hacker News