Office Hours — How do you structure prompts and workflows to get AI coding assistants to produce reliable, production-ready code instead of plausible-looking garbage?

How do you structure prompts and workflows to get AI coding assistants to produce reliable, production-ready code instead of plausible-looking garbage?

The problem isn’t that models are dumb. It’s that you’re asking them to do hard things with zero feedback signal. A model can generate syntactically correct Python that compiles and runs, fail spectacularly at runtime, and have no way to know it failed. Most teams treat this like a prompt engineering problem when it’s actually a systems problem.

The Core Issue: No Feedback Loop

The biggest structural mistake is treating code generation as a write-once activity. You prompt the model, it outputs code, you review it, and if it’s wrong you either fix it yourself or re-prompt. That’s not how humans write production code. Humans write code, run tests, see failures, fix them, and iterate.

AI coding systems need the same loop. The difference is the model should be in the loop too.

Structure Your Workflow for Verification, Not Generation

Set up your prompts and workflows so the model can see concrete failure signals and fix its own mistakes.

The real pattern: Give the model write access to a sandboxed environment (or a test-driven workflow), let it generate code, run the tests, feed back the failures, and let it iterate. This is not “iterative prompting” where you manually tweak the prompt each time. This is automatic feedback-driven refinement.

Example workflow:

1. Model generates initial code + test file
2. Tests run in sandbox
3. Model sees: "FAILED test_user_creation: KeyError 'email'"
4. Model regenerates the function that failed
5. Tests run again
6. Success: model moves to next task

The output quality jumps dramatically because the model has visibility into what broke, not a vague “make this better.”

Prompt Structure: Task Decomposition + Verification Checkpoints

Don’t ask the model to build the whole feature at once. Break it into small, verifiable pieces.

# Instead of: "Build a user authentication system"
# Ask:
"""
1. Implement UserRepository.create(email, password_hash) -> User
   - Must validate email format
   - Must hash password with bcrypt
   - Must raise ValueError if email already exists
   - Implement unit tests first

2. Implement UserRepository.get_by_email(email) -> User | None
   - Must handle case-insensitive lookups
   - Implement unit tests first

3. Implement authenticate(email, password) -> User | None
   - Use UserRepository.get_by_email
   - Use bcrypt.checkpw to verify
   - Return None if user not found or password wrong
   - Implement unit tests first
"""

Each step has a clear success criterion: the tests pass. The model can’t hand-wave its way through. If it generates broken code, the test feedback forces a fix.

Use Model-Specific Strengths

Claude Opus 5 is strong at understanding existing codebases and making coherent architectural decisions. GPT-5.6 Sol is faster for simple transformations. Gemini 3.5 Flash excels at reading and understanding complex code structures. Match the model to the task, not the other way around.

For reliability, Claude Opus 5 tends to be more conservative and explicit about uncertainty. If you’re in a high-stakes environment and latency isn’t critical, use it. If you’re iterating fast and can afford more generations, Gemini 3.5 Flash’s speed lets you fail and retry faster.

Constrain the Output Space

Don’t let the model freely generate code. Constrain what it can output.

Use structured outputs (JSON mode, XML schemas) to force the model to present its code in a parseable format. Ask for code + a brief explanation of what it does + identified assumptions + edge cases it doesn’t handle. This sounds bureaucratic but it’s essential: the act of the model explaining what it did forces it to reason about correctness, and you get visible uncertainty signals (“This assumes the database is already initialized”).

The Tool-Calling Pattern That Actually Works

If your model has access to tools (execute code, run tests, read files), make sure the workflow has friction that prevents hallucination.

Bad: Model calls run_tests() and sees “All tests pass” but never actually looks at the output.

Good: Model calls run_tests(), sees structured output with pass/fail and line coverage, then calls read_file(path) to inspect the code it just wrote before confirming it’s done.

Add a step where the model explicitly logs what it’s doing: “I’m now running the test suite to verify the implementation. If tests fail, I’ll examine the error and fix it.”

Real-World Checklist

Before you declare code “production-ready” from an AI assistant:

• The model ran tests. You have the test output showing pass rates.
• The model read the code back to itself and explained what it does.
• Coverage metrics show the code path you care about is tested.
• The model attempted at least one edge case (empty list, None input, large numbers).
• The model didn’t hallucinate dependencies or imports—all of them exist and are available.
• You ran a linter or type checker and the model fixed the warnings.

This isn’t about trusting the model. It’s about making trust unnecessary through verification.

The Token Cost Reality

Structured workflows with feedback loops cost more tokens upfront. A single prompt might generate bad code cheaply. A task with test-driven iteration might cost 3-5x the tokens. But your actual cost per production feature goes down because you’re not paying engineers to debug and rewrite the garbage.

Databricks’ benchmarking found that code agents optimized for cost (fewer iterations, simpler tasks) actually had worse real-world ROI than agents designed for correctness. They’d generate cheaper code but engineers spent more time fixing it. The inflection point: once agent cost exceeded ~30% of an engineer’s time spent reviewing and fixing the output, adding more tokens to the agent loop became economically rational.

The Honest Part

Models still fail unpredictably on ambiguous tasks. If the spec is vague, the model will fill in gaps confidently and incorrectly. If there’s no clear test signal, the model can’t self-correct. You can’t engineer your way around a fundamentally underspecified problem.

But if you have a clear definition of done (tests pass, linter passes, coverage threshold met), the right workflow structure turns code generation from a lottery into a directed process.

Bottom line: Structure workflows so AI coding assistants see their own failures through automated tests and verification steps, not through human code review. Task decomposition plus test-driven feedback loops are how you get from “confident bullshit” to “actually works.”

Question via Hacker News