Office Hours — How do you architect swarms of AI agents to coordinate work on complex tasks?

How do you architect swarms of AI agents to coordinate work on complex tasks?

The honest answer: most swarms fail because people copy research papers instead of modeling token economics first. You’ll triple your costs before you ever get coordination working.

Start with the economic model, not the architecture.

Multi-agent systems leak tokens like a sieve. Each agent needs context about what others are doing. That context has to stay fresh. If you’re not explicitly tracking token spend per agent per decision, you’ll wake up to a $50K bill on a task that should cost $12. Cursor’s split architecture (frontier models for planning, cheaper models for execution) works because they price it backward: plan once, execute many times. Most teams do the opposite.

Before you wire up any orchestration, calculate the token cost of a single round-trip through your system. Include: the shared context all agents see, the state synchronization overhead, the planning model’s reasoning budget, and fallback retries. Multiply by your expected number of steps. If it’s more expensive than a single frontier model solving the whole thing, you’ve already lost.

Separate concerns: planning, execution, and state.

The architecture that actually works splits three things clearly.

Planning layer: One capable model (Claude Opus 5.5 or GPT-6 Astra) reasons through the whole task once, produces a structured plan, and breaks it into discrete subtasks. This is your expensive, slow step. Do it once per job, not per agent.

Execution layer: Cheaper models (Claude Sonnet 5.5, GPT-6.1 Sol, or open-weight like DeepSeek-V4.1-Flash) run the actual work. They don’t re-reason the entire problem. They execute a specific subtask, report results, and stop. Cache the plan and feed it to all of them.

State layer: A shared, immutable log of what happened. Not a database that agents write to concurrently (chaos). A single source of truth that gets read by all agents, updated by a coordinator, never written to by workers. Claude Code agents running in parallel on macOS and Linux can message each other through a shared terminal context—that’s a good pattern. Tools like Temporal or a simple Redis queue with event sourcing work fine for less fancy scenarios.

Here’s a concrete setup:

# Single planning call—expensive, do once
plan = frontier_model.plan_task(user_request, model="claude-opus-5-5")
# Output: [
#   {"task_id": "retrieve_docs", "input": "..."},
#   {"task_id": "summarize", "depends_on": ["retrieve_docs"]},
#   {"task_id": "format_report", "depends_on": ["summarize"]}
# ]

# Execution layer—cheap, parallel, stateless
results = {}
for task in plan:
    if task["depends_on"]:
        task["context"] = {dep: results[dep] for dep in task["depends_on"]}
    
    result = cheap_model.execute_task(
        task=task,
        model="claude-sonnet-5-5",  # or grok-4.7, deepseek-flash
        system="You are executing a single step in a larger workflow. Do not re-plan."
    )
    results[task["task_id"]] = result
    log_event({"task_id": task["task_id"], "status": "done", "result": result})

return results

The key discipline: the cheap execution layer never gets the full context. It gets only what it needs for that step. It doesn’t see the user’s raw request. It doesn’t re-reason the plan. That’s how you keep token cost linear instead of exponential.

Coordination is usually overrated.

Real swarms in research (like DeepMind’s four Gemini 4 Argon agents beating Minecraft) work because there’s a shared environment with fast feedback: “Did that action work?” A test suite passes or fails. The game state is observable. That’s coordination.

But most business workflows don’t have that. You’re coordinating agents across APIs, databases, and loosely-coupled services where the feedback is slow and messy. In those cases, explicit state machines outperform “emergent coordination” every time. Define your states (waiting for input, processing, needs human review, done) and let agents transition through them sequentially.

DeepSeek-V4.1-Flash at $0.30/$1.20 per M (off-peak) makes orchestrating many weak agents economical for the first time. But orchestration overhead still matters. Three sequential cheap calls beat one poorly-written prompt using a frontier model maybe 30% of the time. The other 70%, you’ve just added latency and debugging complexity for no gain.

Handle the hard case: partial failures and retries.

Here’s where most architectures break. One agent fails. Now what? Do you replay the whole thing? Retry just that step? Roll back state?

The robust pattern: each agent writes an idempotent event. Include a unique task ID and a hash of the input. If the same agent gets the same input twice, it returns the cached result from the log without re-executing. That handles accidental retries. For real failures, a separate recovery coordinator reads the log, finds the gap, and either retries the failed step or escalates to a human.

Don’t let agents retry each other. That’s how you get exponential blowup. Have one coordinator own retries and have a hard limit (two retries, then human review).

Real example: document review swarm.

You want to review 100 contracts for risky clauses. Here’s what doesn’t work: spawn 100 agents, each reading a contract, all writing findings to a shared database. Chaos. Bottleneck on writes. Redundant work.

Here’s what works:

One frontier model scans a sample of contracts, produces a rubric of “things to check” (liability caps, auto-renewal clauses, etc.).
That rubric gets cached (at Sonnet rates, cache reads are $0.10/M—orders of magnitude cheaper than re-processing).
Cheap models fan out: each one reads a contract, checks it against the cached rubric, writes a structured report.
A single downstream pass aggregates: “How many contracts have auto-renewal without escape clauses?” “Which ones need human review?”

Token cost: roughly $0.50 per document (rubric is one-time, cheap models do the work, no retry storms). You’re done.

If you’d instead had 100 agents all reasoning from scratch and writing to a database, you’d be at $5+ per document and debugging concurrency issues.

The uncomfortable truth about scale.

Swarms work well up to maybe 10 parallel agents working on the same problem. Beyond that, coordination overhead eats your gains. If you need 100 agents working on the same task, you’re not building a swarm—you’re building a data pipeline with batch processing. Use map-reduce, not agent orchestration. Different tools, different mental model.

Multi-agent systems shine when agents are loosely coupled and working on different things: one agent building a feature, another running tests, another writing docs. That’s how Anthropic’s testing disclosed Claude breaches—because agents with real autonomy in different environments can discover things humans miss. That’s also why containment and monitoring are non-negotiable.

Bottom line: Model token economics before you add coordination complexity. Separate planning from execution, use cheap models for workers, and keep state updates centralized and immutable. You’ll spend 80% less on tokens and debug 10x faster than a “fully distributed swarm” that sounded cool on a whiteboard.

Question via Hacker News