Where randomness meets reason

Paper of the Week — Looped Language Models Improve Compositional Tool Calling Looped LLMs cut tool-calling errors by maintaining hidden state across API calls — a quiet fix for multi-step agent reliability that costs zero extra parameters. 2026-08-20T12:00:00.000Z Paper of the Week Paper of the Week researchpapersarxivpractical-ai

Paper of the Week — Looped Language Models Improve Compositional Tool Calling

Looped LLMs cut tool-calling errors by maintaining hidden state across API calls — a quiet fix for multi-step agent reliability that costs zero extra parameters.

researchpapersarxivpractical-ai
Weekly One research paper, broken down for people who build things.

Looped Language Models Improve Compositional Tool Calling

Andrei Cristian Popescu, Haitz Sáez de Ocáriz Borde, Pietro Liò. Published 2026-08-XX. arXiv:2608.18171

One sentence summary

Looping a language model’s output back into its own input across API call steps — without adding parameters — meaningfully improves accuracy on multi-step tool-calling tasks where intermediate state must be preserved.

Why this paper

As agentic frameworks mature, the weak link isn’t single tool calls — it’s compositional chains where a model must track state across three, five, or ten sequential API invocations. Most fixes attack this with more context stuffing or bigger models; this paper tries a structural solution instead.

What they did

The authors take an existing frozen LM and run it in a loop: after each tool call, the model’s intermediate hidden state (or a compressed representation of it) is fed back as additional context for the next step, rather than relying solely on the growing token history. They evaluate this on compositional tool-calling benchmarks that require coordinating multiple APIs and maintaining partial results across turns — the kind of task that breaks naive ReAct-style agents when chains exceed 3–4 steps.

Key findings

  • Looped models outperform standard single-pass baselines on compositional tool-calling, with accuracy gains most pronounced on tasks requiring 4+ sequential tool invocations
  • The improvement holds across model sizes tested, suggesting the mechanism isn’t just compensating for capacity
  • State preservation via recurrence reduces error accumulation compared to relying on the context window alone to track intermediate results
  • The approach requires no fine-tuning — the base model weights are unchanged, making it a drop-in architectural wrapper
  • Performance degrades gracefully on simpler (1–2 step) tasks, so the overhead doesn’t hurt shallow pipelines

Why it matters for practitioners

If you’re building agents that chain tool calls — think: fetch customer record → compute eligibility → call pricing API → write result — context-window-based state tracking silently loses information as chains grow. This paper gives you a principled reason to architect your agent loop so the model sees its own prior hidden state, not just a transcript. It’s especially relevant now that modern agentic frameworks increasingly support MCP and long-horizon task orchestration.

What you can use today

  • Implement a lightweight recurrence wrapper around your existing LLM calls: serialize the model’s reasoning trace or a compressed scratchpad and prepend it as a structured system block on each loop iteration — the paper’s setup approximates this without needing internal access to weights
  • When benchmarking your own multi-step agents, test specifically at 4+ tool-call depth; that’s where the gap between flat-context and recurrent approaches opens up
  • If you’re using Claude’s coding tools or Gemini Managed Agents for long-horizon tasks, consider whether your orchestration layer is preserving intermediate reasoning state or discarding it between steps — this paper suggests the latter is a measurable source of failure