Learned Orchestration: A Case Study of Sakana Fugu

Most orchestration in this series has been written by people: a harness that calls a model in a loop (see From Function Calling to the Agent Loop), a supervisor that hands tasks to subagents (see Many Agents). The structure is code; the models fill it in. Sakana AI’s Fugu (technical report, June 2026) makes the structure itself learned. Its orchestrators are language models trained to look at a request and decide which frontier models should handle it, and how. The workers stay closed, behind their APIs; only the orchestrator is trained.

This post is a case study of that report: how the two designs work, which earlier ideas they reuse, and what its results do and don’t show. All numbers below are as reported by Sakana; none were reproduced here.

Table of contents

Collective intelligence as a model interface

The premise is that frontier models are no longer interchangeable. They specialize: one is strongest at agentic coding, another at long-context retrieval, another at science questions. A user who could send each request to the right model, or split it across several, would beat any one of them. Fugu packages that as one model: behind its API, an orchestrator decides how a pool of workers handles each request. The report’s pool includes Gemini 3.1 Pro, Claude Opus 4.8 and GPT-5.5, the three it compares against, and the pool can be configured (favoring or excluding providers, for compliance, say) without retraining, though the report doesn’t test that.

There are two releases, at two points of the trade between quality and latency:

• Fugu picks a single worker at each turn, so it costs about one frontier call per turn. It’s a router.
• Fugu-Ultra writes a workflow of up to five steps, each a subtask assigned to a worker from a larger pool, with rules for which earlier results each one sees. It costs several calls and more time.

Fugu: a router with a learned head

Fugu descends from an earlier Sakana coordinator, TRINITY (Xu et al., 2025), which also chose a role for each worker; Fugu drops roles and only chooses who answers. Its orchestrator is a language model, but it doesn’t generate text to make its decision. A small selection head sits beside the model’s usual output layer, reads the hidden state at an early position of the input, and outputs one logit per worker. No tokens are generated, so the decision costs part of one forward pass, and the report describes its latency as comparable to calling a frontier model directly. In a multi-turn task (a coding session, say) it decides again at every turn, from the transcript so far: on one Terminal Bench task the report shows it alternating between GPT-5.5 and Claude Opus, bringing in Claude at critical debugging points. It is the same shape as a mixture-of-experts router (Mixture of Experts), one level up: a learned function from a representation to a choice among experts, except that the experts are whole models behind APIs, the choice is a hard top-1 dispatch, and the router can’t be trained by backpropagating through the experts.

Training has two stages.

Supervised fine-tuning on soft labels. For each training task, every worker attempts it, and each worker’s average reward becomes a target distribution by a softmax with temperature $\tau$: $p_i(j) = \exp(\bar r_{i,j}/\tau) / \sum_{j'} \exp(\bar r_{i,j'}/\tau)$. The head is trained to match it, minimizing the KL divergence from the target (Cross-Entropy and the Softmax Gradient). Soft targets matter here: when two workers are nearly as good on a task, the label says so, instead of declaring one of them wrong. Only the head and a small set of parameters in the backbone are trained: for selected weight matrices, the report decomposes each into its singular vectors and trains only the singular-value scales, keeping the rest fixed.

Evolution strategies on whole tasks. Single-step labels say which worker answers a question best, not which one finishes a long task in a coding assistant. The second stage optimizes that directly: the reward is 1 if a multi-turn trajectory in a coding environment (the report names Claude Code, Codex and OpenCode) completes the task, else 0. That reward can’t be differentiated through, it is sparse and noisy, and the trainable parameter set is small and already in a good region after fine-tuning: the setting where evolution strategies work well (the report also found them more stable than further supervised training on these tasks). It uses sep-CMA-ES, following TRINITY: sample a population of perturbed parameter vectors, score each by its average reward over repeated end-to-end runs, recombine the best into the next parent, and adapt the sampling distribution as it goes. No gradients, only scores.

Fugu-Ultra: the Conductor writes workflows

Fugu-Ultra builds on an earlier Sakana paper, the Conductor (Nielsen et al., 2025). The Conductor is a language model trained with reinforcement learning to write an agentic workflow in natural language. Each step names a subtask, the worker that should do it, and an access list: which earlier steps’ outputs that worker gets to see. The last step’s output is the answer. That format can express best-of-N (several workers, same subtask, then one that picks), a sequential chain, debate, or a tree-structured workflow, and the Conductor chooses per request; it can also assign a step to itself.

It is trained with GRPO (GRPO and Reinforcement Learning from Verifiable Rewards): sample a group of workflows for each training question, execute them, and reward each relative to the group. The reward has two conditions, as the report states them: 0 if the workflow can’t be parsed into subtasks, workers and access lists; otherwise 1 if executing it gives the right answer and 0.5 if not. A valid but wrong workflow thus scores half, which teaches the format first and correctness on top of it. (Since GRPO compares each workflow with its group, the 0.5 only matters in groups where some workflows fail to parse.) Unlike the GRPO of GRPO and Reinforcement Learning from Verifiable Rewards, the report trains without a KL penalty toward the starting model.

Orchestration collapse

Extending the Conductor to agents that call tools, the report guards against a failure it names orchestration collapse (it describes the problem and the fix, without an experiment showing either): when agents share one transcript, the first agent to act on the environment sets the course, and later agents follow its path instead of contributing their own, so a team of three different models behaves like one. This is the multi-agent version of herding, and it defeats the purpose of mixing models. The fix is intra-workflow isolation: each agent sees only its own actions, plus whatever the access list explicitly passes to it. A second mechanism, persistent shared memory across the turns of one conversation, lets agents keep what the environment told them in earlier workflows, since complete isolation would make them repeat work. Isolation within a workflow, memory across the workflows of one conversation.

The results, read critically

The report’s main table, and each model’s margin over the best of its three workers on each benchmark:

# Table 1 of the Sakana Fugu technical report (arXiv 2606.21228v2), as
# reported: Fugu-Ultra, Fugu, then the three workers both orchestrate.
# Baselines are provider-reported where available, so the comparison is
# not a controlled one. Margin = orchestrator minus the best single worker.
rows = {
    "SWE Bench Pro":          (73.7, 59.0, 69.2, 54.2, 58.6),
    "Terminal Bench 2.1":     (82.1, 80.2, 74.6, 70.3, 78.2),
    "LiveCodeBench":          (93.2, 92.9, 87.8, 88.5, 85.3),
    "LiveCodeBench Pro":      (90.8, 87.8, 84.8, 82.9, 88.4),
    "Humanity's Last Exam":   (50.0, 47.2, 49.8, 44.4, 41.4),
    "CharXiv Reasoning":      (86.6, 85.1, 84.2, 83.3, 84.1),
    "GPQA Diamond":           (95.5, 95.5, 92.0, 94.3, 93.6),
    "SciCode":                (58.7, 60.1, 53.5, 58.9, 56.1),
    "tau^3 Banking":          (20.6, 21.7, 20.6, 8.4, 20.6),
    "Long Context Reasoning": (73.3, 74.7, 67.7, 72.7, 74.3),
    "MRCRv2":                 (93.6, 86.6, 87.9, 84.9, 94.8),
}
workers = ("Claude Opus 4.8", "Gemini 3.1", "GPT-5.5")
print(f"{'benchmark':24s} {'best worker':>22s} {'Fugu-Ultra':>11s} {'Fugu':>7s}")
ahead = {"Fugu-Ultra": 0, "Fugu": 0}
for name, (ultra, fugu, *w) in rows.items():
    best = max(w)
    who = workers[w.index(best)]
    print(f"{name:24s} {who:>15s} {best:5.1f}  {ultra - best:+10.1f} {fugu - best:+7.1f}")
    ahead["Fugu-Ultra"] += ultra > best; ahead["Fugu"] += fugu > best
print(f"\nahead of the best single worker: Fugu-Ultra on {ahead['Fugu-Ultra']} of {len(rows)}, Fugu on {ahead['Fugu']} of {len(rows)}")
best_on = {}
for name, (_, _, *w) in rows.items():
    best_on.setdefault(workers[w.index(max(w))], []).append(name)
for k, v in best_on.items():
    print(f"  {k} is the best worker on {len(v)}: {', '.join(v)}")

Full script: fugu_margins.py

benchmark                           best worker  Fugu-Ultra    Fugu
SWE Bench Pro            Claude Opus 4.8  69.2        +4.5   -10.2
Terminal Bench 2.1               GPT-5.5  78.2        +3.9    +2.0
LiveCodeBench                 Gemini 3.1  88.5        +4.7    +4.4
LiveCodeBench Pro                GPT-5.5  88.4        +2.4    -0.6
Humanity's Last Exam     Claude Opus 4.8  49.8        +0.2    -2.6
CharXiv Reasoning        Claude Opus 4.8  84.2        +2.4    +0.9
GPQA Diamond                  Gemini 3.1  94.3        +1.2    +1.2
SciCode                       Gemini 3.1  58.9        -0.2    +1.2
tau^3 Banking            Claude Opus 4.8  20.6        +0.0    +1.1
Long Context Reasoning           GPT-5.5  74.3        -1.0    +0.4
MRCRv2                           GPT-5.5  94.8        -1.2    -8.2

ahead of the best single worker: Fugu-Ultra on 7 of 11, Fugu on 7 of 11
  Claude Opus 4.8 is the best worker on 4: SWE Bench Pro, Humanity's Last Exam, CharXiv Reasoning, tau^3 Banking
  GPT-5.5 is the best worker on 4: Terminal Bench 2.1, LiveCodeBench Pro, Long Context Reasoning, MRCRv2
  Gemini 3.1 is the best worker on 3: LiveCodeBench, GPQA Diamond, SciCode

As reported, each variant beats the best of the three named models on 7 of 11 benchmarks, though not the same 7: both are ahead on only 4. Fugu-Ultra’s largest margins are on LiveCodeBench (+4.7), SWE-Bench Pro (+4.5) and Terminal Bench (+3.9); many of the other wins are about a point or less (Fugu-Ultra +0.2 on Humanity’s Last Exam; five of Fugu’s seven between +0.4 and +1.2). Each worker is the best single model somewhere (Claude Opus on four benchmarks, counting its tie with GPT-5.5 on τ³ Banking; GPT-5.5 on four; Gemini on three), which is the premise of the whole approach. The report also shows where routing goes by domain (on Humanity’s Last Exam, math to GPT-5.5 and chemistry and biology to Gemini), and a set of single demonstrations (an automated research loop over three seeds, reading order in classical Japanese letters, CAD, chess, a Rubik’s cube) that are illustrations rather than measurements.

Three things to keep in mind when reading these numbers:

• The comparison isn’t controlled. Baseline scores come from the model providers where available (fully so for SWE-Bench Pro, GPQA and MRCRv2), otherwise from public leaderboards (LiveCodeBench, SciCode, long-context reasoning, τ-bench) or Sakana’s own runs, while Fugu’s were run by Sakana with its own harnesses (on SWE-Bench Pro, the mini SWE-agent harness with up to 1,000 turns). Harnesses, turn limits and judging differ, and differences of a few points are within what setups alone can produce (Evaluating LLMs). No intervals or run counts are given.
• The router sometimes trails a single model. Fugu is 10.2 points behind Claude Opus on SWE-Bench Pro and 8.2 behind GPT-5.5 on MRCRv2. Taken at face value, always calling that one model would have done better there; given the mismatched harnesses, the honest reading is that routing doesn’t show a gain on those two, not that it was proven to lose.
• The table has no cost or latency column. Fugu-Ultra makes several frontier calls per request. A gain of a point or two at several times the cost is a different trade from a free one, and the report’s own framing (Fugu for everyday use, Ultra for the hardest problems) says as much.

None of this makes the results wrong. It makes them the vendor’s measurements of its own system, which is what a technical report is, and the honest reading is “promising, unreplicated”.

What carries over

The ideas outlast this particular system:

• Routing is a learning problem with soft labels from measured outcomes, and the same machinery works at any scale: experts inside a layer (see Mixture of Experts), models behind APIs (here), and cheap versus expensive models in production (see Where Models Run).
• Orchestration can be trained end to end with whatever optimizer fits the reward: supervised targets where per-step labels exist, evolution strategies for a small parameter set and an all-or-nothing outcome, and policy-gradient RL (GRPO) when the orchestrator writes a long output.
• Multi-agent systems need structural safeguards against herding. Isolation by default, with explicit channels for what should be shared, is a design rule for any team of agents, learned or not.

Where this leads

• Engineering Around the Model returns to orchestration written by people: the harness, its loops and graphs, and a definition of done the harness checks.
• Evaluating LLMs gives the tools for reading a table like Fugu’s: error bars, how many questions it takes to tell two models apart, and why models should be compared on the same items.
• Where Models Run applies routing between models in production, between cheap and expensive models and between cloud and device.

Takeaways

• Fugu makes the orchestrator a trained model over closed frontier workers: a router (Fugu) or a workflow writer (Fugu-Ultra).
• The router is a selection head on one hidden state, deciding again at every turn, like an MoE router over whole models; it is trained by KL to soft labels from worker rewards and then by evolution strategies on end-to-end coding tasks.
• The Conductor behind Fugu-Ultra writes subtasks, worker assignments and access lists in natural language, trained with GRPO on a 0 / 0.5 / 1 reward for unparseable, valid but wrong, and correct workflows.
• Shared transcripts make agent teams herd after the first actor (“orchestration collapse”); the report proposes isolating each agent’s trajectory, without measuring the fix.
• As reported, each variant beats the best of three frontier models on 7 of 11 benchmarks (both on the same 4), often by about a point, against baselines from providers and leaderboards rather than a controlled run, with the router well behind the best single model on two and no cost column.

References

Start with the Fugu technical report, then the Conductor paper for how a model learns to write workflows; TRINITY is the router’s predecessor.

• Sakana AI, Sakana Fugu Technical Report (2026): both designs, their training, orchestration collapse and the results table.
• Nielsen et al., Learning to Orchestrate Agents in Natural Language with the Conductor (2025): the Conductor that Fugu-Ultra builds on, a model trained to write workflows.
• Xu et al., TRINITY: An Evolved LLM Coordinator (2025): the router’s predecessor, and the source of its sep-CMA-ES training.
• Hansen and Ostermeier, Completely Derandomized Self-Adaptation in Evolution Strategies (Evolutionary Computation, 2001); Ros and Hansen, A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity (sep-CMA-ES, 2008): CMA-ES, and the separable variant the router’s second stage uses.