Data strategy
Semantic drift and semantic integrity: Stewarding meaning in the age of AI
Japanese AI company Sakana AI recently released Fugu, a multi-agent and dynamic model routing system delivered as a single model API. In simple terms, it provides a clean and straightforward way to leverage a range of different models, by directing queries towards the model it determines to be most appropriate and effective. It consists of a lightweight learned coordinator sitting behind an e OpenAI-compatible endpoint, which assembles a pool of frontier models, assigns them 'Thinker', 'Worker' and 'Verifier' roles, orchestrating them across turns.
The research behind it (TRINITY and the Conductor, both presented at ICLR 2026) is genuinely interesting but there's a question that deserves consideration: is this where model routing should live?
The benchmark gains are real, but they're not evenly distributed
Its performance against benchmarks are impressive. Fugu Ultra posts 93.2 on LiveCodeBench against high 80s for the frontier models it routes between. It edges out Opus 4.8 on SWE-bench Pro. The base Fugu model, meanwhile, trails Opus by ten points on the same benchmark.
Those results suggest Sakana may have solved a meaningful chunk of the routing problem, at least for coding, where task difficulty is legible from the prompt. Frontier models fail on different questions, but a router that picks the right model per task aims to harvest the union of their strengths.
However, the spread between benchmarks also tells us the picture isn’t clean. Even within a single domain, subdomains diverge. Short-horizon competitive coding routes beautifully, but Longer-horizon repository work gets muddier. In other words, despite impressive results, this shouldn’t be viewed as a clear-cut victory for Fugu.
Yes, routing can improve performance
‘Stand up economist’ Yoram Bauman does a wonderful bit where he translates Mankiw's ten principles of economics (watch it; it holds up). His favorite target is the principle that trade can make everyone better off. As he points out, "can" is a word that carries no weight at all. Trade can make everyone better off. But trade can also make everyone worse off. You have committed to nothing.
Sakana's write-up concludes that orchestrating multiple strong models can outperform any individual frontier model. It should be read with Bauman's joke ringing in your ear. Yes, routing can improve performance, but routing can also hand your repository refactor to a model that’s ten points worse at it, which is exactly what their own SWE-bench Pro numbers show for base Fugu.
Both statements are true; neither ultimately tells you what happens to your workload. The only claim that would mean anything is ‘routing significantly improves performance on tasks like yours’, and nobody can make that claim except you, with your evals, on your tasks.
Credit where it’s due
There's one particular thing Sakana got right. Fugu optimizes for quality, not cost.
Most semantic routing pitches amount to ‘you don't need that expensive model’, a vendor deciding your task deserves the cheap one. That framing has always struck me as backwards; if a task isn’t valuable enough for the good model, it’s probably not valuable enough to automate at all. Doing it badly with a dumb model just wastes money more slowly. Fugu at least aims the routing at getting you a better answer.
The pricing model is sensible too. There are no stacked fees, just a single blended rate based on the top model in your pool.
The routing is in the wrong place
Here’s my main problem, though: Fugu moves routing above the application, into a layer the application team neither controls nor observes.
The team building an agentic system holds the domain context. They know which actions are destructive, which subtasks tolerate a fast model, where a hard-coded answer beats any model call. A platform sitting one abstraction higher cannot know any of this. Platform teams that think they’re smarter than their application teams are usually wrong. (I say this as someone with the platform-team scars to prove it.)
Routing should be exposed as a capability and put in the hands of the team building the system. It shouldn’t be automated over their heads. Frameworks are already converging on this. Modern agent harnesses let you bundle model choice and model settings into the capability itself, chosen by the people who understand the task. That’s routing done at the right altitude.
The whole trick in a dozen lines
Part of what irritates me about routing-as-a-product is how little there is to productise. Pydantic AI v2 makes routing a property of the capability, the composable unit that bundles instructions, tools, hooks and model settings. Mark the bundle as deferred and the agent routes itself.
```python
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
provider = OpenAIProvider(base_url="http://localhost:1234/v1", api_key="lm-studio")
general_model = OpenAIChatModel("qwen/qwen3.6-35b-a3b", provider=provider)
coder_model = OpenAIChatModel("qwen2.5-coder-1.5b-instruct-mlx", provider=provider)
def bens_proprietary_routing_algorithm(task: str) -> OpenAIChatModel:
return coder_model if "code" in task else general_model
agent = Agent(instructions="You are an AI assistant.")
def run(task: str) -> str:
model_chosen = bens_proprietary_routing_algorithm(task)
print(f"{model_chosen.model_name=}")
return agent.run_sync(task, model=model_chosen).output
if __name__ == "__main__":
print(run("Reply hello"))
print(run("Explain this code: def add(x, y): return x + y"))
```That router is obviously a joke; routing isn’t trivial, but the point is that it should really belong to the people who hold the domain context. Another advantage of this is that the decision is in your trace, which means you can log it, replay it, override it and eval it. You take each task your system runs, evaluate the candidate models on it and pin the one that wins. Own the judgement, the price/performance trade-off and make it observable.
Opaque by design
There’s also an FAQ entry that should give every engineer pause. Can you see which underlying models Fugu used for a given query? No, you can’t: the routing is proprietary and the information is withheld by design.
Building on non-deterministic systems is already hard. Fugu adds a second non-deterministic layer, one you cannot inspect and governed by rules you cannot see, that can change underneath you when a new frontier model gets folded into the pool a fortnight after release.
Picture debugging a production edge case where the answer to ‘what changed?’ is ‘the router picked a different model, and we won't tell you which or why’. That’s not an engineering platform, it’s a one-armed bandit.
Where this leaves us
If you’re in the exploratory phase without evals or a clear sense of which model fits, Fugu is a reasonable way to get strong answers through one endpoint. However, the moment you start evaluating a real system against real tasks, the calculus flips. Every serious agentic system ends up needing an orchestration layer it owns. So, yes, Fugu proves learned routing can work, but, as the standup economist taught us, that was never in question.
