The router is the architecture

Almost everyone builds the monolith first, and it is the right way to start. One engine, one prompt, every tool attached, pointed at the whole corpus. It handles the first few query types convincingly and the demo goes well.

Then it acquires the fourth capability, and the fifth, and something starts to go wrong that is hard to name. Nothing is broken. It is just that every improvement now costs something elsewhere. Tightening the prompt so it stops being chatty about billing makes it terse in a complaint. Widening retrieval so it can answer setup questions makes it confidently wrong about returns. The team develops a superstition about touching the prompt at all.

The problem is not that the engine is not clever enough. It is that you have asked it to optimise for several different definitions of a good answer at once.

The weak argument, and the durable one

The usual case for decomposition is that a specialist outperforms a generalist on its niche. It is the weakest version of the argument and I would not build on it, because it is really a claim about model capability, and claims about model capability expire. A great deal of routing scaffolding built two or three years ago was compensating for weak instruction-following and small context windows. On a current model much of it is pure overhead: added latency, added failure surface, and a hop that would have been unnecessary in one pass.

The argument that survives is about objectives. Good does not mean the same thing across intents, and it is not close.

Intent	What "good" actually means	What failure looks like
Billing query	Precision, traceability to a source, no interpretation. A refusal is acceptable.	A plausible number. Worse than no answer, because it will be acted on.
How do I	Coverage and completeness. A partial answer is genuinely useful.	Silence when the content exists.
Complaint	Tone, acknowledgement, speed to a human with authority.	An efficient, correct answer that reads as dismissal.
Bereavement notification	Suppression of everything automated, and immediate handoff.	Any autonomous response at all.
Buying	Relevance and steer, with the constraints stated honestly.	Recommending something that will be returned.

One engine has one temperature, one retrieval profile, one guardrail set, one escalation threshold and one tolerance for saying nothing. Whatever you tune it to, it is wrong for most of that table. That is not a capability problem you can wait out.

The second durable reason is that you can only evaluate and govern what you have partitioned. A monolith's test set is unbounded, so in practice it is never really measured; you get a vibe and a handful of favourite queries. Split by intent and each route has a finite, buildable eval set, its own accuracy bar, and its own answer to the question of whether it is allowed to act autonomously at all. That last part is what makes oversight tractable rather than aspirational: you can say which routes a human has to stand behind, and mean it.

The problem with the fix

Here is where I would push back on my own recommendation, because the failure mode is real and it is not obvious.

The router becomes the monolith. You have taken every architectural decision and concentrated it into one classification, and that classification runs on the weakest signal in the entire system: the customer's opening line, before any clarification, before any context is gathered. "It is not working." "This is wrong." "Help." The router is being asked to make the highest-leverage decision in the architecture from the least information anyone will have at any point in the conversation. It is the same weak-query problem as a retrieval gate, moved somewhere it does more damage.

And a misroute is worse than no route. The monolith at least had the whole picture and could wander into the right answer. A specialist that has been handed the wrong frame answers confidently and coherently within it, which is much harder for anyone to spot.

Intents are not single-label, and they do not hold still. "I want to send this back and cancel the subscription, and frankly I have had enough" is three intents and a sentiment, and single-label routing forces a lie. Meanwhile the taxonomy drifts as the business changes, and every one of these systems grows an "other" bucket that quietly becomes the largest route.

Every handoff is a chance to lose the thread. Splitting one conversation across several agents means context has to travel between them, which is exactly the whisper game, reintroduced at the architecture level. Decomposition does not avoid that problem. It creates more places for it to happen.

And it is easy to split too far. Six routes where three of them carry five per cent of volume each is three agents you will never gather enough traffic to evaluate, each with its own prompt, eval set, regression suite and deployment path. You have bought all of the maintenance cost of specialisation and none of the measurability that justified it.

What actually makes it work

The finding holds. It is just conditional, and these are the conditions.

Route on context, not on the utterance alone. You usually know their product, their open orders, their last contact, whether there is a complaint in flight. That context is often a stronger signal than anything they typed, and it is already available if the canonical view reaches the decision point. Routing on three words when you are holding the customer's whole history is a self-inflicted wound.
Treat the route as a hypothesis, not a verdict. A specialist that discovers it has the wrong frame must be able to hand back rather than press on. One-shot irreversible routing is what turns a classification error into a bad outcome.
Give low confidence somewhere to go. When the router is unsure, the options are to ask one clarifying question, or to fall through to a generalist, or to reach a human. What it must not do is pick the best of five bad labels and proceed as though it were certain. Forcing a label is how you convert "we did not know" into "we were wrong".
Share state canonically, so the handoff carries everything. Agents should read and write one persisted view of the conversation rather than passing summaries to each other. A summary is where the customer's stated constraint goes to die.
Split by risk as well as by topic. Intent is the obvious axis and it is not always the useful one. The boundary that earns its keep most reliably is between what can answer and what can act: retrieval and advice on one side, anything that spends money, changes an account or sends something irreversible on the other. That line is where oversight and audit belong, and it rarely follows the topic taxonomy.
Only split where you can measure. A route needs enough volume to build a real eval set. Below that, it belongs inside a broader route with a documented reason, not as an agent nobody can tell is working.

When to split, in three questions

Does "a good answer" mean something materially different here, in tone, risk tolerance, or what an acceptable failure looks like? Would improving this path risk regressing another one? Is there enough volume to build an evaluation set for it? Three yeses and it should be its own route. One or none and you are adding a hop, a handoff and a maintenance burden to buy an org chart.

The short version

Decompose, but understand what you are actually buying. Not a cleverer answer: a system where each path has its own definition of good, its own bar, and its own answer to whether it is allowed to act alone. That is worth real architectural cost, and it is the version of this that will still be right after the next model release.

Then spend your attention on the router, because it has become the most important component you have, and it is running on the least information in the building. Feed it context rather than keywords, let it be unsure, and let the decision be revisited. A confident classifier over a vague opening line is not an architecture. It is a coin toss with good manners.