Welcome to AIEdTalks’ Newsletter!

In today's edition:

Let’s dive in.

Last issue we routed tools — eight sharp options instead of eighty blurry ones. This issue we go one layer up: routing the models themselves.

Here's a story that will sound familiar. An engineer ships an AI assistant. One model wired into everything — the best one he can afford, because why risk quality? Intent detection, summarization, extraction, the actual hard reasoning. One integration. It just works.

Then he does something most teams never do: he logs every LLM call with a task label and reads the logs.

Roughly 70% of his calls were simple — classify this, extract that, shorten this. Work a model a tenth the price handles fine. Defaulting everything to the frontier model was about 38% of his wasted spend. He added a small classifier in front — simple requests go to the cheap model, hard ones to the expensive one — and his monthly bill dropped from ~$800 to under $160.

One writeup, one bill — treat the exact dollars as illustrative. But the pattern is everywhere, and his one-liner is the whole issue: using a frontier model for intent classification is like hiring a neurosurgeon to take your blood pressure.

You already know this pattern. You'd never run every query on your biggest database instance. You'd never send static assets through your application servers. You put a load balancer in front and route by what the request needs. Model routing is exactly that — a load balancer with opinions.

Model providers sell tiers, the way clouds sell instance classes. Small, fast, cheap (Haiku, GPT-mini). Middle, balanced (Sonnet). Large, slow, expensive (Opus). The price spread between the cheapest and the most capable tier is roughly 100×. That spread is the entire opportunity.

A model router is a small decision layer in front of your LLM calls. Per request, it answers one question: what's the cheapest model that can do this well? A database engineer would recognize it instantly — it's a cost-based query planner. The optimizer doesn't run every query on the most expensive plan. It estimates and picks the cheapest adequate one.

The research backs the intuition. RouteLLM (Berkeley/LMSYS, peer-reviewed) routed between GPT-4 and a model 100× cheaper and hit 95% of GPT-4 quality while sending only 14–26% of calls to GPT-4 — up to 75% cheaper on their benchmark. Benchmark numbers, not your numbers. But proof the technique works.

And this is where multi-agent systems raise the stakes. Anthropic's research agent uses an expensive model as the orchestrator and cheaper models as workers. It beat a single top-tier agent by 90.2% on their internal eval — but multi-agent systems burn roughly 15× the tokens of a chat. Every sub-agent you spawn is a routing decision. Tier the models and the 15× is survivable. Send everything to the frontier model and the fan-out sends your bill with it.

The silent wrong answer. A router misroutes in two directions, and they are not symmetric. Simple work sent to the expensive model just wastes money — the safe failure. Hard work sent to the cheap model ships a confident, wrong answer. No error, no timeout, nothing in your logs. The engineer from our story hit this: his cheap model handled 94% of intent calls, but on the ambiguous 6% — "can you help me fix this?" — it picked a branch and confidently sent users down the wrong one, where the bigger model would have asked a clarifying question. He only caught it by tracking how often users had to re-explain themselves. OpenAI hit the public version at GPT-5's launch: the built-in router sent too many hard queries to the small model, and to the world, "GPT-5" just felt dumb. When routing works it's invisible. When it fails, it looks like your whole product broke.

The cascade that costs more than no cascade. The popular pattern is try-cheap-first: call the small model, check the answer, escalate if the check fails. Feels free. It isn't. Your blended cost per request is:

where f is the fraction the cheap model resolves. When f is high, you win big. But if a provider update quietly breaks your cheap model's output format and your check starts failing, f collapses — and now you pay for the cheap attempt and the frontier call on nearly every request. One 2026 benchmark measured a cascade escalating two-thirds of queries: 1.66× the cost and lower accuracy than just always calling the best model. Escalation rate is a metric. If nobody's watching it, it climbs for days before the bill tells you.

Check 1 — the five-minute spend audit. Group last month's calls by task type and model. You're looking for one tell: high-volume, simple tasks running on your most expensive model.

If classification or extraction shows up next to your frontier model in the top rows, you found the neurosurgeon. If your logs don't have task_type and model per call — that's finding number one. Add them tonight; the audit waits a month.

Check 2 — the 200-request replay. Never trust a vendor's savings percentage. Sample ~200 real production requests, run each through your current model and a cheaper candidate, blind-score both.

Don't just read the average — read the failures. Where the cheap model breaks tells you exactly which slice of traffic is safe to route down, and which slice pays the neurosurgeon for good reason.

If more than half your traffic can safely ride the cheap tier, build the router. If it can't, an ambitious routing project saves you almost nothing — skip it and ship features.

Route your models, or your bill routes your roadmap. There is no third option.

Ran the spend audit? Reply with your most expensive (task, model) pair — the best confessions go in a future issue.

Before you go, just a few public service announcements:

See you soon,

AIEdTalks’ Newsletter Team