Welcome to AIEdTalks’ Newsletter!
In today's edition:
The Model Is 20% of Your Agent
Let’s dive in.
Today’s Edition
AI TOPIC
The Model Is 20% of Your Agent
Welcome back. Most of our issues name one thing that breaks. This one names the layer they all break inside — and walks its full anatomy, part by part.
Here's what's inside:
Why LangChain took the same model from 52.8% to 66.5% without touching a weight
The analogy that makes the whole thing click for backend engineers
The twelve parts of a harness — grouped into the six that carry the weight
The arithmetic that explains why your agent dies around step ten
The seven bets every harness quietly makes
Two checks tonight to prove the model was never your problem
AI SYSTEMS: The harness is 80% of your agent
The model is the smallest part of the thing you're building.
Back in More Tools, Dumber Agent, we watched an agent get worse as we added tools. That wasn't the model. It was the layer wrapped around it — and that layer finally has a name: the harness.
Here's the proof. In early 2026, LangChain froze their coding agent's model — GPT-5.2-Codex, no weight changes, no fine-tuning — and rebuilt only the code around it: the prompt, the tools, the middleware. Terminal-Bench 2.0 went from 52.8% to 66.5%. Thirteen-plus points. Enough, that week, to move from outside the top 30 to fifth place.
(That ranking has since slid to around 27th as everyone else's harness caught up. Hold onto that. It's the real lesson, and it's at the end.)
Same model. Different wrapper. Twenty-plus spots. LangChain's Vivek Trivedy said it in one line: if you're not the model, you're the harness.
You didn't build an agent
Here's the distinction that trips people up. The agent is the behavior you see — goal-directed, tool-using, self-correcting. The harness is the machinery producing it. When someone says "I built an agent," they built a harness and aimed it at a model. Anthropic says it plainly: the SDK is "the agent harness that powers Claude Code." OpenAI's Codex team uses the words "agent" and "harness" interchangeably for the same thing — everything that isn't the model.
The analogy that makes it click
You already have the right mental model. You just haven't pointed it at agents yet.
A raw LLM is a CPU. Fast, powerful, helpless alone. No RAM, no disk, no I/O. It can't remember the last step, run a command, or read a file. Every one of those is something you bolt on.
The harness is the operating system:
The context window is RAM — fast, and painfully small.
Your databases and files are the disk — large, slow, loaded on demand.
Your tools are the device drivers — how the CPU touches the world.
The harness is the OS scheduling all of it.
The framing is Beren Millidge's, from a 2023 essay: we reinvented the Von Neumann architecture. You would never judge a CPU by running it with no RAM, no drivers, and no OS. That is precisely what "the model isn't good enough" usually means.
Three levels, one system
There are three concentric rings of engineering around the model, and most teams only see the inner one:
Prompt engineering — the words you send.
Context engineering — what the model sees, and when.
Harness engineering — both of those, plus orchestration, state, error recovery, verification, and safety.
The harness is not a wrapper around a prompt. It is the whole system that makes autonomous behavior possible.
The anatomy: twelve parts, six that carry the weight
Practitioners now count about twelve components in a production harness. Six of them are load-bearing. You've met all six already in past issues — we just never lined them up.
The orchestration loop. The heartbeat: assemble prompt, call the model, run any tool calls, feed results back, repeat. Anthropic calls their runtime a "dumb loop" — all the intelligence lives in the model, the loop just manages turns.
Tools. The agent's hands: schemas the model can see, executed in a sandbox, results fed back as observations. More is not better. (We went deep in More Tools, Dumber Agent.)
Memory. Short-term is the conversation; long-term persists across sessions via files like CLAUDE.md. One rule matters most: the agent treats memory as a hint and verifies against real state before acting. (See Cross-Session Memory.)
Context management. Where agents fail silently. Performance drops 30%+ when the key fact lands mid-window — "context rot." The fixes are compaction, hiding old tool outputs, just-in-time retrieval, and subagents that explore widely but return a 1–2k-token summary. (See Prompt Caching.)
Error handling. The math section below. Catch failures, return them to the model as messages it can recover from, and cap retries — Stripe's production harness stops at two. (See Circuit Breaker.)
Verification. What separates a demo from a product: tests and linters, screenshots for UI, or an LLM-as-judge. Boris Cherny, who built Claude Code, measured it — give the model a way to check its own work and quality jumps two to three times.
The other six — prompt construction, output parsing, state management, guardrails, subagent orchestration, and the model-calling wrapper — matter, but they're plumbing around these six.
Why it dies around step ten
Here's the number that should bother you. Take a ten-step task. Give every step a 99% success rate — better than most tool calls manage in production.
0.99 ** 10 = 0.904
A chain of near-perfect steps is a 90% coin-ish flip end to end. Stretch it to twenty steps and you drop under 82%. This is the oldest lesson in distributed systems wearing a new hat: reliability compounds downward. Your demo had three steps. Production has thirty. The model didn't get dumber — the chain got longer, and nothing in your harness was catching failures before they compounded.
The loop in motion
One cycle, start to finish: the harness assembles the input (system prompt + tools + memory + history + your message, important bits at the beginning and end because the middle gets lost). The model emits text, tool calls, or both. No tool calls means it's done. Tool calls get validated, permission-checked, and run — reads in parallel, writes serially. Results are packaged back as messages; errors are handed back so the model can self-correct. Context is appended, compacted if it's getting full, and the loop repeats until the model stops, a turn or token budget is hit, or a guardrail trips. For tasks longer than one context window, the filesystem becomes the memory: write progress to disk, re-read it next session, continue.
How the labs actually build it
Same pattern, different bets. Anthropic keeps the loop thin and deletes harness code as models get smarter — planning steps that a new model can do on its own get removed. OpenAI's Codex shares one harness across CLI, IDE, and web, which is why the same model "feels better on Codex surfaces than a generic chat window." LangGraph makes the loop an explicit state graph you can inspect and resume. Thin harness or thick graph is a real choice, not a default.
The seven bets every harness makes
Every harness architect answers the same seven questions, whether they notice or not:
Single-agent or multi-agent — both Anthropic and OpenAI say max out one agent first; split only past ~10 overlapping tools.
ReAct or plan-then-execute — planning ahead can cut latency hard; one approach reports a 3.6x speedup.
Context strategy — clearing, summarizing, masking, note-taking, delegation. One method cut tokens 26–54% while keeping 95%+ accuracy.
Verification design — deterministic checks (tests) versus a judge model (semantics).
Permission and safety — auto-approve for speed, or gate each action for safety.
Tool scoping — fewer tools, better results. Vercel cut 80% of v0's tools and improved it; Claude Code lazy-loads for a 95% context reduction.
Harness thickness — how much logic lives in your code versus the model. This one keeps shifting toward the model.
Find out if you have this problem tonight
Check one — name the layer. Pull your last five agent failures. For each, write down which part owned it: loop, tools, memory, context, error handling, verification. If you keep writing "the model was dumb," look again. It's almost always one of the six.
Check two — the frozen-model test. Take one failing task. Change nothing about the model. Add exactly one thing: a check that re-verifies output against the spec before the agent is allowed to stop.
python
def verify_before_exit(task_spec, agent_output):
"""Block the agent from declaring 'done' until its
own output actually passes the spec."""
checks = run_spec_checks(task_spec, agent_output) # {check_name: bool}
if not all(checks.values()):
failed = [name for name, ok in checks.items() if not ok]
return f"Not done. Still failing: {failed}. Keep going."
return None # passed — allow exitIf that one hook fixes the task, you just settled the argument on your own stack.
The harness is the product
Remember the agent that slid from 5th to 27th without a single line changing? Nobody broke it. Everyone else just built a better OS around the same CPU.
Scaffolding is temporary by design — it comes down as the building stands. Same here: the strongest teams delete harness code as models improve. Manus rebuilt theirs five times in six months, each pass removing complexity, not adding it. Steal their test for your own design: if a more powerful model makes your agent better without new harness complexity, your harness is sound. If you keep bolting on scaffolding to hold it together, you're building on sand.
The model is the commodity. Everyone rents the same one. The harness is the part that's yours.
So stop waiting for the next model to save your agent. It won't — it helps everyone equally, which means it helps no one relatively. The gap between your demo and your production system was never the model.
Build the harness, or keep blaming the model. There is no third option.
👋 That’s All Folks!
Before you go, just a few public service announcements:
See you soon,
AIEdTalks’ Newsletter Team
