There is a pattern I have watched repeat for two years, and it is almost always the same script.
A team builds an agent. It breaks in production. Outputs are wrong, actions are unreliable, users lose trust. The diagnosis is instant and unanimous: the model is not smart enough. So the team upgrades to the newest frontier model, sees a marginal bump, and hits the same wall in a slightly different shape three weeks later.
This is not a model problem. It never was.
The New Stack named it directly this summer. The bottleneck for AI agents is not the model anymore. It is the context layer. And the piece describes the exact loop I keep seeing in the field. There is a pattern that repeats for two years. A team builds an agent, hits reliability problems, upgrades the model, sees marginal improvement, and hits the same reliability problems in a slightly different form. The diagnosis is always the same: the model was not smart enough. The fix is always to try a new, smarter model. The result is always the same: still broken.
Even Karpathy landed here. He noted that a large fraction of his recent token throughput was going less into manipulating code and more into manipulating knowledge. He was not running a smarter model. He was building better infrastructure.
What Actually Breaks Is Not the Weights
From the forward deployed seat, agent failures cluster into three boring, unglamorous categories. None of them are fixed by a better model.
Stale retrieval. Your vector store is behind reality. The agent confidently operates on a version of your business that stopped being true a month ago. As one practitioner put it, a model with a larger context window still wastes tokens if the retrieval mechanism is read every file and hope, and a model trained on more data still hallucinates when the library it needs shipped a breaking change after training.
Missing tool results. The agent needed the account record, the tool call timed out or returned nothing, and rather than stopping, it invented a plausible answer. The model did exactly what it was designed to do. Your plumbing failed.
Context bloat. The window fills with boilerplate, redundant chunks, and abandoned reasoning before the real task even loads. This is measurable, not vibes. Dex Horthy, who wrote the 12 Factor Agents guidelines, identified the dumb zone where performance degrades once more than roughly forty percent of the context window is consumed. And retrieval that just stuffs the window makes it worse: similar results are often redundant, so if your top ten chunks come from the same document section you have wasted nine slots on near-duplicate information, and the model gets one perspective repeated ten times instead of ten diverse ones.
Bigger context windows are not the escape hatch either. Chroma Research tested this and the results are brutal. Their Context Rot study tested eighteen frontier models on simple retrieval tasks and found every one degraded as input length grew. The decline was not gradual: models hit accuracy cliffs well before the rated limit, with a 200K window showing serious loss by 50K tokens.
The CODN Nobody Puts on the Invoice
Here is where the Cost of Doing Nothing lives, and it is not zero.
Every degraded output is rework. Every hallucinated action is a trust withdrawal you cannot easily deposit back. And the token math compounds. Engineering leaders at large companies keep describing the same trap: the problems compound at enterprise scale, they deployed agents, usage exploded, and now they are staring at inference bills growing faster than the value the agents provide.
That is the trap. Upgrading the model when your context is broken does not just fail to fix the problem. It fails more expensively, because you are paying frontier prices to feed garbage into a bigger window.
Vendors have noticed the money is here now. Amazon CloudWatch shipped Coding Agent Insights to show engineering leaders how AI coding agents perform across an organization, and vendors do not build monitoring products for problems that are not yet costing teams real money. Context rot became a line item.
Where the Leverage Actually Sits
Stop shopping for models. Start engineering what the model sees. The discipline already has a name and a shape.
Treat context as a system, not a prompt. Context engineering is the discipline of filling the window with the right information: instructions, retrieved knowledge, memory, tool descriptions, and prior outputs, all structured so the model can use them. Lance Martin formalized the strategies as write, select, compress, and isolate.
Practically, that means: version your retrieval so freshness is a guarantee, not a hope. Instrument tool calls so a failed fetch stops the agent instead of triggering a confident guess. Budget the window ruthlessly and stay out of the dumb zone. Rank and dedupe what you inject instead of dumping the top ten.
The New Stack is right that this is the hard part. It is harder to build than switching models and harder to maintain. It is also the difference between an agent operating in your organization’s actual reality and one confidently operating in a hallucinated version of it.
The next capability jump for your agents is not on a model provider’s roadmap. It is in the retrieval, the tools, and the token budget you own today. That is where the leverage sits. Go build it.