GPT-4.1's per-token price is lower than Claude Sonnet's on both input and output [S1]. Yet across 417 real agent tasks, GPT-4.1 cost $155 while Sonnet cost $79 [S1]. The cheaper model was nearly twice as expensive to run, and the reason has nothing to do with the model itself.

The number that shouldn't exist

IBM Research ran both models on the AppWorld Test Challenge: 417 tasks, same CodeAct agent framework, same work [S1]. Sonnet finished the batch for $79, or $0.19 per task. GPT-4.1 ran $155, or $0.37 per task [S1].

This should not have happened. GPT-4.1's token pricing is lower than Sonnet's on both input and output [S1]. By the logic most teams use to pick models (compare the price per million tokens, pick the cheaper one), GPT-4.1 should have won.

It gets stranger. Sonnet took roughly three times as many reasoning steps to finish the same tasks [S1]. More steps means more tokens. More tokens means more cost. Sonnet did more work, at a higher per-token rate, and still cost half as much.

The cache that changed everything

The explanation is caching [S1]. When an agent works through a multi-step task, it tends to reuse large chunks of context across steps [S1]. The system prompt, the task description, earlier tool outputs: all of it gets sent back to the model on every call. Caching lets the provider store that repeated context and charge less for re-reading it.

Sonnet's cache-read pricing was low enough that it benefited disproportionately from this pattern [S1]. The savings from cached context were enough to overcome both its higher base price and its longer trajectories. GPT-4.1, with less favourable cache economics, paid full freight on more of its tokens.

The sticker price told you one story. The cache hit rate told you another. Only the second one matched the bill.

From picking models to solving systems

Most routing systems treat model selection as a classification problem: look at the task, label it, assign it to a model [S1]. The IBM team argues this framing misses what actually drives cost and performance in production.

The authors stopped treating routing as classification and started treating it as an optimisation problem [S1]. In production, a router has to balance cost, latency, model specialisation, and reliability at the same time [S1]. Enterprise deployments add more constraints: compliance requirements, data residency rules, privacy limits, approved model lists [S1].

Then there is the infrastructure layer. Which hardware a model runs on, whether the cache is warm, how busy the endpoint is: these factors often dominate end-to-end response times [S1]. A model that is cheaper per token but running on a cold cache can end up slower and more expensive than a pricier model with a warm cache.

Task difficulty is often invisible at routing time [S1]. You cannot reliably know how hard a task is before the model starts working on it, which means you cannot reliably know which model will handle it most efficiently.

Even the decision of when to route matters. Routing once per task adds minimal overhead. Routing at every step introduces latency and operational complexity [S1]. More decision points mean more chances to get it wrong.

What it means

The single most important takeaway for anyone building with AI agents: per-token price is not your cost. Your cost is a function of how your workload interacts with the provider's caching, how many steps your agent takes, and what infrastructure the model is running on at the moment of the call.

A team that picks GPT-4.1 because it is cheaper per token than Sonnet may find their agent bills are double what they expected. A team that picks Sonnet because it is smarter may find it is also cheaper, for reasons that have nothing to do with intelligence.

This also means the model routing problem is harder than it looks. If you cannot predict task difficulty at routing time, and if infrastructure conditions dominate latency, and if cache behaviour determines cost, then a simple classifier that maps task type to model name will miss the things that actually matter. The IBM team's shift from classification to optimisation is an admission that the simple version of this problem does not work for real agent workloads.

Model performance is contextual, not absolute. Routing is the same story at a systems level.

What it means for business

For a two-person startup running agents on customer support tickets: your model choice should depend on whether your workload reuses context across steps. If it does, and most agent workloads do, a model with aggressive cache pricing may beat a cheaper-per-token competitor, even if that competitor takes fewer steps.

For a mid-size company with compliance constraints: the router needs to respect compliance and data residency rules before cost even enters the equation [S1]. A model that is cheapest and fastest may be disqualified on those grounds alone.

For any team building agent pipelines: the decision of how often to route matters as much as which model to route to. Routing once per task is cheap. Routing at every step gives you more control but adds latency and complexity [S1]. Most teams should start with per-task routing and only move to per-step routing when they have evidence it pays for itself.

The practical move this week: pull your last month of agent API bills. Check whether your provider is charging you for cached reads or full-price re-reads. If you cannot tell, that is the first thing to fix before optimising anything else.

What we don't know yet

The cost comparison comes from a single benchmark: the AppWorld Test Challenge, 417 tasks, one agent architecture (CodeAct) [S1]. Results may not generalise to other tasks, other agents, or other workloads. The authors themselves note that cache behaviour is highly workload-dependent [S1].

Cloud model pricing and cache-pricing structures change frequently. The specific cost hierarchy reported here may be time-bound. A provider could adjust cache-read pricing tomorrow and flip the result.

The claim that "most routing systems" treat selection as a classification problem is stated without survey data or citations [S1]. It is a characterisation by the authors, not a measured finding.

The authors are IBM Research and Hugging Face, both of which have an interest in promoting their own routing and optimisation framework. Competitive claims should be read as directional, not definitive.

Anthropic released Claude Sonnet 5 on 30 June 2026, describing it as "the most agentic Sonnet model yet" [P4]. The AppWorld results in this study predate that release. Whether Sonnet 5 changes the caching and cost dynamics described here is an open question. The next thing to watch is whether independent benchmarks reproduce these cost patterns across different agent frameworks and the latest model versions.

If you want to catch that comparison when it lands, subscribe and we will have it in your inbox before the benchmark dust settles.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.