Tokenmaxxing has largely died. It has dawned on enterprises that leaderboards ranking employees on raw AI usage are at odds with sanity. In its place, a new initiative focuses on model routing, where only the most demanding prompts are sent to the most expensive LLMs. While sensible at first glance, the practice has problems both incidental and fundamental.
Model routing is a simple concept: assign every prompt an implicit complexity and route it to whatever LLM is suitable for said complexity. Refactoring an entire codebase or finding multi-step vulnerabilities may be a job worthy of expensive Mythos-class models (assuming you have access), whereas “How do I turn Bluetooth on in Windows” shouldn’t even need to cost a tenth of a cent.
The concept is sound as long as you don’t look at it too closely. Many organizations have proven themselves incapable of understanding the use cases for specific LLMs over the past few years, and end users are prone to default to whatever the best model available is. That’s particularly true for anyone whose KPIs, until recently, rewarded burning tokens rather than spending them judiciously.
We’re not used to AI model choice
Treat any trending AI buzzword with a giant grain of salt. We’ve seen the demise of ‘prompt engineering’ early on in the dawn of widespread LLM use and more recently an end to tokenmaxxing, the corporate fashion of treating token consumption as a proxy for AI adoption. For the latter, it seems Uber burning through a year’s worth of AI budget in four months of Claude Code token spend earlier in 2026 was the canary in the coalmine. However, even just a month ago, signs still pointed to enterprise AI usage defaulting to the biggest model available 95 percent of the time.
With Claude Fable 5 costing twice as much per token as Anthropic’s previous state-of-the-art Opus 4.8, we hope for finance departments’ sake that this 95 percent figure will rapidly shrink. GitHub Copilot users were also in for a rude awakening as the coding tool has moved to usage-based billing. Murmurs of the typical 20/100/200 dollars per month plans being heavily subsidized have come into sharp relief as the ‘real cost’ of AI tokens is supposedly revealing itself. That ‘real cost’ narrative is a complete red herring, as it happens. Sure, the cost of a Claude Pro/Max subscription is vanishingly small if you treat the API token use equivalent as the standard to compare it to. But Anthropic’s inference margins are reportedly 70 percent (!), which can cover lots of AI buildout spending that, incidentally, they aren’t always footing the bill for. API list prices are a policy and not some kind of ground truth, meaning subscription-based usage is far less subsidized than it seems.
We don’t know the exact margins for individual models, but suffice it to say that organizations spending less on AI because of smarter model choices would reduce the financial gains for the likes of Anthropic and OpenAI. Model routing is thus at odds with their current API pricing structure. For subscription tiers, though, obfuscation of the LLM picked for each prompt may well become commonplace. If the prompts are usually table stakes anyway, why bother serving the best of the best? Users and enterprises at large aren’t used to picking LLM model sizes, and they may not catch up before the choice is taken away regardless.
Who routes?
That all assumes users and organizations are stuck picking between model family members. Indeed, Claude users may well be quite familiar with what constitutes a “Fable task” versus a “Sonnet task”. Rote boilerplate code is perfect for the Sonnet workhorse and advanced planning of your project, taking into account all the specific needs you have, may require Fable. But what if you’re looking to switch it up even more and pick Z.ai’s GLM-5.2 for some task that is not-quite-Opus-but-not-quite-Sonnet shaped? How about the tiny Qwen and Gemma models that can run locally for many, leaving you with just the electricity costs for your workstation?
The choices are endless, but the limits are well hidden. Coding harnesses each present different advantages and disadvantages. Consider Claude Code: it does allow you to integrate external LLMs, but heavily favors Claude models for practical use cases. The same is largely true for OpenAI’s Codex and Z.ai’s newfangled ZCode. Even if you did use these with third-party models, their system prompts and basic setups are simply not intended or optimized for routing to LLMs dynamically.
The problem with third-party model routers is that they do not optimize costs as much as the subscription tiers do, unless you exceed those limits by some margin. Thus, organizations are incentivized to stick with one provider and mostly forget about routing. As tokenmaxxing fades into obscurity, enterprises will ask tougher questions when negotiating with LLM providers or resellers. Then again, those folks are busy enough with the Fortune 500 as is, reaping far bigger rewards per contract than elsewhere, leaving other organizations to navigate the routing question on their own. All of the above, though, is incidental. Tooling matures, habits change, contracts get renegotiated. The fundamental problem is more technical than that, as we’ll get into.
Built-in overhead
Much has been made about how AI models have themselves improved. Far from the laughable concept of LLMs already ‘self-improving’ in the year 2026, human experts have themselves come up with ways to optimize inference speed, context windows, dynamic reasoning steps, tool calling and wrestling with kernel-level GPU code. This has led to the LLMs being systems in and of themselves, rather than merely a bunch of Generative Pre-trained Transformers scaled to trillions of parameters.
Although these advances have helped LLM providers reduce their costs and increase margins, they have unequivocally improved their usefulness too. Models are now capable of sticking closer to user preferences, recalling earlier chats, compacting ongoing conversations and keeping tiny details in massive codebases in context. These advances don’t all carry over into multi-LLM routers.
Every LLM carries the built-in overhead of reprocessing its context on each turn. After a tool call returns a result, the model effectively starts over the full conversation. Nevertheless, prompt caching, compaction and other proprietary methods are saving lots of these costs. OpenAI and Anthropic have the scale to keep recently computed tokens in a readily accessible memory format, utilizing them for any long-horizon tasks and massively reducing costs for both themselves and users. That machinery only pays off if consecutive requests land on the same model at the same provider, which is the arrangement that can actually be called a ‘subsidy’ on users in an ecosystem lock-in for either of these AI companies.
Route dynamically across models and providers, and all that efficiency is thrown away. Every switch is a cold start and features a bunch of overhead that most won’t be aware of. The disaggregated nature of model routing is therefore fundamentally disadvantaged. Even if you end up with a lower AI token spend now, don’t expect the large-scale adoption of model routing to remain unanswered. The wasted recomputation will raise costs for every provider involved, even if usage shifts towards smaller models. In other words: routing saves money per prompt while destroying the efficiencies that made prompts cheap in the first place.
Conclusion: expect another overtaking move
Anthropic and OpenAI are well ahead of the state of the art in a sense. Even though the world is currently becoming familiar with Fable 5/Mythos 5 and the GPT-5.6 variants, their creators are already onto the next step change. Expect model routing to have been well understood by both parties already. The AI ecosystem already anticipated an end to tokenmaxxing, given the highly predictable customer response to higher billing. In a way, the end was manufactured, or at least gladly accepted, by the AI providers, who knew they needed to raise margins at some point. 2026 was as good a time as any, with AI tooling becoming far more potent since Opus and GPT releases in the past 6 months or so.
Expect the model providers to deliver some path to built-in routing. Anthropic rather ungracefully drops Fable 5 users down to Opus 4.8 as a safety feature, but this behavior can and will be recast in a more PR-savvy way. With Mixture-of-Experts now a well established architecture for years, LLMs are already optimizing internally for throughput and cost savings. The next step is what could effectively be called a Mixture-of-Models even if the underlying architecture is nothing like MoE. Crucially, provider-side routing dodges the downsides of routing across providers. Dress up a move to provider-side routing nicely and it will come across as a responsible, customer-friendly response to the tokenmaxxing debacle. Whoever finds a way to formulate that well will be in the good graces of many. And as a result, vendor lock-in will at last become a lasting advantage for the frontier AI labs.
Also read: As Fable 5 returns, Anthropic wants to write the frontier AI rulebook