Not everyone owns a Ferrari, but I’m sure those who do rarely use it to pick their children up from school. It is less energy-efficient, not as safe or spacious and may garner unwanted attention. The analogy is simple: in AI, you do not always need the most powerful tool for everyday tasks.
It is a trap many businesses have fallen into since the first wave of AI, when early adopters were trying to prove they could access the cutting edge of the technology. Everyone wanted to have the most powerful AI model, the most sought-after accelerators and the fastest route from the pilot stage to implementation.
But as AI moves from pilot into production, new costs are emerging, largely associated with usage. A proof-of-concept used by a handful of people can look cheap, but running the same application thousands of times a day, or by agents calling upon multiple models for one task, is an entirely different proposition.
The mistake is not owning a “Ferrari”, but using the same model, depending on the same hardware choices or taking the same deployment path for every project. That is where the next phase of enterprise AI is getting interesting, with new efficiencies coming from matching each workload to the most suitable model and accelerator, rather than defaulting to your biggest “engine”.
Every model has a different role to play
There is a popular idea among some enterprises that AI is a ranking exercise, in which you should standardize as much as possible on the smartest model available. In theory it sounds tidy, but it ignores the complex differences between each AI task and workload.
For instance, a customer service copilot needs to respond quickly enough that the user never notices the machinery behind it. A large document summarization job can wait longer, with the user expecting the AI to “think” much more carefully. Government applications of AI are likely to instead prioritize sovereignty and ownership, making where and how workloads run just as important as performance.
This logic applies from the size of the model to the infrastructure underneath it. Smaller models can often handle routine classification, extraction and routing tasks quicker and at lower cost, while mid-sized models support a great deal of everyday reasoning work. The largest, most powerful models are important but may be used more selectively, when their capability and cost are justified – for example, in complex reasoning, deep strategic planning and high-stakes tasks where accuracy and nuance are critical.
Equally, to maximize token efficiency, enterprises should consider how different workloads behave and why they do not all require the same type of accelerator. With a diversity of silicon to draw on, different combinations of models and hardware can be selected to suit different requirements, whether the priority is speed, throughput, cost or control.
As such, rather than fitting every workload around an assumed preference, enterprise teams should start with the desired outcome. A retailer running a customer service assistant will have very different priorities from a government department handling sensitive information.
The former is likely to prioritize response time, user experience and cost at scale, while the latter places greater weight on sovereignty, security and where the workload runs. The infrastructure should leave enough choice to match different workloads and changing requirements without being locked into one route.
Building on a diversity of silicon
If an organization invests heavily in one type of accelerator, it can be tempting to run every workload through it. This is where the wrong assumptions, made at enterprise scale, can increase costs rapidly. Using the same piece of hardware for every task can mean overpaying for performance with minimal or no benefit to the user.
Low-latency applications need hardware optimized for responsiveness, while high-volume workloads may benefit more from throughput and cost efficiency. Running both on the same infrastructure can mean paying for performance that one of them never uses.
In most cases, building on a diversity of silicon – heterogeneous accelerators that can be aligned to each workload – will enable enterprises to make smarter infrastructure decisions. In some scenarios, organizations may even prefer to combine hardware from different providers that complement each other. NVIDIA GPUs will make sense for many enterprise scenarios, but choice matters.
Being able to select across providers such as AMD, Qualcomm, Microsoft, Cerebras and others lets enterprises align infrastructure to their specific needs and improve efficiency. Cerebras, for example, has run OpenAI's gpt-oss-120b at around 3,000 tokens per second – a level of speed that is especially valuable for applications where response time is critical.
The deployment environment also matters, particularly in large-scale enterprises or at the government level, where a great deal of sensitive information needs to stay within a specific jurisdiction. In Europe, there is strong demand for infrastructure developed in-market and operated around local requirements.
For enterprises operating across markets, that means infrastructure choices cannot be separated from questions of data residency and local control. Increasingly, the answer is infrastructure that is sovereign enabled – with in-country accelerators, consumption-based pricing and no upfront capital commitment – rather than a single hardware decision made once and lived with for years. Being locked into one route leaves little room to respond when those requirements change.
Understanding AI tokens and efficiency
Another consideration is that AI spending behaves differently from traditional enterprise software, where a license has a relatively predictable cost. Businesses will face a larger AI bill when an agent takes more steps, or a powerful model is pulled into more tasks than necessary, which makes usage itself a poor proxy for value.
Enterprises are already starting to rethink how freely employees can consume tokens, particularly in coding roles. Uber, for example, reportedly used its entire 2026 AI budget in four months, largely on Claude Code. The lesson from this type of example is that giving more people more tokens does not necessarily equate to greater value.
Agentic AI provides another example. Unlike an employee who occasionally asks a routine standalone question, an AI agent can work through a long sequence of steps, read files, generate code, test it and review the output for mistakes. Agentic AI, especially in coding, can be uniquely expensive compared to reasoning and chat, with highly variable token usage where increased consumption does not always improve accuracy.
As counting tokens does not provide the complete picture, enterprises should consider measuring success on outcomes rather than on performance or capability alone. It can be useful to use faster AI models, more powerful hardware or longer-running agents, but only when the outcome justifies what they consume.
For a customer service agent that could be the cost of resolving a case or, for an employee copilot, whether human employees are making faster or more complete progress on meaningful work.
However, the aim should not be to cut token use altogether. IT leaders should seek to understand exactly what they’re paying for and whether it has a clear business case. That is the definition of AI efficiency we should be moving towards. Rather than seeking maximum usage or minimum cost, CIOs and their teams should ensure they return enough value from each workload to justify the investment that goes into running it.
Saving the Ferrari for the racetrack
There will always be AI workloads that require the largest, most powerful model, the fastest accelerator or the most expensive deployment option. The point is to stop making those choices by default. The next phase of widespread AI production requires a more deliberate approach to every workload.
Start with the task, choose a model that’s capable enough to do it well, run it on the accelerator suited to its performance needs, and make sure the deployment environment meets the security and sovereignty requirements around it. Then measure whether the result is worth what it costs.
This approach may be less headline-grabbing, but it is what will separate resilient, production-scale enterprise and government deployments from experimentation. As AI becomes part of everyday workflows, efficiency will increasingly stem from making smarter choices about where each capability is needed – and from having the diversity of models, silicon and deployment options to act on them.
We've featured the best AI tool.
This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.
The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit