Saurabh Agrawal, CTO: Software engineering at DVT. (Image: AI-enhanced)
The first question most technology leaders ask about enterprise AI is simple: what will it cost? The problem is that the answer many receive is incomplete.
A provider's rate per million tokens is often treated as the starting point for budgeting. In reality, it is only a price point, not a budget. The eventual cost of an AI service depends on the model's tokeniser, how much information accompanies each request, the length of the response, how often the system retries and the infrastructure supporting it.
A design that appears inexpensive during a pilot can become costly when it starts handling thousands or millions of requests.
Tokenomics helps leaders understand where those costs come from. It shows how design choices translate into costs, and how seemingly small technical decisions can affect performance, risk and return on investment.
A token is the unit a model uses to process text, code or another input. It is not always a complete word, and different models can break the same content into very different token counts.
During a recent DVT Insights webinar, I ran the same 41-character sentence through several models' tokenisers. Depending on the model, it was broken into anywhere between eight and 18 tokens. The sentence never changed. The processing cost did.
For illustration, if the same workload moves from 1 million tokens to 1.3 million while the rate stays the same, the input cost rises by 30%. Providers may introduce new tokenisers when releasing or updating models, so the bill can change even if the published rate stays the same.
Choose a model by testing it on the work it will actually do. Compare the cost of a usable result, including retries, response time and human review. A low token price offers little value if the output needs repeated correction, while a premium model is an unnecessary expense if a smaller one can meet the same requirements.
The user's prompt is often only a small part of what the model receives. An enterprise application may also send system instructions, conversation history, source code, retrieved documents, files and descriptions of connected tools. Every one of these consumes tokens.
A monthly invoice shows total model spend. Understanding what drives that spend requires tracking how much context each request carries.
If a model only needs three paragraphs from a document, sending the entire file adds cost to every request. If a task requires one tool, sending descriptions of every connected tool has the same effect.
More context does not necessarily produce a better answer. Irrelevant material can slow responses and make useful information harder for the model to locate.
Context should be built for the task, not treated as a place to put everything. Simple controls such as limiting conversation history, retrieving only relevant content and enabling only the tools required for a task can make a substantial difference.
A unit test, meeting summary, customer response and architecture review do not require the same level of reasoning. Sending all four to the largest available model may be convenient, but it is rarely economical.
Model routing allows organisations to send routine work to a smaller or specialised model while reserving premium models for tasks that genuinely justify the additional cost. The policy should account for complexity, business risk, response time and the required level of quality.
This is particularly important as providers release new models and change default behaviours. Organisations should define which workloads can use which model classes rather than leaving those decisions to individual users.
Open-weight and self-hosted models may also form part of the strategy. However, they are not free. Hardware acquisition, depreciation, power, cooling, security, maintenance and engineering support all need to be included in the calculation.
Not every request needs immediate processing, and not every prompt needs to be generated from scratch.
Prompt caching can reduce costs by re-using recurring instructions or inputs. Batch processing can lower costs for workloads such as document classification, bulk summarisation and report generation.
For example, classifying thousands of support tickets overnight does not require the same response time as a customer-facing chatbot. The architecture should reflect the business requirement rather than treating every request as immediate and unique.
Once AI is embedded across applications and teams, leaders need visibility into which systems are calling which models, how many tokens they consume and where failures or retries occur.
A month-end invoice tells you what you spent. It does not tell you why. Therefore observability is central to tokenomics. Cost should be tracked by application, model and use case, then linked back to business outcomes.
If an AI tool helps resolve a software defect, leaders should know what the request cost, how long it took and whether it improved delivery speed or quality.
A lower token count can still represent poor value if the answer is wrong. A higher cost may be justified if it improves an important outcome. Without that visibility, organisations manage spend rather than value.
AI governance usually focuses on privacy, security, compliance and responsible use. It should also cover model selection, context management, routing policies and spend.
That means approving models for defined workloads, setting context and output limits for high-volume applications, agreeing cost thresholds and monitoring changes in tokeniser behaviour. It also means deciding who may approve a move to a more expensive model and what evidence is required to support that decision.
Every production AI service should have clear ownership, defined performance expectations, cost thresholds and an agreed process for changing models or prompts.
Without these controls, organisations often discover problems only after costs rise or performance deteriorates. Token counts are not the outcome. The real measure of success is whether AI improves delivery speed, software quality, decision-making or customer service at a cost the organisation understands and accepts.
Technology leaders who can trace AI spend back to a design decision, a workload and a business outcome can make better decisions about where to invest and where to reduce waste. Understanding those costs gives organisations a firmer basis for scaling AI.
Watch the DVT Insights webinar, LLM tokenomics: The economics of tokens, on DVT’s YouTube channel for the full discussion on token usage, model selection, architecture and governance.