Enterprise cloud bills are about to rise dramatically as AI moves to token-based pricing. Gone are the days of flat fees for AI access; now, every query, every prompt, every generation costs tokens, and those tokens add up fast.

What are AI tokens?

Tokens are the smallest unit of text that a large language model (LLM) processes. A token is roughly four characters or about three-quarters of a word. When you send a prompt to an AI model, it is broken into tokens, and the model also generates its response in tokens. This tokenization is invisible to users but forms the basis of billing.

Why token pricing is more expensive

During the early days of generative AI, companies offered all-you-can-eat subscriptions or cheap per-user fees. That era is over. Token pricing reveals the true cost of AI inference, which is far higher than previously subsidized. For example, a $200 monthly subscription for some models actually covered thousands of dollars in token usage. As labs and cloud providers seek profitability, they are passing those costs to customers.

Token economics basics

Token pricing is now standard across major AI providers. They charge separately for input tokens (the prompt you send) and output tokens (the model's response). Prices vary by model complexity, context window size, and whether caching or agentic loops are used. A single chat session with a large context window and many agent loops can burn through millions of tokens, costing hundreds of dollars.

Supply constraints keep prices high

Although token prices have fallen since 2023, they have recently plateaued due to hardware and power shortages. GPU supply remains constrained, and new frontier models are more expensive to run. This creates a classic Jevons paradox: as unit costs stabilize or fall slightly, overall usage explodes, leading to higher total spend. Some enterprises report monthly AI costs doubling even as token prices gradually decline.

How FinOps teams are adapting

FinOps professionals, accustomed to cloud cost management, now face a new challenge: managing AI token spend. Traditional cloud tools cannot track model-level or token-level costs. Companies like SAP have built internal frameworks to gain visibility into which models, which teams, and which use cases are driving costs. Key metrics include input-to-output token ratios, cached token ratios, and "token to spend drift" to detect cost spikes from model mix changes.

The value measurement problem

Measuring the value derived from AI tokens remains an unsolved problem. While a token has a clear cost, its business value varies enormously. One token might help a developer fix a bug in seconds, while another might be wasted on an irrelevant query. Enterprises struggle to connect token spend to revenue, productivity, or customer satisfaction. This gap makes it hard to justify AI investments and to set budgets.

New business models emerge

Vendors are experimenting with various pricing models: credit-based systems (where credits map to tokens), hybrid subscription-plus-usage, and direct pass-through of token costs. The trend is toward transparency, though many customers dislike the unpredictability. Microsoft's shift of GitHub Copilot to usage-based pricing angered many power users who relied on unlimited tokens.

Tokenomics: a new discipline

The Linux Foundation has introduced the term "tokenomics" to describe the full lifecycle of tokens from production to consumption to value creation. This includes everything from energy and capital used to mint tokens, to forecasting and optimizing consumption, to monetizing the output. Tokenomics aims to give enterprises a framework to manage AI costs holistically.

Societal impacts of token pricing

Token pricing creates a divide between those who can afford top-tier AI models and those who cannot. Within companies, some teams get the latest models while others are relegated to cheaper but less capable alternatives. This can stifle innovation as experimentation becomes expensive. There are also concerns about job displacement: workers who master AI tools become more valuable, while those without access fall behind.

As token pricing becomes the norm, enterprises must adapt their FinOps practices, invest in cost visibility tools, and develop strategies to maximize the value of every token. The era of cheap AI experiments is over; the era of careful token economy has begun.

Source: ZDNET News