Tokens: the invisible currency powering the generative AI economy

Have you ever used Claude Code and suddenly received a message telling you that you had exhausted your message quota and needed to upgrade your subscription or top up your account? Rest assured, you are not the only one to encounter this kind of alert. It appears when you have apparently used up all the tokens at your disposal.

In this module, you will learn what a token is and how tokens directly affect the cost and limits of these services.

What is a token?

When you type a message intended for an AI, the tool does not read your words as you write them. That is not part of its capabilities. Instead, it perceives the message as a sequence of small text fragments called "tokens."

For example, the word "learning" can be split into two tokens: "learn" and "ing." A short, common word like "cat" usually constitutes a single token. A long or rare word like "tokenisation" may be broken into three or four pieces.

In English, a token is roughly equivalent to three-quarters of a word. A typical 10-word sentence contains around 13 to 15 tokens. Depending on the language, the process of splitting the sentence varies, as is the case in French.

How do providers make money from tokens?

The business model of all major AI providers — Anthropic, OpenAI, Google, Mistral — is based on token consumption.

What many people do not realize is that output tokens also factor into the bill, and they cost significantly more than input tokens — generally between 3 and 8 times more depending on the model. The reason is simple: generating a token is far more computationally demanding than reading one.

The hidden cost for businesses

Assessing the total cost of a generative AI product is considerably more complex, because tokens accumulate in ways that are easy to underestimate. A simple interaction that, from the user's perspective, looks like a two-line exchange may, behind the scenes, constitute a context of 20,000 or 30,000 tokens.

Beyond tokens: what comes next?

Tokens are something like the invisible currency of AI: most users never think about them, yet they determine what can be built, how much it costs, and how far a model can reason before hitting its limits.

For developers, understanding tokens means writing better prompts, choosing the right model for the task, and designing systems that do not waste context.

In conclusion, the companies that will use generative AI most effectively are not necessarily those with access to the best models.