Just enough compute

How times have changed. It was only a couple of months ago that being the highest consumer of tokens within your company would get you an honorific title like "Token Legend".

Meta's CTO Andrew Bosworth made the news when his response to an employee who burned the equivalent of their salary on AI computing tokens, was:

"Keep doing it. No limit."

The equation he saw was a simple one, token consumption corresponded to productivity:

$\text{Token Consumption} \implies \text{Productivity}$

or more accurately, increased token consumption equated to increased productivity:

$\uparrow \text{Token Consumption} \implies \uparrow \text{Productivity}$

Today, some are paying the real price for that assumption; like Uber, which exhausted its entire AI budget in four months.

The cost hasn’t fallen only to consumers; AI companies have been offering services at a loss for quite some time. OpenAI’s financials revealed it ran at a $20.9 billion loss in 2025. The formula they operated on was this:

$\uparrow \text{Users} \times \uparrow \text{Use} \implies \uparrow \text{Revenue}$

More users, using our products more often will result in more revenue.

But compute does not come for free, and someone needs to pay. Both OpenAI and Anthropic have taken steps towards more aggressive pricing models in a bid to reclaim some of that money. OpenAI moved from request-based (i.e., per-prompt) billing to usage-based billing, and Anthropic is planning to reduce the types of activities permitted under its heavily subsidised subscription plans.

At the same time, larger, more powerful models are being released, roughly twice as expensive as their predecessors. Boardroom swagger alone isn't going to solve this equation for us.

So, what if the metric was the wrong one in the first place? Productivity is meaningless when divorced from value. Those executives were indeed onto something: AI adoption is vital for business success, and one way to increase adoption is to reduce friction — by incentivising the use of AI models and reducing the stigma associated with failure. However, AI literacy goes beyond AI adoption, and likewise, metrics shouldn't stop at token use.

It falls to Product Owners to define and measure customer value, to Operations to identify where toil can be reduced and savings made, and to the Security Team to measure process improvements, reliability, and reduction in incidents. The way we conduct business may have changed, but the value proposition remains the same. We need to do the much harder, more complex engineering work of measuring thin slices to see which actions lead to success, then iterating on these to distil the real value. This is also a far more sustainable approach overall (let's not ignore the water consumption required by data-centres); as engineers, we need to consider how we conserve resources, too. The principle of "just enough" ought to be one we are familiar with from testing, and it applies equally here.

Transferring the cost of compute is no bad thing; it gives power back to engineers to be deliberate in how they work and which model they use — to track progress, inputs, outputs, and define measurable improvements. The problem may be more complex than it first appeared, but in my mind it’s a far more engaging one.