Falling prices, rising bills

---

The objection you should raise before your accountant does

Here is the argument against buying an on-premise AI box, stated as strongly as I can put it:

"Cloud AI keeps getting cheaper. Why buy hardware when the price of using someone else's keeps dropping?"

It is a good objection. The first half of it is simply true.

Per-token prices really are falling, and fast

The cost of a unit of AI has collapsed and continues to collapse:

• Anthropic's flagship Claude Opus went from $15 / $75 per million tokens, input and output, to $5 / $25. A cut of roughly two thirds. (Anthropic pricing)
• Frontier input prices across the industry have fallen from around $30 per million in the GPT-4 era to a few dollars.
• The broad trend in cost per unit of intelligence drops on the order of 10× per year. (a16z, "LLMflation")

So anyone selling you a private box on the grounds that cloud tokens are expensive is wrong, and a numerate client will catch it inside a minute. I do not make that argument, and you should be wary of anyone who does.

And yet the bill still went up

The price per token fell. The number of tokens per task exploded.

Modern systems do not answer in one shot. Reasoning models work through hundreds of intermediate steps. Agents loop: read, act, check, retry, emitting ten to a hundred times more tokens per job than a simple prompt did. (Ethan Ding, on tokens getting more expensive) So even as each token gets cheaper, total tokens per real task climb faster, and the monthly invoice grows.

The result is what every heavy user is living through: unit prices down, total bills up, and worst of all, unpredictable. A single complex agentic task can cost cents or dollars depending on how many loops it takes, and you find out when the bill arrives.

The honest pitch is not cheaper, it is fixed

This is the real value of a box in your office, and it holds up because it is true:

A private box has a fixed cost. You buy it once, you power it, and it runs, whether your staff make ten queries a day or ten thousand, whether a task takes one step or fifty. There is no meter. Your AI cost is a known number on a spreadsheet, immune to the per-task blowout quietly eating everyone's cloud bill.

For a firm that values a predictable overhead over a variable one, that is worth more than shaving a few dollars off a million tokens. It matters most for the always-on, high-volume, unglamorous work, the overnight document processing and continuous triage where a metered bill balloons and a fixed box simply runs.

Where this argument stops

• If your usage is light and occasional, the cloud's variable cost is genuinely lower and a box is overkill. Fixed cost only wins at volume, or where predictability itself has value.
• If you need the absolute frontier for the hardest tasks, pay the cloud for those specifically. That is the right tool, and a box does not replace it. The size of that gap, and which work falls either side of it, is measured in the frontier–local gap.
• The economics are strongest for high-volume, bounded, private work. Outside that, I will say so.

There is a related trap on the hardware side. A machine that comfortably fits a large model can still be too slow to use it, because these boxes are limited by memory bandwidth rather than capacity. What actually runs on the box covers why the specification sheet will not tell you that, and what to ask for instead.

The one-line version

Cloud AI is not getting more expensive per token. It is getting less predictable per task. A box in your office trades a shrinking but volatile bill for a fixed one you control.

Whether that trade is worth it depends on your volume, and on how much you value knowing the number in advance. If the answer is that you do not much care, you probably do not need me. Before you spend anything, retrieval or training is worth reading, because choosing wrong there costs more than the hardware does.