The Token Bill Nobody Forecasted

The first AI invoice that makes a CFO sit up straight is rarely the one you budgeted for. You modeled the cost of the feature — so many requests a day, so many tokens each, multiply, done. Then you shipped an agent instead of a chatbot, and the arithmetic quietly stopped applying. The bill that arrives isn't double your estimate. It's a different shape entirely, and the reason is worth understanding before it's your turn to explain it in a finance meeting.

I've watched smart teams forecast AI cost the way they'd forecast a database: pick a unit, count the units, price the units. That works when each user action maps to one model call. It falls apart the moment an action maps to a workflow — and agents are workflows. One user request fans out into a plan, a dozen tool calls, a few retries when a tool returns something unexpected, a summarization pass, maybe a critique step. The user did one thing. The agent did thirty. Your per-request estimate was off not by a margin but by a multiplier you never put in the spreadsheet.

Where the money actually goes

If you want to forecast agent cost, you have to understand the cost drivers that don't exist in a single-call world. There are about four that matter.

Fan-out. A single agentic task is many model calls, not one. The plan-act-observe loop runs until the agent decides it's done, and "done" is a model judgment, not a fixed number. A task that resolves in three steps for one input might take eleven for another. Your cost-per-task has variance you can't see at design time, which means your monthly total is a distribution, not a number.

Context growth. This is the sneaky one. In a multi-step agent, every step typically re-sends the accumulating history — the original request, prior tool outputs, intermediate reasoning. Token cost per step doesn't stay flat; it climbs as the conversation grows, because you're paying to re-read the whole transcript each turn. A ten-step task isn't ten times the cost of one step. It's worse, because the later steps are fat. Long-context models with million-token windows make this trap deeper, not shallower — the ceiling went up, and so did the floor.

Retries and dead ends. When a tool errors or returns garbage, a well-built agent tries again, reasons about the failure, takes another path. That's the resilience you wanted. It's also tokens you're paying for that produced no user-visible output. A flaky downstream API doesn't just hurt reliability; it shows up on the model bill.

Model choice per step. Not every step needs your most expensive model. The planning step might; the "extract the date from this string" step absolutely does not. Teams that route every step to the frontier model are paying premium rates for clerical work, and it adds up fast across the fan-out.

How to forecast and control it

The fix isn't to fear agents. It's to instrument and engineer them like the cost centers they are, and the moves are concrete.

First, measure cost per completed task, not cost per call. The unit the business cares about is "resolved support ticket" or "merged PR" or "processed invoice," and that unit might be five calls or fifty. Track tokens-per-task and watch its distribution, especially the tail. The expensive truth lives in the 95th percentile, where one gnarly input loops twenty times. Budget against the tail, not the average, or the tail will budget against you.

Second, route by step. Send the hard reasoning to the capable model and the trivial extraction to a cheap, fast one. The major platforms now ship model routers that do this per request and report meaningful savings from sending each task to the cheapest model that's good enough (Microsoft Foundry blog). Whether you use a platform router or build the routing yourself, the principle holds: stop paying frontier prices for fifth-grade tasks.

Third, cache aggressively and prune context. If every step re-sends an identical system prompt and tool catalog, prompt caching turns that from a recurring charge into a near-free one. And be ruthless about what history each step actually needs — most steps don't need the full transcript, they need the last result and the goal. Context discipline is the single biggest lever on the context-growth problem.

Fourth, cap the loop. Set a maximum step count and a token budget per task, and decide in advance what happens when an agent hits it — escalate to a human, return a partial answer, fail loudly. An agent with no ceiling is an open-ended spend authorization, and "it ran in a loop overnight" is a real way to discover that.

Fifth, put spend alerts on AI line items specifically, not buried in the general cloud bill. The whole point is to never be ambushed; a per-workflow alert is how you find out on Tuesday instead of at month-end.

The takeaway

Agent cost isn't a bigger version of chatbot cost — it's a different function, driven by fan-out, growing context, retries, and indiscriminate model choice. Forecast it by measuring tokens per completed task and budgeting against the tail. Control it by routing cheap work to cheap models, caching and pruning context, and capping the loop. Do that and agents are affordable and predictable. Skip it and you'll meet the token bill nobody forecasted — and you'll be the one explaining it.

Sources: What's New in Microsoft Foundry, Feb 2026 (Model Router & cost routing)