Your LLM Cost Model Is Linear. Your Agent Is Not.

Someone on your team ships an agent. It works. Two weeks later, finance asks why the API line item grew by a factor of six when the user count grew by forty percent.

You check the dashboard. Request volume is roughly flat. Nobody deployed a bigger model. There is no loop bug, no runaway process, no incident. Every individual request looks reasonable when you inspect it.

The bill is still six times bigger, and the explanation is not in your monitoring, because your monitoring counts requests and the thing that changed was the shape of them.

This post is arithmetic rather than research. There are no citations in it, and it does not need any — the entire argument is a consequence of how transformer inference is billed, and you can verify every number here with a calculator. That is deliberate. Cost surprises in AI systems are almost never caused by a fact you did not know. They are caused by a multiplication you did not do.

The mental model everyone starts with

Ask an engineer to estimate what an agent run costs and you will almost always get some version of this:

cost  =  steps  x  tokens_per_step  x  price_per_token

Every term is real. The formula is still wrong, and it is wrong in the specific way that matters: it treats tokens_per_step as a constant.

It is not a constant. It is a function of the step number.

Why every step is more expensive than the last

Language models are stateless. The model does not remember step 4 when it runs step 5. The only way it knows what happened is that you send the entire history back to it, every single time.

So on step 1, you send the system prompt and tool definitions. On step 2, you send the system prompt, the tool definitions, the step 1 tool call, and the step 1 result. On step 12, you send all of that plus eleven steps of accumulated calls and results.

The conversation is not a stream you append to. It is a document you retransmit in full, once per step, growing the whole time.

That single property is the entire story. It turns a loop that looks linear in your code into a cost curve that is quadratic in your invoice.

What your code looks like        What you are billed for
─────────────────────────        ──────────────────────────
for step in range(n):            step 1  ▏
    call_model()                 step 2  ▎
                                 step 3  ▍
one line                         step 4  ▋
n iterations                     step 5  ▉
looks linear                     ...     ████████████
                                 step n  ████████████████████

The arithmetic

Let me make this concrete, because the size of the effect is genuinely hard to intuit.

Take an agent with a realistic shape:

Fixed prefix: 4,000 tokens of system prompt and tool definitions. Resent every step.
Per-step increment: 800 tokens. The model’s tool call plus whatever the tool returns.

Input tokens on step k are 4000 + 800 x (k-1). Summing over a full run of n steps gives:

total_input  =  4000n  +  800 x n(n-1)/2

The first term is linear. The second term is quadratic, and it is the one that eats you.

Steps	Naive estimate	Actual input tokens	Ratio
10	40,000	76,000	1.9x
30	120,000	468,000	3.9x
60	240,000	1,656,000	6.9x

The naive column assumes each step costs the prefix. Reality diverges immediately and keeps diverging.

Now the number that should reorganize how you think about agent design. Compare the 30-step run to the 60-step run:

30 steps    468,000 tokens    ██████
60 steps  1,656,000 tokens    ████████████████████

Doubling the step count multiplied the cost by 3.5x.

Not 2x. Nothing about your traffic doubled twice. The agent simply started taking longer on average — maybe because you added a tool, maybe because your users got more ambitious, maybe because a model update made it more thorough — and the bill responded with an exponent your forecast did not have.

This is why AI cost overruns feel inexplicable from the outside. Every individual decision was reasonable. The composition was not.

Why “just use a cheaper model” usually fails

When the bill arrives, the reflex is to switch to a cheaper model. It is the wrong first move, and the arithmetic shows why cleanly.

Total cost is roughly:

cost  =  token_volume(architecture)  x  price(model)

Switching models changes the second term. It is a constant factor. If you find a model at one-fifth the price, you cut the bill by five, once, and that is the entire benefit available from that lever forever.

Meanwhile the first term is governed by an exponent. A modest drift in average run length — 30 steps to 45 — takes you from 468,000 tokens to 972,000, which is 2.1x. That single behavioural drift eats more than 40% of your 5x saving, and drift is not a one-time event.

Then the second-order effect, which is the one that actually bites: cheaper models often need more steps to accomplish the same task. They misread a tool result, retry, take a detour, need an extra verification pass. You reduced the price per token by 5x and increased n by 40%, which multiplies token volume by roughly 2x.

You can absolutely switch to a model that is five times cheaper per token and watch your bill go up. Fix the volume, then shop for price. In that order.

What caching does and does not do

Prompt caching is the other standard answer, and it is genuinely worth implementing — but it is routinely oversold, and the distinction is worth being precise about.

Caching lets you pay a reduced rate on tokens the provider has already processed. Your stable prefix — system prompt, tool definitions — is the ideal candidate, since it is identical on every call.

Here is what it does not do: stop the resending. Your context still grows every step, and you still pay for every token in it on every call. Caching lowers the rate on the repeated portion. It does not change the fact that the repeated portion is growing.

In the language of the formula: caching reduces the coefficient. It does not touch the exponent.

Concretely, a 60-step run is still several times a 30-step run after caching. The curve has the same shape, drawn slightly lower. If your cost problem is that runs are getting longer, caching buys you time, not a solution.

One practical note that follows directly: put your genuinely stable content first and your volatile content last. Caching works on prefixes, so a single early token that changes per request — a timestamp, a user ID, a session token injected at the top of your system prompt — can invalidate everything after it. That is a five-minute fix that people leave broken for months.

The term you actually control

Look at the formula again:

total_input  =  prefix x n  +  increment x n(n-1)/2

You have three knobs. They are not equally useful.

prefix is multiplied by n. Worth trimming, but it is the linear term, and it is the one people obsess over. Shaving 500 tokens off your system prompt feels productive and moves the smallest number.

n appears squared. Capping it is powerful, and I will come back to it.

increment multiplies the entire quadratic term. This is the highest-leverage number in the system, and almost nobody instruments it.

Here is why it dominates. A tool result does not cost you once. It enters the context and is re-sent on every remaining step. A 5,000-token result returned at step 5 of a 40-step run is paid for roughly 35 more times after the step that produced it.

So when a tool returns a full API response, an entire file, or an unfiltered query result, it is not a one-off expense. It is a subscription that runs until the end of the trajectory.

Tool returns 5,000 raw tokens at step 5, run continues to step 40
   → that payload is retransmitted ~35 more times

Tool returns 200-token summary plus a reference
   → 25x cheaper, for the rest of the run, from a change on the tool side

Engineers instinctively optimize the prompt, which is paid once per step. The tool output is the term that compounds. If you profile only one thing after reading this, profile the size distribution of your tool results.

Retries cost more than you are accounting for

Standard retry logic treats every retry as equivalent: catch the error, back off, try again. That is a latency strategy. It is not a cost strategy, and applied to agents it is quietly expensive.

A retry re-sends the entire accumulated context. So the cost of a retry depends on where it happens:

Failure point	Approximate retry cost
Step 3	negligible
Step 20	a full 20-step context
Step 40	a full 40-step context

Late failures are dramatically more expensive than early ones. And there is a behavioural trap layered on top: late failures are the ones people are most inclined to retry, because so much work has already gone into the run. Abandoning at step 40 feels wasteful. Financially, it is often the correct call — the retry costs more than the entire first 15 steps did.

Two things follow. Validate as early in the trajectory as you can, because the cost of discovering a problem rises with the step number. And treat a run that has already failed twice deep in its trajectory as a candidate for abandonment rather than a third attempt.

Why subagents can be cheaper, despite appearances

Multi-agent architectures look obviously more expensive. More agents, more calls, more tokens. Often true per agent, and often wrong in total — for a reason that falls straight out of the arithmetic.

Compare two ways to do the same work.

Single agent, 40 steps. Every tool result from every subtask lands in one context and is re-sent for the remainder of the run. One accumulation, growing the whole way, paying the quadratic on the full 40.

Parent plus four subagents, 10 steps each. Each subagent accumulates its own short context and pays a quadratic on 10, which is small. Critically, the parent never sees those 40 steps of intermediate output. It receives four summaries.

Four small quadratics beat one large one, because the term grows with the square. Four runs of 10 steps produce far fewer total tokens than one run of 40.

But this only holds if the boundary is real. If your subagents return full transcripts, or the parent re-reads everything they touched, you have paid for the fan-out and kept the accumulation. You get the cost of both designs and the benefit of neither.

The value of a subagent is context isolation, not parallelism. Parallelism makes it faster. Isolation makes it cheaper. If your subagents are not throwing away most of their context at the boundary, they are not doing the job that justifies them.

What to actually do

Roughly in order of value per unit of effort:

Instrument tokens per run, not requests per day. Almost every AI cost surprise is invisible in request-count dashboards, because request count is not the variable that moved. If you track one new metric, track the distribution of tokens per completed run — and watch the tail, not the mean. The p99 run is where the money is.
Cap tool output size at the tool, not in the prompt. Truncate, summarize, or return a reference the agent can dereference on demand. This is usually a small change on the tool side and it is the single biggest lever available, because it multiplies the quadratic term.
Set a hard step ceiling. It bounds the quadratic directly. Note what it removes: in a run that grows every step, the final steps are the most expensive steps of the entire run, so a cap does not shave an average slice — it removes the most costly one. This is also why a runaway loop is financially unlike an ordinary bug. The cost per iteration is still climbing while it spins.
Compact aggressively at a threshold. When context crosses some size, summarize the middle and continue. This resets the growth curve. Most frameworks support it and most teams leave it at defaults tuned for capability rather than cost.
Order your prompt for cache stability. Stable content first, volatile content last. One early per-request token can invalidate the entire cached prefix.
Make failures cheap and early. Validate inputs before the expensive trajectory starts, not at step 30. Cost of discovery rises with step number.
Use subagents for isolation, and enforce the boundary. Return summaries. If a subagent hands back a full transcript, you have paid for the architecture without buying the saving.
Shop for model price last. It is real money and it is a constant factor. It will not save an architecture with a bad exponent, and chasing it first tends to increase step counts, which is the term you least want to grow.

The takeaway

The reason AI bills surprise people is not that tokens are expensive. It is that the cost of an agent run is quadratic in its length, while essentially every tool we use to reason about it — request dashboards, per-call pricing pages, unit tests, back-of-envelope estimates — is linear.

Your architecture has an exponent. Your forecast does not. The gap between them is the invoice.

The good news is that the fix is mostly not clever. Cap the loop. Shrink what your tools return. Throw context away at boundaries. Measure the distribution rather than the average. None of that requires a better model, a new framework, or a vendor negotiation — it requires doing the multiplication before finance does it for you.

FAQ

Why did my LLM API bill suddenly jump without more users?

The most common cause is that your agent’s average run got longer, not that your traffic grew. Token cost per run does not scale with the number of steps, it scales with roughly the square of the number of steps, because every step re-sends the entire accumulated conversation as input. An agent whose average run went from 15 steps to 30 steps did not double in cost. Using a typical setup where each step adds around 800 tokens to a 4,000 token base, 15 steps costs about 144,000 input tokens and 30 steps costs about 468,000 — a 3.25x increase from a 2x change in behaviour. Nothing in your traffic dashboard shows this, because request count barely moved. Look at tokens per run over time, not requests per day.

Does prompt caching actually reduce agent costs?

It reduces the coefficient, not the growth rate, and that distinction decides how much you save. Caching lets you pay a reduced rate on tokens the provider has already seen, so your stable prefix — system prompt, tool definitions, fixed instructions — becomes much cheaper to re-send. But the re-sending itself does not stop. Your conversation still grows every step, and you still pay for every token in it on every call, just at a lower per-token rate for the cached portion. So caching might cut the bill meaningfully, but a 60-step run still costs several times a 30-step run. Caching is genuinely worth implementing, and it is not a substitute for bounding how much context you accumulate.

Is switching to a cheaper model the best way to cut LLM costs?

It is the most common first move and usually the least effective one, because it targets the wrong variable. Total cost is roughly the token volume your architecture generates multiplied by the price per token. Switching models changes the price per token, which is a constant factor — you might cut it by 5x. But if the architecture is generating quadratic token growth, that 5x saving gets consumed by a modest increase in average run length. Worse, cheaper models often need more steps to complete the same task, and steps are the term that grows quadratically. It is entirely possible to switch to a model that is 5x cheaper per token and see your bill go up. Fix the token volume first, then shop for price.

Why are multi-agent systems disproportionately expensive?

Because each subagent carries its own independently growing context, so you are not running one quadratic curve, you are running several in parallel. Five subagents doing 20 steps each is not equivalent to one agent doing 100 steps — it is five separate accumulations, each paying to re-send its own history. The counterintuitive part is that this is often still cheaper than the single-agent alternative, because each subagent’s context stays short and returns only a summary to the parent. The parent never absorbs the 20 steps of intermediate tool output. Multi-agent architectures are expensive per agent and can be cheaper in total, and which effect wins depends entirely on how much context you let cross the boundary.

How do I estimate an agent’s cost before building it?

Estimate three numbers and the arithmetic follows. First, the fixed prefix: system prompt plus tool definitions, resent every single step. Second, the average increment per step: the model’s output plus whatever the tool returns. Third, the expected step count. Total input tokens are approximately steps times prefix, plus increment times steps times steps minus one over two. The second term is the one that surprises people, and it dominates once step counts pass roughly ten. Run this before you write code, because it tells you immediately whether your tool outputs are affordable. A tool that returns 5,000 tokens of raw JSON on every call will dominate your entire bill by step 20, and you will not discover that from a unit test.

Do maximum step limits actually save money?

More than almost any other single change, because they bound the quadratic term directly rather than shaving the constant. The important detail is where the savings come from. In a run that grows quadratically, the last few steps are the most expensive steps in the entire run — step 50 sends far more input tokens than step 5. So a step cap does not remove an average slice of your cost, it removes the most expensive slice. Cutting a 60-step ceiling to 40 steps removes roughly the top third of steps but a much larger share of the tokens. This is also why runaway loops are financially dangerous in a way that ordinary bugs are not: the cost per step is still climbing while the loop spins.

Why do failed runs and retries cost more than expected?

Because a retry re-sends everything accumulated up to the point of failure, so the cost of a retry depends on where it happens, not just how often. Retrying a failure at step 3 costs almost nothing. Retrying at step 40 costs the entire 40-step context again. Most retry logic treats all retries as equivalent and applies uniform exponential backoff, which is a latency strategy rather than a cost strategy. The financial consequence is that late-stage failures are the expensive ones, and they are also the ones most likely to be retried, because more work has already been invested. Failures should be detected as early as possible, and a run that has already failed twice deep in its trajectory is usually worth abandoning rather than retrying.

What is the single biggest lever on agent token cost?

The size of what your tools return, because that number is multiplied by every subsequent step. A tool result does not cost you once. It enters the context and is re-sent on every remaining step of the run, so a 5,000 token result at step 5 of a 40-step run is paid for roughly 35 more times. Reducing that result to a 200 token summary plus a reference the agent can dereference on demand is the highest-leverage change available in most systems, and it is usually a small code change on the tool side rather than an architectural rewrite. Engineers instinctively optimize the prompt, which is paid once per step. The tool output is the term that compounds.

A note on the numbers

Every figure in this post is derived, not reported. The 4,000-token prefix and 800-token increment are illustrative parameters chosen because they are typical of a small production agent — substitute your own and the shape of the conclusion does not change, only its magnitude. The formula is the load-bearing part:

total_input  =  prefix x n  +  increment x n(n-1)/2

Deliberately absent: current per-token prices from any vendor. They change frequently, they differ by model and by tier, and quoting them would date this post within weeks while adding nothing to the argument. Multiply the token volumes here by whatever rate you are actually paying.

The one claim worth stating plainly, because it is the only thing you need to accept for the rest to follow: models are stateless, so context is retransmitted in full on every step. Everything else in this post is that fact plus arithmetic.