Prompt caching only saves money on repeated input tokens, not on fresh generation. If your workload keeps reusing the same system prompt, tool schema, or long context, the bill can drop fast, but the savings stop the moment the prompt changes.
What prompt caching actually changes
For developers, prompt caching is not magic, it is just a pricing shortcut for reused context. The model still has to process the request, but the repeated prefix can be billed differently when the provider recognizes it as cacheable. That means the biggest wins usually come from long, stable inputs like system instructions, tool definitions, and large retrieved context blocks that get sent over and over.
The important part is scope. Caching helps the input side of the bill. It does not make output tokens cheaper, and it does not help if your prompt is always unique. If you are debugging a chatty agent that generates a lot of text, the output side can still dominate even when input caching is working well.
That is why cost conversations get clearer when you look at the full picture, not just a single model price. The AI Cost Index gives a simple blended view, while live model prices on 4,747 models show how much room there is between frontier and budget options.
The math: a worked example with real token numbers
Let’s use a simple repeated-call example. Say your agent sends 120,000 input tokens of stable context on every run, plus 8,000 output tokens of fresh work. You run that call 50 times in a day. Without caching, the input portion is paid 50 times.
Using gpt-4o at $2.5 input and $10 output per 1M tokens, the arithmetic looks like this:
Input cost per call = 120,000 / 1,000,000 * $2.5 = $0.30
Output cost per call = 8,000 / 1,000,000 * $10 = $0.08
Total per call = $0.38
50 calls = 50 * $0.38 = $19.00Now imagine 100,000 of those input tokens are repeated and cacheable on every call, and only 20,000 are truly fresh. The fresh input cost becomes:
Fresh input cost per call = 20,000 / 1,000,000 * $2.5 = $0.05
Output cost per call = 8,000 / 1,000,000 * $10 = $0.08
Total per call = $0.13
50 calls = 50 * $0.13 = $6.50That is a $12.50 difference over just 50 calls, and it comes entirely from avoiding repeated input work. Same model, same task, same output, lower bill because the repeated prefix stopped being fully charged as fresh input.
The catch is obvious once you write the arithmetic down. If your prompt changes every time, or if the expensive part is the model’s answer, caching won’t rescue you. It is a lever, not a discount coupon for everything.
Which workloads benefit most
Prompt caching is most useful when your app or coding agent repeats a large static prefix. Typical examples include:
- Long system prompts that define agent behavior
- Tool schemas and function signatures that rarely change
- Repository context that gets reused across many turns
- RAG blocks that are stable for a session or project
- Multi-step agent loops that resend the same instructions
It helps less when each request is short, highly variable, or dominated by output. A tiny prompt with a long answer is not a caching story, it is a generation story. If you want to know which side of the bill matters most, compare input and output usage in the same workflow, not just the headline model name.
That is also why some teams think a model is “expensive” when the real issue is prompt shape. A bloated instruction block can cost more than the task itself. If you want a broader market view of how models are priced, the live catalog in models is the cleanest place to sanity-check the raw numbers.
Comparison table: same task, different pricing shape
Here is a compact comparison using only the prices we can state with confidence. The point is not that one model is always best, but that input-heavy and output-heavy workloads behave differently.
| Model | Input | Output | What that means in practice |
|---|---|---|---|
| gpt-4o-mini | $0.15 / 1M | $0.6 / 1M | Cheap enough that repeated input matters, but output is still the bigger line item. |
| deepseek-chat | $0.28 / 1M | $0.42 / 1M | Very low blended cost, so caching helps most when context is large and repeated. |
| gpt-4o | $2.5 / 1M | $10 / 1M | Input caching can save real money fast when the same prefix gets replayed often. |
| claude-sonnet-4-5 | $3 / 1M | $15 / 1M | Useful when quality matters, but output-heavy usage gets expensive quickly. |
For a blended market snapshot, the AI Cost Index currently shows the Frontier basket at $4.64 per 1M tokens and the Budget basket at $0.74 per 1M tokens, both blended 3:1 input:output. That gap is a reminder that even before caching, model choice has a huge effect on your spend.
Why caching sometimes looks better than it is
There are a few ways caching can make the bill look smaller without actually changing the real workload very much.
- Repeated prompts hide prompt bloat, which means you may keep paying to resend the same giant context until you notice it.
- Short sessions can overstate savings, because one lucky cache hit can make a small sample look better than the long-run average.
- Output-heavy tasks still cost money, even if input caching is perfect.
- Provider pricing rules differ, so the same prompt pattern can behave differently across vendors.
This is why a raw token count is not enough. You need to know whether those tokens were fresh, cached, input, output, or reasoning. If you want to see how those dimensions show up in the wild, the public State of AI page is a better lens than a single invoice screenshot.
There is also a strategic angle. If you are building a coding agent, cacheable context is often a design problem, not a pricing problem. Smaller prompts, better session reuse, and less redundant tool chatter all reduce the amount of text you have to pay to send in the first place.
How to track this in your own stack
The practical way to measure caching is to compare repeated runs of the same task and inspect the token breakdown, not just the final dollar amount. You want to see input, output, cache, reasoning, latency, and success side by side, broken down by provider, model, platform, and use-case.
That is exactly the kind of data MyTokenTracker is built to read. The quickest way to try it is npx mytokentracker, which reads the usage logs your coding agents already keep locally and prints a 30-day cost table at API prices. Your usage data never leaves the machine unless you choose to sync.
If you want a dashboard, create a free account and run npx mytokentracker init. It asks for the API token, uploads history, then syncs every 30 minutes. It works with Claude Code, Codex, GitHub Copilot CLI, Antigravity, OpenCode, Amp, Qwen Code and more. Only daily totals per agent and model are uploaded, never prompts, code, or file names.
For teams that want to compare what they are getting for the money, the value-for-money view is a good complement to raw spend. Sometimes the cheapest workflow is not the one with the lowest model price, it is the one that finishes the job with fewer retries.
FAQ
Does prompt caching lower output token cost?
No. Caching helps with repeated input, not with the text the model generates. If your workflow produces long answers, summaries, or code diffs, output can still be the main cost driver.
Is caching worth optimizing for small prompts?
Usually not. If your prompts are short, the savings are small and easy to drown out with normal variance. Caching becomes interesting when the repeated prefix is large enough to matter in the arithmetic.
How do I know if caching is actually helping?
Track the same task over multiple runs and compare the input breakdown, not just total cost. If repeated context is shrinking while output stays stable, caching or prompt reuse is doing real work. If the input side barely changes, look elsewhere first.
If you want to see your own repeated-context bill in plain numbers, run MyTokenTracker and inspect the last 30 days.