Cached tokens don’t change the sticker price of a model, they change how much of your repeated context gets billed at the lower cache rate instead of the full input rate. If you ship long prompts, agent loops, or shared system context, that difference is often the gap between “this feels fine” and “why is this bill weirdly high?”
What cached tokens are actually doing
When a model sees the same prefix again, part of that repeated context can be served from cache. The practical effect is simple: the provider can charge less for tokens it doesn’t need to process from scratch. That matters most when your app sends the same instructions, schemas, tool definitions, or conversation history over and over.
Think of it as a discount on repeated input, not a magic shortcut that makes the whole request free. You still pay for fresh input, output, and any reasoning or thinking tokens if the model exposes them separately. The bill drops only when the repeated portion is large enough to matter.
If you want the bigger picture on how AI spend is measured across models and tools, the AI Cost Index is the right lens. It shows blended pricing so you can compare models without getting lost in one-off request patterns.
Why repeated context is the silent budget killer
Most dev teams don’t notice cache effects until they have one of these patterns:
- A giant system prompt copied into every request.
- Tool schemas and policy text that never change.
- Agent loops that resend the same conversation history.
- Multi-agent workflows where each agent inherits the same base instructions.
In all four cases, the model is doing the same work over and over. If that repeated prefix is eligible for caching, you’re no longer paying full freight for every duplicate token. If it isn’t, you’re burning budget on context that didn’t need to be reprocessed in the first place.
This is why long-context apps can feel cheap in a prototype and then get annoying in production. The prototype has tiny traffic. The production version turns a few repeated prompts into millions of tokens a month.
A worked example with real token math
Let’s use a concrete example with the only prices we can safely cite here. Suppose you run a workflow on gpt-4o, which is priced at $2.5 input and $10 output per 1M tokens. Imagine a request with 120,000 input tokens and 30,000 output tokens.
Without caching, the cost is:
(120,000 / 1,000,000) * $2.5 = $0.30 input cost
(30,000 / 1,000,000) * $10 = $0.30 output cost
Total = $0.60Now assume 80,000 of those input tokens are repeated context that can be cached, and only 40,000 are fresh input. If the repeated portion is billed lower than normal input, your cost drops because only the fresh part pays the full input rate. The exact savings depend on the provider’s cache pricing, but the mechanism is the same: fewer full-price input tokens.
Here’s the same idea in a simpler form using token counts only, so you can reason about your own app before you even know the exact cache discount:
- Original input: 120,000 tokens
- Repeated context: 80,000 tokens
- Fresh context: 40,000 tokens
- Output: 30,000 tokens
If your app repeats that pattern 1,000 times, you are no longer talking about a cute optimization. You are talking about 80 million tokens of reusable context. That is exactly the kind of workload where caching starts to matter.
Which models make cache savings easier to notice
Cache savings are easiest to spot on workloads that already have a meaningful input bill. The more expensive the input side, the more valuable repeated-context reduction becomes. That is why model choice and prompt shape matter together, not separately.
| Model | Input price per 1M tokens | Output price per 1M tokens | What that means in practice |
|---|---|---|---|
| gpt-4o | $2.5 | $10 | Repeated input can still add up fast because output is also pricey. |
| claude-sonnet-4-5 | $3 | $15 | Great candidate for careful prompt reuse because output is expensive. |
| gemini-2.5-flash | $0.3 | $2.5 | Already cheaper on input, so caching helps, but the absolute savings are smaller. |
| deepseek-chat | $0.28 | $0.42 | Low prices make caching less dramatic, but repeated context still wastes tokens. |
If you want to compare models by blended cost instead of raw input and output rates, the live model price list is the better starting point. For a broader market view, the public State of AI page is useful context.
How to design prompts that benefit from caching
The easiest way to get cache wins is to stop making the model reread junk. That sounds obvious, but most systems accidentally do the opposite.
- Keep the stable prefix stable. Put system instructions, policy text, and schema definitions in a fixed order.
- Avoid reshuffling content. If the same information appears in a different order each request, you reduce cache reuse.
- Separate volatile data from static instructions. Don’t mix changing user content into the same giant block as your permanent prompt.
- Trim conversation history aggressively. If the model only needs the last few turns, don’t resend the whole novel.
- Watch tool payloads. Big function schemas and tool docs are often the hidden repeat offender.
There’s a tradeoff, of course. Over-optimizing for cache reuse can make prompts harder to maintain. The goal isn’t to write weird prompt code, it’s to make repeated context intentional instead of accidental.
How to track this without guessing
Guessing at cache savings is how teams end up arguing from vibes. The better move is to measure actual usage, then see where repeated context shows up in the logs. That’s exactly what MyTokenTracker is for.
Run npx mytokentracker locally and it reads the usage logs coding agents already keep on your machine, then prints a 30-day cost table at API prices. Your usage data never leaves the machine. If you want a dashboard, create a free account and run npx mytokentracker init, which uploads history and then syncs every 30 minutes. It works with Claude Code, Codex, GitHub Copilot CLI, Antigravity, OpenCode, Amp, Qwen Code and more.
For deeper breakdowns, it tracks cost, input/output/cache/reasoning tokens, latency, and success, broken down by provider, model, platform, and use-case. That makes it much easier to spot the exact workflow where repeated context is bloating your bill. Only daily totals per agent and model are uploaded, never prompts, code, or file names.
If you want to dig into the raw data philosophy behind the product, the open dataset is documented on the data page. The data is open under CC BY 4.0.
FAQ
Are cached tokens always cheaper?
Usually, yes, but the real question is how much cheaper and for which part of the prompt. Cached tokens reduce the cost of repeated context, not the whole request. If your prompt is mostly fresh user content, caching won’t move the needle much.
Do cached tokens change model quality?
No. Caching changes billing and latency characteristics, not model intelligence. The model still sees the same context, it just doesn’t need to reprocess repeated input the same way every time.
What should I measure first if I suspect cache waste?
Start with the size of your repeated prefix, then compare it to total input tokens over a day or week. If the repeated portion is large and the workflow runs often, you’ve probably found a real savings target.
If you want to see what your own repeated context is really costing, try MyTokenTracker and look at your 30-day token table.