Prompt caching lets a model reuse a prompt prefix it has already processed, so the stable part of each request is billed at a reduced rate instead of the full input price. Whether that turns into a real prompt caching cost reduction depends on how many requests reuse the prefix before it expires, set against what the provider charges to write it, read it, and sometimes store it. Reuse a long prefix often and the stable part of the bill drops sharply. Change it on every request and you pay for writes without collecting the reads.
The mechanics differ by provider. OpenAI matches eligible prefixes automatically, Anthropic and AWS Bedrock use explicit cache markers, and Google Gemini bills cached context by storage time as well as by read. A discount percentage from one provider does not transfer to another, so the number worth acting on is the one you calculate from your own prefix size, hit rate, and price table.
This page does that calculation for one concrete case, Claude Sonnet 4.6 at Anthropic's list prices, then covers provider differences, prompt layout, and measurement. The short answer: cache a long, stable prefix when the read price is below the ordinary input price and the prefix is reused often enough to pay for writes and storage, and keep changing user data after the cache boundary.
The prices and assumptions behind the math
Every figure in the next two sections comes from this price table and these assumptions, so you can swap in your own.
| Claude Sonnet 4.6 rate | Price per million tokens | Relative to base input |
|---|---|---|
| Base input | $3 | Baseline |
| 5-minute cache write | $3.75 | A quarter more than base |
| 1-hour cache write | $6 | Double base |
| Cache hit | $0.30 | One tenth of base |
These are the list prices on Anthropic's pricing page when this guide was published. They are a dated example, not a permanent price, so recheck the official table before using them in a quote. The right-hand column is my arithmetic on those prices.
The worked scenarios hold these assumptions fixed:
- Workload: 1,000 requests share one 10,000-token stable prefix, and each request adds 1,000 dynamic tokens.
- Writes: one 5-minute cache write, counted as one of the 1,000 requests.
- Hit rate: an input you choose, not a prediction. The table uses 10%, 25%, 50%, and 80%, and the formula example uses 900 hits.
- Misses: every request that is neither the write nor a hit reads the stable prefix at the ordinary $3 input price.
- Exclusions: TTL storage, dynamic tokens, and output tokens. The results are the cost of the stable prefix only.
Prompt cache break-even formula
Let S be stable prefix tokens, D dynamic tokens per request, P the ordinary input price per token, W the cache-write price, R the cache-read price, Q total requests, writes the number of cache writes, hits the number of cache reads, and storage the TTL storage cost.
Without caching:
Uncached cost = Q × (S + D) × P
With caching:
Cached cost = S × (writes × W + hits × R + misses × P) + Q × D × P + storage
Here, misses = Q - writes - hits. The formula assumes one stable prefix and leaves out output tokens; add output cost to both sides when caching changes the number or length of responses. The dynamic term Q × D × P appears on both sides, which is why the scenarios can ignore it: the saving comes from the stable prefix, less any storage.
With S = 10,000, D = 1,000, P = $3/MTok, W = $3.75/MTok, and R = $0.30/MTok, one write followed by 900 hits and 99 ordinary misses makes the stable-prefix cost about $5.71, compared with $30.00 without caching. That is before TTL storage and dynamic-token costs.
Worked per token, the break-even point on these prices is low. A 5-minute write costs a quarter of base input more than an uncached read, and each hit saves nine tenths of base input, so a single hit before the entry expires recovers the write premium. A 1-hour write costs a full extra base price, so it needs at least two hits within its TTL to come out ahead. That is arithmetic on list prices, not a measured result.
The equation also shows where caching goes wrong: a cache read may be cheap, but every write still costs money, and a prefix that changes on every request pays for writes without collecting enough reads. If a miss in your setup creates a fresh cache entry, count it under writes, not misses.
Prompt cache hit rate scenarios
The same Claude example, held at 1,000 requests and 10,000 stable tokens, with one cache write and no storage charge:
| Cache hit rate | Cache hits | Stable-prefix cost | Uncached baseline | Saving before storage |
|---|---|---|---|---|
| 10% | 100 | $27.31 | $30.00 | $2.69 |
| 25% | 250 | $23.26 | $30.00 | $6.74 |
| 50% | 500 | $16.51 | $30.00 | $13.49 |
| 80% | 800 | $8.41 | $30.00 | $21.59 |
Because one hit already covers a 5-minute write at these prices, every row comes out ahead, from $2.69 at 10% to $21.59 at 80%. The open questions are how far ahead, and whether storage or repeated writes eat the difference. None of this predicts the hit rate a production application will reach: request cadence, prefix length, tenant isolation, and invalidation rules decide that. Recalculate when the provider changes rates or cache semantics.
How prompt caching works
Prompt caching stores or reuses a previously processed section of a prompt. That section is usually a stable prefix holding system instructions, tool definitions, schemas, or reference material, with the dynamic user request after it. A cache hit avoids charging the stable tokens at the ordinary input rate, though the provider may charge for writing or storing the prefix.
| Term | What it means for cost |
|---|---|
| Cache write | Tokens processed and stored for future reuse |
| Cache read cost | Price for stored tokens reused on a matching request |
| Cache miss | The prefix fails to match or has expired |
| TTL | How long the cached state remains available |
| Dynamic suffix | Per-user or per-request tokens billed normally |
| Storage | Provider charge for keeping explicit cached content |
AWS describes its cache checkpoints as markers for a contiguous prompt subsection. With explicit checkpoints or OpenAI's automatic prefix matching alike, prompt layout becomes part of the cost calculation.
Prompt caching is not semantic caching, which reuses an earlier answer for a meaningfully similar request, and it is not RAG, which retrieves documents at request time. The three can work together, but their hit rates and costs belong in separate columns.
Prompt caching cost by provider
Providers use related ideas with different billing units and controls:
| Provider | Cost and control facts to model | Main implementation risk |
|---|---|---|
| Anthropic Claude | Base input, 5-minute writes, 1-hour writes, cache hits, and output are separate rates. | A longer TTL can cost more, while changes earlier in the prefix invalidate later content. |
| OpenAI API | Eligible common prefixes can receive discounted cached input; usage reports cached tokens. | Inactivity clears caches, and a changed prefix reduces reuse. |
| AWS Bedrock | Cache checkpoints, model-specific minimums, reduced read rates, and model-specific TTLs. | A prefix below the model minimum may run successfully without being cached. |
| Google Gemini | Context-cache tokens and storage time have separate model-specific prices. | A low request cadence can leave storage cost larger than read savings. |
OpenAI's documented implementation starts automatic caching at prompts longer than 1,024 tokens, uses the longest matching prefix, exposes cached_tokens in usage, and publishes cached-input rates per model. To compare providers fairly, hold prefix length, reuse count, TTL, and output workload constant across all of them.
AWS prompt caching and cached input token pricing
On Bedrock, separate ordinary input, cache writes, cache reads, and TTL storage before calculating any discount. Pricing depends on the selected model and its cache behavior. AWS lists a 1,024-token cache minimum for Claude Sonnet 4.6 and a default 5-minute TTL in its current model table, and because minimums are model-specific, a request below the minimum can succeed without creating a cache. Reuse the prefix enough times before expiry to recover the write and storage charges.
Gemini context cache and storage pricing
Google's pricing page separates context-cache price from storage price and labels changes by model and date, and Google's documentation distinguishes implicit from explicit caching. Treat the cost of cached context as a per-model number rather than one universal rate, on Gemini and on every other provider here.
How to place a cacheable prompt prefix
Order content from most to least stable. System instructions, examples, schemas, and shared tool definitions usually change less often than conversation state. A user message, retrieved result, timestamp, or tenant-specific permission should sit after the cache boundary when the provider's API supports that layout. Where the provider requires an exact match, keep the prefix byte-for-byte stable, then check cached-token usage rather than assuming requests hit the cache.
AWS processes cache checkpoints in an order that can include tools, system content, and messages, and it warns that changing earlier content can invalidate later cache sections. More generally, four things can produce a miss: changed content inside the matched prefix, an incompatible model or cache configuration, a prefix below the minimum length, or a request that arrives after the TTL. Exact matching and expiry behavior differ by provider.
Anthropic's cache controls offer a 5-minute or a 1-hour TTL. The longer option makes sense only when requests arrive often enough to pay for the higher write price; on the Sonnet 4.6 prices above, that means at least two hits per write instead of one. ProjectDiscovery's production write-up describes separate breakpoints for a static system prompt, static tools, and changing conversation context. It is a useful layout pattern, not a universal savings result.
Use one cache key per compatible tenant, model, tool schema, and instruction set. Mixing tenants or stale permissions into a shared prefix creates a correctness problem even when it improves the hit rate.
How to measure prompt caching cost
The cleanest test is the same request shape sent with and without caching. Record one row per provider request:
- Model: model name and service tier.
- Tokens: stable prefix tokens, dynamic input tokens, and output tokens.
- Cache activity: cache-write tokens, cache-read tokens, and the miss reason.
- Cache lifetime: creation time, last hit, expiry time, and TTL.
- Cost: provider-reported cost, plus the price-table version used for any estimate.
- Outcome: request result, latency, and tenant or workload identifier.
Langfuse's cost-tracking docs warn that usage buckets must be mutually exclusive. An OpenAI-style response reports inclusive prompt tokens, which must be split into ordinary and cached input before aggregation. Skip that split and a dashboard can overstate spend while reporting a healthy hit rate.
Keep these numbers apart from semantic caching and RAG metrics. A cache hit can reduce input processing without reducing output tokens, tool calls, or retrieval costs, so measure those paths independently before announcing a percentage reduction. If the cached prefix sits inside an agent loop, roll the result up to AI agent cost per successful task, because a cheaper input that triggers more retries can raise the cost of a finished outcome.
FAQ
Does prompt caching reduce latency?
It can reduce input processing work and may lower time to first token, but the result depends on provider, model, prefix size, request queue, and whether the request actually hit the cache. Measure latency for matched cached and uncached requests. A billing discount is not proof of a latency improvement.
Is prompt caching the same as KV cache?
No. A model's KV cache holds attention state while it serves a token sequence and consumes memory. Provider prompt caching reuses a stable prefix across requests and can change input-token billing and latency. A prompt-cache hit does not tell you how much local GPU memory the model needs; the KV cache size guide covers the memory formula and serving limits.
Related coverage
- Instrumentation: for trace and cache-event tracking, see the Langfuse vs Opik observability comparison.
- Outcome cost: for the rollup from token savings to finished work, see AI agent cost per successful task.
- Self-hosting: for the self-hosted side of the same decision, see GPU cost per million tokens.
References
- Anthropic pricing - https://docs.anthropic.com/en/docs/about-claude/pricing
- AWS Bedrock prompt caching - https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
- Google Gemini pricing - https://ai.google.dev/gemini-api/docs/pricing
- Langfuse cost tracking - https://langfuse.com/docs/observability/features/token-and-cost-tracking
- OpenAI prompt caching - https://openai.com/index/api-prompt-caching/
- ProjectDiscovery caching case - https://projectdiscovery.io/blog/how-we-cut-llm-cost-with-prompt-caching
