Eight of ten requests hit the prompt cache. That does not tell you whether 80% of your input tokens were served from it. A hit might reuse 1,000 tokens out of a 100,000-token request. A miss might rewrite the longest prefix of the day.
Request hit rate counts requests. The bill counts tokens in pricing categories. Start with those categories.
A four-term bill
For a provider that reports these categories separately, let:
U = uncached input tokens
W = cache-write tokens
R = cache-read tokens
O = output tokens
cost = (U × input_rate + W × write_rate
+ R × read_rate + O × output_rate) / 1,000,000Rates in that equation are currency per million tokens. Additional provider charges, such as tool usage, belong in additional terms. If a provider’s usage schema includes cached tokens inside total input rather than alongside it, normalize the categories before summing. Otherwise the calculation double-counts input.
For Anthropic’s API, the prompt-caching documentation distinguishes input_tokens, cache_creation_input_tokens, and cache_read_input_tokens. Split writes by TTL when different write rates apply. Check the actual model, platform and tier rather than assuming one discount applies everywhere.
Ten requests with an 80% hit rate
Suppose each request has an 80,000-token stable prefix and 20,000 other input tokens. Two requests write the prefix; eight read it. Across the batch:
| Category | Tokens |
|---|---|
| Uncached input | 200,000 |
| Cache writes | 160,000 |
| Cache reads | 640,000 |
| Total input across categories | 1,000,000 |
The request hit rate is 80%. The fraction of input tokens read from cache is 64%.
Use hypothetical rates of $10 for uncached input, $12.50 for writes and $1 for reads per million tokens. The batch’s input cost is $2 + $2 + $0.64 = $4.64, compared with $10 if all input used the base rate. That is 53.6% input-cost savings. Output costs are additional and unchanged by this example.
These round rates illustrate accounting; they are not a current model price quote. The Python fixture checks the arithmetic with decimal numbers and makes no provider calls.
Prefix stability is a dependency graph
Anthropic’s cache prefix follows tools, then system content, then messages. A changing timestamp near the beginning can invalidate reuse of everything after it. A tool-schema edit can have a larger consequence than its character count suggests.
A stable section also needs a supported cache breakpoint. The documentation describes explicit writes at breakpoints and a bounded lookback for matching previous entries; stable text alone does not guarantee that an entry was created at every useful position. Default TTL is five minutes, and minimum cacheable lengths depend on the model and platform.
Move truly volatile information after the reusable section when that preserves meaning. Do not freeze security policy, current permissions or time-sensitive facts merely to preserve cache hits. Correct context is a constraint on the optimization.
Parallel calls can start cold together
If five workers fork the same long prompt before the first response has begun, they cannot all assume the first request has already populated a reusable cache entry. Anthropic documents that cache availability begins after the first response starts.
Measure cold fan-out and warm fan-out separately. Parallel work can reduce wall time while increasing total output tokens, repeated reasoning and tool calls. A shared cached prefix discounts only part of that bill.
Instrument the change you can explain
For each request, record model/version, pricing tier, prefix version, TTL, the four token categories, latency and outcome. Aggregate token counts before calculating ratios. An average of per-request cache percentages can overweight tiny requests.
Then compare before and after a prompt or tool-schema change. Did write tokens rise? Did read tokens fall? Did task success remain stable? Did lower latency come from caching, shorter output, or a different model?
The revised Cost of Context uses this accounting to compare long context and retrieval. The prompt-cache architecture chapter connects it to stable-prefix design. A good cache dashboard should explain the bill in those terms, not just celebrate a green hit-rate number.