Cached Input Tokens: Mechanics, Break-Evens & Provider Rules
Prompt caching is the single largest cost-reduction lever in modern AI development. When your prompt shares an identical prefix with previous queries, providers skip transformer re-computation and discount your input rate by up to 90%.
1. What Exactly Is a Cached Input Token?
In traditional stateless API calls, every request forces the inference server to compute attention matrices from scratch for the entire system prompt, tool definitions, and few-shot examples.
With Prompt Caching, the server stores the precomputed Key-Value (KV) activations in GPU VRAM or fast host memory. When a new request arrives with a matching prefix, the server loads the precomputed KV tensors instantly, avoiding expensive matrix multiplications and slashing input latency to milliseconds.
2. How Caching Differs Across the Major AI Providers
Explicit caching via `cache_control: {"type": "ephemeral"}`. Minimum 1,024 tokens. Cache writes incur a 1.25x surcharge on the first call; cache reads receive a 90% discount. Cache TTL is 5 minutes, automatically refreshed upon each hit.
Gemini 3.8 Flash offers completely free cache reads ($0.00/1M). Long-running caches incur an active storage fee of $0.50/1M tokens/hour (promotional rate through 2026).
Automatic prefix caching on prompts >= 1,024 tokens. No write surcharge. The server automatically routes requests to cluster nodes holding matching cached prefixes.
Automatic prefix caching at 64-token chunk granularities. DeepSeek V4.1 Flash drops from $0.14/1M to an industry-record $0.014/1M on cached reads.
3. The Mathematical Break-Even Formula
When a provider charges a write surcharge (like Anthropic's 1.25x), prompt caching is an investment: you pay slightly more on request 1 to save 90% on requests 2 through $N$.
The minimum number of cache hits required to achieve net financial savings is:
For Claude Sonnet 5 ($2.00 in, $2.50 write, $0.20 read), break-even occurs at exactly 1.28 requests. In other words, if your cached prefix is read even twice within its 5-minute window, caching produces immediate net profit!
Model Your Break-Even Point
Simulate cache hit rates and compute net monthly ROI.