Prompt caching

You're building a document Q&A app. A user uploads a 40-page contract and asks three questions about it in a row. Under the default API behaviour, Claude re-tokenises, re-embeds, and re-contextualises that entire contract on every single call — doing identical preprocessing work three times and making you pay for it three times. Prompt caching is the feature that says "don't do that."

What happens on a normal request

When you call client.messages.create(), the API doesn't immediately start generating text. It first does a large amount of preprocessing on your input:

1. Tokenisation — split the prompt into tokens.
2. Embedding — convert each token into an internal vector.
3. Contextualisation — refine the embeddings based on surrounding content.
4. Generation — finally, produce the output token by token.

The first three steps are expensive. They're also completely deterministic — give the model the same input and you get the same preprocessing.

And then, by default, the API throws all that work away. Your next request starts from scratch, even if 95% of the input is identical to the request you just made.

Where this starts to hurt

Imagine a conversation where a user is refining a summary of the same long document:

• Request 1: "Here's the document. Summarise it in three paragraphs." → Claude preprocesses the document, generates a summary.
• Request 2: "Make the summary shorter." → Claude re-preprocesses the exact same document from scratch to change one sentence.
• Request 3: "Now make it less formal." → Again, the same preprocessing on the same 6,000 tokens of contract.

Every request repeats the expensive work. Claude, if it could speak, would be saying: "I literally just processed that document twenty seconds ago — I could have kept the results." Without prompt caching, you pay for that redundant work on every call.

What prompt caching does

Prompt caching stores the preprocessing results in a short-lived cache so follow-up requests with the same content can skip the redundant work. The first request writes the cache. Subsequent requests read from it. Same model, same answer, fraction of the cost and latency.

The cache is like a lookup table keyed on the exact content of the prompt. If the next request has the same content at the start, Claude reuses the cached preprocessing. If the content differs by even a single character, the cache doesn't match and the full preprocessing runs again.

The benefits

• Faster responses. Requests that hit the cache skip the preprocessing step entirely, so they return noticeably faster — especially for long prompts where preprocessing dominates latency.
• Lower costs. Cached tokens are billed at a fraction of the normal input rate. For workloads with long repeated prompts (document analysis, agent loops with big system prompts, code review tools), the savings can be 80% or more of your input token cost.
• Automatic optimisation. Once you enable caching, the initial request writes, every follow-up read. You don't have to manually manage the cache contents — the API handles hits and misses for you.

The limitations

Three constraints to know up front, so you can evaluate whether caching will actually help you:

• Cache duration is one hour. Content that hasn't been hit in the last hour gets evicted. Prompt caching is designed for workloads with frequent requests, not long-term storage.
• Content has to be literally identical up to the cache boundary. Adding a word at the beginning of a cached prompt invalidates the whole cache. (The next lesson covers the specific rules in detail.)
• Caching only pays off with repeated content. Sending unique prompts every time means every request is a cache miss — you pay the overhead of writing to cache with no reuse benefit.

When prompt caching is the right call

The sweet spot is any workload with frequent requests sharing a large common prefix. Specifically:

• Document analysis. Ask multiple questions about the same 40-page PDF within an hour. First question writes the cache; subsequent ones read from it.
• Chat applications with large system prompts. A coding assistant with a 6,000-token system prompt benefits enormously because that prompt is identical on every turn.
• Agent loops with large tool definitions. Tool schemas can be thousands of tokens and they don't change between turns — cache them once, reuse them on every iteration of the loop.
• Iterative editing. A user working on the same draft across multiple turns — the base content stays constant, only small deltas change at the end.
• Few-shot examples in prompts. Long example blocks at the top of a prompt are natural cache targets because they rarely change between calls.

When it's not worth turning on

• Low-volume workloads. If you make fewer than a handful of requests per hour, caching has no opportunity to pay off.
• Every request is unique. User-generated content with no shared prefix, one-shot classifications with fresh input each time — no reuse, no savings.
• Short prompts. There's a minimum size (covered in the next lesson) below which caching doesn't activate.

The default question for any high-volume Claude workload should be "can I cache the repeated portion?" In the next lesson, you'll see exactly how to mark the parts of a prompt you want cached — cache breakpoints, the cache_control field, the minimum token threshold, and the order in which the API processes everything. The mechanics are simple once you see them; getting them right is how you turn a 10x cost reduction from a theoretical benefit into a real line item on your bill.