Prompt caching in action

You know what prompt caching does and you know the rules. This lesson shows you what it looks like in code — how to add breakpoints to tool schemas and system prompts without mutating your source data, how to read the cache hit/miss signal from the response, and what happens when you change part of your request while the rest stays stable.

Where the savings live

In a real production application, two parts of a request are almost always good caching candidates:

• Tool definitions — for a Claude-powered agent with a handful of tools, the combined schemas often run 1.5K–3K tokens. They never change between turns of a conversation.
• System prompts — a well-engineered coding-assistant or customer-support system prompt can easily hit 5K–10K tokens. It might be edited once a week; it runs on every single request.

Caching these two things alone, in applications that hit them frequently, often produces the bulk of the savings prompt caching can deliver. Everything else is usually rounding noise.

Caching tool schemas — without mutating them

Cache breakpoints go on the last tool in the list. The whole list before that point gets cached as a unit. The key is to do this without modifying the original tools list you built when defining the schemas — if you mutate it, you'll accidentally mark schemas in other code paths too, and silly bugs follow.

Do the copy-then-add dance inside your chat() helper:

if tools:
    tools_clone = tools.copy()              # shallow copy of the list
    last_tool = tools_clone[-1].copy()      # shallow copy of the last schema
    last_tool["cache_control"] = {"type": "ephemeral"}
    tools_clone[-1] = last_tool
    params["tools"] = tools_clone

Two copies are important:

• tools.copy() — a new list, so when you replace the last element you're not reaching into the original.
• tools_clone[-1].copy() — a new dict for the last tool, so adding cache_control doesn't mutate the source schema.

You could write tools[-1]["cache_control"] = {"type": "ephemeral"} and skip the copies. Don't. It mutates your original schemas in place, and the next time you reorder your tools or use them in a different code path without caching, you'll be quietly caching the wrong schema. Copy, then modify. Always.

Caching system prompts

System prompts are normally passed as a plain string: params["system"] = "You are a helpful assistant...". Plain strings don't have a place to put a cache_control field. To cache a system prompt, you convert it to a structured text block:

if system:
    params["system"] = [
        {
            "type": "text",
            "text": system,
            "cache_control": {"type": "ephemeral"},
        }
    ]

The string becomes a list containing one text block, with the cache control field attached. From Claude's perspective, it's the same system prompt — just expressed in longhand so the caching hook has somewhere to live.

Reading the response usage object

Every response includes a usage field that now carries more detail than "input tokens" and "output tokens." When caching is involved, two new fields matter:

• cache_creation_input_tokens — the number of input tokens Claude wrote to the cache on this request. Nonzero on the first request (and on any request where the cache content changed).
• cache_read_input_tokens — the number of input tokens Claude read from the cache on this request. Nonzero on cache hits.

A typical caching session looks like:

First request — writing the cache:

usage.input_tokens              = 0
usage.cache_creation_input_tokens = 1772
usage.cache_read_input_tokens     = 0

Claude preprocessed 1,772 tokens and wrote them to the cache. You pay the full price for this.

Second request — cache hit:

usage.input_tokens              = 45
usage.cache_creation_input_tokens = 0
usage.cache_read_input_tokens     = 1772

Claude read the 1,772 cached tokens instead of reprocessing them, and processed 45 new tokens (the new user message). You pay roughly 10% of the normal rate for the cached portion and full rate for the 45 new tokens.

Third request, after the system prompt changed:

usage.input_tokens              = 45
usage.cache_creation_input_tokens = 6234
usage.cache_read_input_tokens     = 0

Cache miss. Full preprocessing and a fresh cache write. The new size reflects the edited system prompt.

Check these fields in development. If you enabled caching and expect to see reads but you're still seeing creates on every request, something in your "stable" content is subtly changing between calls — often trailing whitespace, an f-string interpolation that varies, or a timestamp slipped into a supposedly-static prompt.

Cache sensitivity is absolute

Any character change invalidates the whole cache for that component. This is worth repeating because it's the source of almost every "I enabled caching and nothing changed" bug:

• Changing "You are a helpful assistant" to "You are a helpful, polite assistant" — cache miss.
• Inserting a newline between two paragraphs — cache miss.
• A dynamic timestamp interpolated into the system prompt — cache miss every call.
• A user's name or session ID hidden in the system prompt — cache miss every call.

The fix for dynamic content is either move it out of the cached component into a later one that isn't cached, or eliminate it entirely by handling the variability elsewhere. If your system prompt includes "Today is 2024-05-22," take the date out of the system prompt and put it in the first user message instead — now the system prompt is stable and cache-hit-eligible.

Partial cache reads across the layers

The four-layer cache ordering (tools → system → messages) means you can change one layer without invalidating the others. Say you:

1. Cache your tools (breakpoint on last tool).
2. Cache your system prompt (breakpoint on system).
3. Cache partway into the conversation history.

Now if you change only the system prompt, you'll see:

• Tools: cache hit (they didn't change).
• System: cache write (new content).
• Conversation history: cache write (because the earlier cache miss cascades forward).

If instead you change only the latest user message, you'll see:

• Tools: cache hit.
• System: cache hit.
• Conversation history up to breakpoint: cache hit.
• New user message: fresh processing.

This is the ideal case for caching — the most commonly changing thing (the latest message) is the only thing that isn't cached, and everything else rides free. In a high-volume agent, you can drive input token cost down dramatically.

Practical heuristics

• Always cache tools and system prompts if they're above the 1,024-token threshold and stable.
• Never interpolate dynamic content into cached layers — no timestamps, session IDs, or user-specific data inside the system prompt.
• Check usage during development to confirm you're actually getting cache hits.
• Put caching behind a feature flag so you can disable it for debugging without rewriting the calling code.
• Remember the one-hour TTL — caching helps sustained high-frequency workloads, not once-a-day batch jobs.

Prompt caching is one of the highest-leverage optimisations in the Claude API. The mechanics are simple, the savings are real, and the common pitfalls are all about treating "stable content" more carefully than you would if nobody were watching.