Two cache_control segments so agent memory writes don't evict the whole Claude system prompt

A chat agent pays for its whole system prompt on every message — and on every iteration of every tool-use loop. Anthropic's prompt cache cuts cached reads to ~10% of the input price, but it works as a prefix hash: change one byte, and everything from that byte to the end of the cached block is a miss. That interacts badly with agent memory — the moment your agent learns something new about the user, it rewrites its own prompt.

My Telegram coach bot is stateless: each message is a fresh Claude call with instructions, the athlete's profile, zones, goal, long-term facts, and a set of MCP tools; a tool-use loop re-sends all of it 3–5 times per answer.

First win was unrelated to caching: sending all ~55 tool schemas with every "thanks!" was silly, so tools are filtered by message keywords into groups (a core set always included) — about 75% fewer tokens on simple messages.

Then long-term memory arrived and broke caching. With a single cache marker at the end of the system prompt, every save_fact call rewrote the facts block, changed the prefix hash, and evicted everything — including the instruction text that never changes.

The fix is embarrassingly small: two cache_control: ephemeral markers instead of one. Segment one is the static instructions (~780 tokens, byte-identical across users and days). Segment two is the per-user tail (~240 tokens: date, profile, zones, goal, facts, language). A memory write now invalidates only the tail; the static prefix stays hot. Anthropic allows 4 markers per request — we use 2.

The non-obvious part was what we had to refuse to build:

• Top-N fact selection per message (only inject facts relevant to the current question) — rejected. Any per-message selection makes the tail differ per message, so the tail cache never hits. Loading all facts every time is cheaper than loading fewer, differently each time.
• Aggressive tool filtering has the same tension: tools sit above the system prompt in the cached prefix, so a message that pulls in an extra tool group re-reads everything once. Liveable because the default group set covers most messages, and within one message's tool loop the list is constant — and the loop iterations are where the cache pays most.

Net effect: within the 5-minute TTL, loop iterations and follow-up messages read the prompt at ~10% price, and frequent memory writes re-cache ~240 tokens instead of the ~1000+ of the whole prompt.

Reproduce this

Prerequisites: an app calling the Claude API directly with the Anthropic SDK and using prompt caching; a system prompt with a distinguishable invariant part (instructions) and a volatile part (per-user / per-day data, agent memory). Works for any provider with prefix-based caching — the marker syntax below is Anthropic-specific.

Your agent will need from you: your current system-prompt assembly code, and a list of which parts change per user, per day, or per memory write.

Steps:

1. Split system into an array of two text blocks, each with cache_control: {"type": "ephemeral"}: block one = instructions, guaranteed byte-stable (no interpolated dates, names, or counters); block two = everything volatile (date, profile, memory), ordered rarest-changing first for readability — after the marker it's one hash anyway.
2. Keep everything above the system prompt in the request — the tools array most of all — byte-stable across calls; if you must vary tools per message, make the default selection cover the common case, and never reorder or reword tool schemas casually.
3. Resist per-message personalization inside cached blocks (top-N memory selection, "relevant context only") — selection defeats the cache; include-everything usually costs less.
4. Read usage in every response: cache_creation_input_tokens and cache_read_input_tokens tell you exactly what happened.

Verify: send two messages within 5 minutes — the second must show cache_read_input_tokens covering roughly the whole prompt. Then trigger a memory write and send a third message: cache_read must still cover the static block, and cache_creation must be roughly the size of your volatile tail only. If a memory write causes cache_creation the size of the full prompt, your static block isn't byte-stable — hunt the interpolated value.