Prompt caching for owners: why the second question costs less

Here’s something that surprises most owners the first time I explain it. When an AI workflow answers a question, the model doesn’t remember your instructions from last time. It reads them again. ALL of them. Every single request.

So if your workflow starts each request with three pages of instructions, a price list, and a policy manual, you’re paying for the model to read those three pages every time someone fills out a form. Prompt caching is the fix for that, and it’s one of the easiest savings there is.

How it works

You mark the part of the request that repeats, usually the instructions and reference material at the top. The first request processes it normally and the provider keeps a cached copy. If the next request starts with exactly the same text, the model picks up from the cache instead of reading it fresh.

On Anthropic’s API, per their documentation (Anthropic):

• Writing to the cache costs 1.25 times the normal input price for the default 5 minute cache, or 2 times for a 1 hour cache.
• Reading from the cache costs about a tenth of the normal input price on most models, and less on some newer ones.
• The default cache lasts 5 minutes, and every hit resets the timer.
• There’s a minimum size, which depends on the model. Short prompts just aren’t cached, no error.

So the first request costs a little more, and every request after it that hits the cache costs a lot less for the repeated part.

Where it helps

Caching pays off when two things are true: a big chunk of each request is identical, and requests come in often enough to keep the cache warm.

• A form-processing workflow that runs dozens of times a day with the same instructions and field definitions.
• A support assistant that carries your whole FAQ and policy text in every request.
• A document workflow that asks several questions about the same long contract, one after another.
• A chat, where each new message re-sends the whole conversation so far.

Our own AI readiness audit caches its instructions for exactly this reason. The instructions are the same for every visitor, the answers are the only part that changes.

Where it doesn’t

• Low volume. If a workflow runs twice a day, the 5 minute cache expires between runs and you pay the write premium for nothing. The 1 hour option can help for bursts, BUT if it’s truly a couple of requests a day, the cost is small enough not to matter anyway.
• Requests that change at the top. Caching matches from the start of the request. If you put today’s date or the customer’s name in the first line, nothing after it can be reused. Put the stable stuff first and the changing stuff last.
• Small prompts. Below the model’s minimum, nothing gets cached.

The one design rule

If you remember one thing from this article, make it this. Order every request from most stable to least stable:

• Instructions and rules.
• Reference material (price list, policies, product details).
• The conversation or document for this specific job.
• The actual question.

That ordering is free, it makes caching work, and it tends to make the prompt easier for a person to maintain too.

How it fits with picking the right model

Caching makes each model cheaper to feed. It doesn’t change which model a job should go to. Those are two separate levers, and the bigger one is usually still model choice, which I covered in Route AI work by how much judgment it needs. After that comes the size of what you send every time, which is its own topic: Context is the cost.

If you’re running an AI workflow and you’re not sure whether caching is turned on, ask whoever built it. It’s usually a small change. If you’d like me to take a look, let me know. Thanks!