An agent doesn't send the model one prompt. It sends the same prompt again on every turn: its instructions, the definitions of its tools and the conversation so far, with one new piece on the end. An agent that calls four tools talks to the model five times, and pays for the start of the prompt five times.
Model providers know this, and keep recently used prompts in a cache. Reading from the cache costs about a tenth of the normal input price. OpenAI does it on its own once a prompt passes about 1,000 tokens. Claude, and Gemini through OpenRouter, only cache up to a marker in the request.
Until this week, Cruq AI didn't send that marker. Every turn of every Claude agent paid full price for text the provider had seen seconds earlier.
Where the markers go
We now add two.
After the agent's own instructions. These are the same on every run of the agent, so a second run within five minutes starts with them already cached. Anything that changes from run to run, like the run's context or the agent's plan, goes after this marker. If it went before, every run would look new and nothing would be reused.
After the latest message, on agents that use tools. Each turn of the loop then reads everything up to the previous turn from the cache, and pays full price only for the new tool result.
Agents without tools get the first marker only. They answer in one turn, and writing a cache entry costs a little more than a normal read (1.25x on Claude), so a cache nobody reads again is a small loss.
What it saved
We ran the same agent three times: Claude Sonnet 4.5, a pricing desk with a calculator tool, asked to work out an order total one step at a time. Six model calls per run, about 3,000 input tokens each, and the same answer every time.
| Run | Cost |
|---|---|
| Without markers | $0.063 |
| With markers, first run | $0.024 |
| With markers, run again within 5 minutes | $0.016 |
That's 62% less on a first run and 74% less on a repeat. On the first run, the first call writes about 2,800 tokens to the cache. Each later call reads about 3,000 from it and writes only the 90 or so tokens that are new.
Because you bring your own provider key, the saving lands on your provider bill, not ours.
What you need to do
Nothing. It applies to every Claude and Gemini agent reached through OpenRouter. OpenAI models were already caching on their own.
How much you save depends on three things:
- Short prompts don't cache. Most Claude models need about 1,000 tokens before they cache anything. A tool loop passes that quickly; a two-line agent asked one question may not.
- The cache lasts five minutes, and each read resets the clock. Busy agents stay warm; one that runs once a day starts cold each time.
- Keep the instructions stable. If an agent's instructions start with today's date or a customer's name, every run is different from the first word, and nothing is reused. Pass details like that as run context instead. It goes after the cached part.
Seeing it
Every model call is now recorded on the run, with its tokens, cost and time. The run detail page in AI Monitoring shows how much of each call's input came from the cache, for example "3.1k in (97% cached)". The CLI shows the same with cruqai trace <run-id> --models, and the models docs have the details.
If you run agents on Cruq AI and want help reading your numbers, get in touch.
