AI Gateway Automatic Prompt Caching

Some providers like Anthropic and MiniMax require explicit cache control markers to enable prompt caching, while others like OpenAI, Google, and DeepSeek cache automatically (sometimes called "implicit caching"). Use caching: 'auto' to let AI Gateway handle this for you. It applies the appropriate caching strategy based on the provider.

Supported providers: Automatic caching works with Anthropic (direct, Vertex, and Bedrock), MiniMax, and Alibaba.

How it works

When you set caching: 'auto' and the request routes to a provider that requires explicit cache markers, AI Gateway adds cache_control breakpoints to your messages. This covers Anthropic, MiniMax, and Alibaba. AI Gateway uses the cache markers expected by each provider. For providers with implicit caching (OpenAI, Google, DeepSeek), no modification is needed and caching works automatically.

For explicit-caching providers, AI Gateway places markers at two positions:

• On the last message. Each request writes a cache entry covering the full prompt, and the next request in the conversation reads that entry as a prefix, paying full price only for what it appends. Multi-turn and agentic traffic caches well because of this marker: tool-use loops and agent conversations extend the previous request instead of repeating a static prefix.
• On the message before the last user message (falling back to the system message). A request whose tail changed still reads the cache up to this stable prefix.

You can add a third marker with a cache anchor.

Default behavior: When caching is not set, AI Gateway passes your request through without modification. Providers with implicit caching still cache automatically. For Anthropic, you'll need to set caching: 'auto' or manually add cache markers to your messages.

Cache lifetime

Anthropic cache entries expire after five minutes by default. If your agentic workflow pauses for longer than five minutes, you can request a one-hour lifetime with the cache_ttl field on the Responses API:

body: JSON.stringify({
  model: 'anthropic/claude-sonnet-5',
  caching: 'auto',
  cache_ttl: '1h',
  input: 'Review this codebase and suggest improvements.',
})

The field accepts 5m (five minutes) or 1h (one hour). AI Gateway applies the selected lifetime to every automatic breakpoint, including a breakpoint created by cache_anchor_items. The field has no effect unless you also set caching: 'auto'.

Anthropic charges different cache-write rates for each lifetime:

A one-hour entry costs an additional 0.75× base input to write. One read after the five-minute entry would have expired saves 0.9× base input and repays that premium. Use one hour for long-running agents and conversations that commonly pause for more than five minutes. Keep the five-minute default when requests normally continue within five minutes.

cache_ttl is advisory. AI Gateway uses the default lifetime for unsupported values and never rejects the request because of the hint. The response includes an unsupported warning in provider_metadata.gateway.warnings:

{
  "provider_metadata": {
    "gateway": {
      "warnings": [
        {
          "type": "unsupported",
          "feature": "cache_ttl",
          "details": "Unsupported cache_ttl; the default cache lifetime was used."
        }
      ]
    }
  }
}

For streaming requests, the warning appears on the terminal response.completed or response.incomplete event.

Cache anchor

The two default markers assume the conversation only grows at the tail. Agentic clients often rewrite the middle of the prompt instead: summarizing older turns, pruning tool output, or compacting history. When that happens, both default markers land after the mutation point and the cache read misses everything.

If your client knows how much of the prompt is stable, it can say so with the cache_anchor_items field on the Responses API:

body: JSON.stringify({
  model: 'anthropic/claude-sonnet-5',
  caching: 'auto',
  cache_anchor_items: 12,
  input: [
    /* the first 12 input items are byte-stable across future requests */
  ],
})

cache_anchor_items: N declares that the first N items of input will be sent byte-for-byte identically on future requests. AI Gateway places an additional cache marker at that position, so a request that later mutates the prompt after the anchor still reads the cache up to it.

The anchor is advisory. Values that can't be used (not a positive integer, past the end of the prompt, or colliding with a marker that's already placed) are ignored, and caching degrades to the two-marker behavior above. An unusable anchor never causes a request to fail.

The anchor only applies when caching: 'auto' is set, and like the other markers it only modifies requests to explicit-caching providers.

Improve cache hits with cache affinity

Send the same x-session-affinity header with related requests so AI Gateway can forward it to the selected provider. Providers that support session affinity can use the value to preserve prompt-cache locality.

Use an opaque, stable value for each conversation, agent run, or related workflow. Don't include personal data in the value or reuse one value across unrelated sessions. The header does not affect your routing configuration or which provider AI Gateway selects, and it does not guarantee a cache hit.

Add cache affinity with AI SDK

Pass x-session-affinity to each AI SDK call with the headers option:

import { streamText } from 'ai';

const result = streamText({
  model: 'deepseek/deepseek-v4-flash-0731',
  prompt: 'Hello world',
  headers: {
    'x-session-affinity': 'session_123',
  },
});

Add cache affinity to compatibility API requests

AI Gateway forwards x-session-affinity through its OpenAI-compatible, Anthropic-compatible, and OpenResponses API formats. Set it as a default SDK client header or include it directly in each HTTP request:

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
  defaultHeaders: {
    'x-session-affinity': 'session_123',
  },
});

const response = await openai.chat.completions.create({
  model: 'deepseek/deepseek-v4-flash-0731',
  messages: [{ role: 'user', content: 'Hello world' }],
});

Cost tradeoff

On Anthropic, cache writes cost 1.25× the base input rate and cache reads cost 0.1×. The break-even is a single read: one follow-up request that reuses the cached prompt saves 0.9× per token read, more than offsetting the 0.25× write premium. Multi-turn conversations, agents, and tool-use loops re-read everything the previous turn wrote on every turn, so they come out well ahead. For true one-shot requests, where no follow-up ever reads the cache entry, the write premium is a small net cost. If your traffic is strictly one-shot, prefer manual cache markers (or no caching) over caching: 'auto'.

To restrict routing to only models that cache automatically (implicit caching), use has: ['implicit-caching'].

Examples

Use a stable system prompt long enough to meet the model's cache threshold. For a self-contained test, create a synthetic reference file:

awk 'BEGIN { for (i = 1; i <= 400; i++) print "Deployment check " i ": Production deployments require two approving reviewers and all automated checks to pass before release." }' > deployment-runbook.txt

The AI SDK does not return a separate cache-hit boolean. A positive cache-read token count confirms that the provider read part of the prompt from cache. A zero count can mean a cache miss, an ineligible prompt, or unavailable provider usage details.

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

Manual caching

For fine-grained control over what gets cached, you can manually add cache markers instead of using caching: 'auto'. This gives you control over exactly which parts of your prompt are cached.

• Anthropic Messages API: Add cache_control: { type: 'ephemeral' } to specific messages.
• AI SDK: Use the cacheControl property on messages.
• OpenAI Chat Completions API: See prompt caching in the advanced guide.

Provider behavior

Caching in virtual models

Pin caching: 'auto' on a virtual model to cache repeated prompt prefixes for its traffic, or set it to @default to drop the request's choice and apply the system default.