# OpenAI Chat Completions Configuration with AI Gateway

Control which providers serve your request, what happens when one fails, and how much of your prompt is cached between calls. For controlling how much a model thinks before answering, see [Reasoning](/content/docs/ai-gateway/sdks-and-apis/openai-chat-completions/reasoning/index.html).

## [Provider options](/content/docs/ai-gateway/sdks-and-apis/openai-chat-completions/advanced#provider-options/index.html)

The AI Gateway can route your requests across multiple AI providers for better reliability and performance. You can control which providers are used and in what order through the `providerOptions` parameter.

### Example request

#### TypeScript
```typescript
import OpenAI from 'openai';

const apiKey = process.env.AI_GATEWAY_API_KEY || process.env.VERCEL_OIDC_TOKEN;

const openai = new OpenAI({
  apiKey,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});

const completion = await openai.chat.completions.create({
  model: 'anthropic/claude-opus-5',
  messages: [
    {
      role: 'user',
      content:
        'Tell me the history of the San Francisco Mission-style burrito in two paragraphs.',
    },
  ],
  stream: false,
  ...{
    providerOptions: {
      gateway: {
        order: ['vertex', 'anthropic'], // Try Vertex AI first, then Anthropic
      },
    },
  },
});

console.log('Assistant:', completion.choices[0].message.content);
console.log('Tokens used:', completion.usage);
```

#### Python
```python
import os
from openai import OpenAI

api_key = os.getenv('AI_GATEWAY_API_KEY') or os.getenv('VERCEL_OIDC_TOKEN')

client = OpenAI(
    api_key=api_key,
    base_url='https://ai-gateway.vercel.sh/v1'
)

completion = client.chat.completions.create(
    model='anthropic/claude-opus-5',
    messages=[
        {
            'role': 'user',
            'content': 'Tell me the history of the San Francisco Mission-style burrito in two paragraphs.'
        }
    ],
    stream=False,
    extra_body={
        'providerOptions': {
            'gateway': {
                'order': ['vertex', 'anthropic']
            }
        }
    }
)

print('Assistant:', completion.choices[0].message.content)
print('Tokens used:', completion.usage)
```

#### cURL
```bash
curl -X POST "https://ai-gateway.vercel.sh/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-opus-5",
    "messages": [
      {
        "role": "user",
        "content": "Tell me the history of the San Francisco Mission-style burrito in two paragraphs."
      }
    ],
    "stream": false,
    "providerOptions": {
      "gateway": {
        "order": [
          "vertex",
          "anthropic"
        ]
      }
    }
  }'
```

Provider routing: In this example, the gateway will first attempt to use Vertex AI to serve the Claude model. If Vertex AI is unavailable or fails, it will fall back to Anthropic. Other providers are still available but will only be used after the specified providers.

## [Model fallbacks](/content/docs/ai-gateway/sdks-and-apis/openai-chat-completions/advanced#model-fallbacks/index.html)

You can specify fallback models that will be tried in order if the primary model fails. There are two ways to do this:

### [Option 1: Direct `models` field](/content/docs/ai-gateway/sdks-and-apis/openai-chat-completions/advanced#option-1:-direct-models-field/index.html)

The simplest way is to use the `models` field directly at the top level of your request:

#### TypeScript
```typescript
import OpenAI from 'openai';

const apiKey = process.env.AI_GATEWAY_API_KEY || process.env.VERCEL_OIDC_TOKEN;

const openai = new OpenAI({
  apiKey,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});

const completion = await openai.chat.completions.create({
  model: 'openai/gpt-6-astra', // Primary model
  ...{
    models: ['anthropic/claude-opus-5', 'google/gemini-3.6-flash'],
  }, // Fallback models 
  messages: [
    {
      role: 'user',
      content: 'Write a haiku about TypeScript.',
    },
  ],
  stream: false,
});

console.log('Assistant:', completion.choices[0].message.content);
// Check which model was actually used
console.log('Model used:', completion.model);
```

### [Option 2: Via provider options](/content/docs/ai-gateway/sdks-and-apis/openai-chat-completions/advanced#option-2:-via-provider-options/index.html)

Alternatively, you can specify model fallbacks through the `providerOptions.gateway.models` field:

#### TypeScript
```typescript
import OpenAI from 'openai';

const apiKey = process.env.AI_GATEWAY_API_KEY || process.env.VERCEL_OIDC_TOKEN;

const openai = new OpenAI({
  apiKey,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});

const completion = await openai.chat.completions.create({
  model: 'openai/gpt-6-astra', // Primary model
  messages: [
    {
      role: 'user',
      content: 'Write a haiku about TypeScript.',
    },
  ],
  stream: false,
  ...{
    providerOptions: {
      gateway: {
        models: ['anthropic/claude-opus-5', 'google/gemini-3.6-flash'], // Fallback models
      },
    },
  },
});

console.log('Assistant:', completion.choices[0].message.content);
console.log('Model used:', completion.model);
```

## [Streaming with provider options](/content/docs/ai-gateway/sdks-and-apis/openai-chat-completions/advanced#streaming-with-provider-options/index.html)

Provider options work with streaming requests as well:

#### TypeScript
```typescript
import OpenAI from 'openai';

const apiKey = process.env.AI_GATEWAY_API_KEY || process.env.VERCEL_OIDC_TOKEN;

const openai = new OpenAI({
  apiKey,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});

const stream = await openai.chat.completions.create({
  model: 'anthropic/claude-opus-5',
  messages: [
    {
      role: 'user',
      content:
        'Tell me the history of the San Francisco Mission-style burrito in two paragraphs.',
    },
  ],
  stream: true,
  ...{
    providerOptions: {
      gateway: {
        order: ['vertex', 'anthropic'],
      },
    },
  },
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}
```

## [Provider timeouts](/content/docs/ai-gateway/sdks-and-apis/openai-chat-completions/advanced#provider-timeouts/index.html)

You can set per-provider timeouts for BYOK credentials to trigger fast failover when a provider is slow to respond. Pass `providerTimeouts` in `providerOptions.gateway`:
```json
"providerOptions": {
  "gateway": {
    "providerTimeouts": {
      "byok": { "anthropic": 3000, "bedrock": 5000 }
    }
  }
}
```

## [Prompt caching](/content/docs/ai-gateway/sdks-and-apis/openai-chat-completions/advanced#prompt-caching/index.html)

Anthropic Claude models support prompt caching, which can significantly reduce costs and latency for repeated prompts. You can enable caching automatically or manually.