AI Gateway Provider Filtering, Ordering, and Sorting

By default, AI Gateway dynamically chooses providers based on recent uptime and latency. You can override this behavior to control which providers handle your requests and in what order using order, only, and sort in providerOptions.gateway. To instead filter by a capability of the model itself, see Model Filtering.

Provider ordering

Use the order array to specify the sequence in which providers should be attempted. Provider ordering works with a model served by more than one provider, such as anthropic/claude-sonnet-5.

Providers are specified using their slug string. You can find the slugs in the table of available providers. To find which providers serve a model programmatically, get its provider endpoints.

You can also copy the provider slug using the copy button next to a provider's name on a model's detail page:

Through the Vercel Dashboard:

1. Click the AI Gateway tab
2. Click Model List on the left
3. Click a model entry in the list

Through the AI Gateway site: Visit a model's page on the AI Gateway models page (e.g., Claude Sonnet 5). The bottom section of the page lists the available providers for that model. The copy button next to a provider's name will copy their slug for pasting.

Getting started

1. Install the AI SDK package

First, ensure you have the necessary package installed:

pnpm install ai@latest

2. Configure the provider order in your request

Use the providerOptions.gateway.order configuration:

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

TypeScript Example

import { streamText } from 'ai';

const result = streamText({
     model: 'anthropic/claude-sonnet-5',
     prompt: 'Explain quantum computing in two sentences.',
     providerOptions: {
       gateway: {
         order: ['bedrock', 'anthropic'],
       },
     },
});

for await (const chunk of result.textStream) {
     process.stdout.write(chunk);
}

Python Example

import asyncio
import ai

async def main():
       model = ai.get_model("anthropic/claude-sonnet-5")
       messages = [ai.user_message("Explain quantum computing in two sentences.")]
       params = ai.InferenceRequestParams(
           extra_body={"providerOptions": {"gateway": {"order": ["bedrock", "anthropic"]}}}
       )
       async with ai.stream(model, messages, params=params) as stream:
           async for event in stream:
               if isinstance(event, ai.events.TextDelta):
                   print(event.chunk, end="", flush=True)
       print()

asyncio.run(main())

Test the routing behavior

Read providerMetadata.gateway.routing.finalProvider to print the provider that served the response:

import { generateText } from 'ai';

const result = await generateText({
     model: 'anthropic/claude-sonnet-5',
     prompt: 'Explain quantum computing in two sentences.',
     providerOptions: {
       gateway: {
         order: ['bedrock', 'anthropic'],
       },
     },
});

const gatewayMetadata = result.providerMetadata?.gateway as
     | { routing?: { finalProvider?: string } }
     | undefined;

console.log(result.text);
console.log('Provider:', gatewayMetadata?.routing?.finalProvider ?? 'unknown');

Provider filtering

Restrict providers with the only filter

Use the only array to restrict routing to a specific subset of providers. Providers are specified by their slug and are matched against the model's available providers.

Using only together with order

When both only and order are provided, the only filter is applied first to define the allowed set, and then order defines the priority within that filtered set. Practically, the end result is the same as taking your order list and intersecting it with the only list.

Provider sorting

Use the sort option to rank providers by a performance or cost metric. The gateway sorts the available providers by the chosen metric and tries them in that order, falling back through the list if a provider fails.

Quick reference

All options are set under providerOptions.gateway in the AI SDK, or under providerOptions in the REST API / OpenAI-compatible Chat Completions API. The Chat Completions API also accepts a top-level provider shorthand (e.g., "provider": { "sort": "tps" }).