Configure provider key failover

Configure multiple provider keys in an access key configuration routing rule. The gateway tries keys in order when one hits a rate limit, runs out of quota, or fails authentication.

How failover works

When a routing rule step lists multiple provider keys:

The gateway tries the first key.
On a rate limit (429), or an auth failure (401 or 403) because the provider rejected the key, it tries the next key.
When no keys are left, the gateway falls back to paying with credits if the rule has an ngrok step after this one. Otherwise it moves to the next model you listed, or returns the last error.

Server errors (5xx), timeouts, and connection failures don’t try the next key. Another key for the same provider would likely fail the same way, so the gateway moves straight to the next model you listed. To fail over on those errors, add fallback models. Failover is configured per provider in the access key configuration—not in your application code.

Via dashboard

Add multiple provider keys for the same provider.
Edit an access key configuration.
Add or edit a routing rule for the provider.
Attach multiple keys to the Bring your own API key step in the order you want them tried.

Via the AI Gateway API

curl -X POST https://api.ngrok.ai/access-key-configurations \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "OpenAI with failover",
    "router": {
      "rules": [
        {
          "provider": "openai",
          "steps": [
            {
              "type": "user",
              "keySelectionStrategy": "ordered",
              "keys": [
                { "id": "aigpk_primary" },
                { "id": "aigpk_backup" }
              ]
            }
          ]
        }
      ]
    }
  }'

See the Access Key Configurations API reference. Assign the configuration to your access key. Your app sends only the access key:

client = OpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",
)

Next steps

Bring your own provider key: Store upstream credentials
Access Key Configurations: Routing rules and scope
How It Works: Full request flow

Concepts

Getting Started With

SDKs

Agents

Guides

Observability

Examples

Reference

API Reference

How failover works

Via dashboard

Via the AI Gateway API

Next steps

​How failover works

​Via dashboard

​Via the AI Gateway API

​Next steps

How failover works

Via dashboard

Via the AI Gateway API

Next steps