How It Works

​

Request flow

When you send a request to https://gateway.ngrok.ai:

1. Your app sends a request with your access key to https://gateway.ngrok.ai
2. The gateway validates the key and loads its access key configuration
3. The gateway resolves the model from your request body
4. Provider credentials are selected from the configuration’s routing rules: a key ngrok.ai supplies when you pay with credits, or your stored provider keys
5. Unsupported parameters are stripped for the chosen model before forwarding, and listed in the Ngrok-AIG-Removed-Params response header
6. The request is forwarded to the upstream provider
7. On failure, the gateway retries with the next provider key or model candidate
8. The response is returned to your app and usage is recorded

Manage keys, configurations, providers, and credits at app.ngrok.ai. Your apps send traffic to gateway.ngrok.ai.

​

Authentication

Every request must include a valid access key. Send it as Authorization: Bearer ng-xxxxx-g1-xxxxx (OpenAI-compatible APIs) or x-api-key (Anthropic native API). Provider keys stay in app.ngrok.ai. The AI Gateway injects those from your configuration after validating the access key.

​

Response headers

Every response from gateway.ngrok.ai carries ngrok’s own headers describing how the gateway handled the request. All five are exposed through CORS, so browser clients can read them. These are ngrok’s headers rather than the provider’s, so a managed-key caller and a Bring Your Own Keys caller see the same values. Provider rate-limit headers behave differently: the gateway forwards those only when the attempt used your own provider key.

​

The model header is authoritative

The gateway passes the response body through byte-for-byte. The model or modelVersion field inside the body is the provider’s own word and is never rewritten to name the model that served the response. On a multi-model failover the body can therefore name a different model than the header. Read Ngrok-AIG-Model when you need to know what answered.

​

Streamed responses

On a streamed response the headers commit with the first flushed byte. The attempt count is final at that point, because the gateway cannot retry once bytes have reached your client.

​

Trace IDs are not a substitute

Ngrok-Trace-Id is emitted only when the trace is sampled, it is not exposed through CORS, and it identifies the whole client request rather than a single attempt. Use Ngrok-AIG-Request-Id to identify a request.

​

Model selection

The gateway determines which model and provider to use from your request. To choose model IDs and provider-qualified model names, see Choose a model.

​

Access key configurations

Configurations define:

• Access scope: allowed providers and models
• Routing rules: pay with credits or use your own provider keys, per provider

When a configuration has no routing rule for a provider, requests to that provider are paid with credits only. Stored provider keys are not used without a rule. An access key without a configuration uses allow-all scope. Built-in models you can pay for with credits use an ngrok.ai key by default when you have credits.

​

Failover

The gateway retries automatically when upstream requests fail.

​

What triggers failover?

• Rate limits (HTTP 429)
• Auth errors (HTTP 401 or 403) because the provider rejected the key
• Timeouts, server errors (HTTP 5xx), and connection failures

Other 4xx errors return to your client right away, because another model would fail the same way.

​

Failover order

What the gateway tries next depends on the error:

• Auth error or rate limit: the next provider key in the routing rule step. When no keys are left and the rule has an ngrok step after the Bring your own API key step, the gateway falls back to paying with credits.
• Timeout, server error, or connection failure: the gateway skips the remaining keys for that provider and moves to the next model.

The next model comes from your request’s models array, or from another provider that serves the same model. See Handle gateway errors for the full order. A routing rule uses the first step that has usable credentials. The gateway doesn’t try later steps, except an ngrok step after a Bring your own API key step. To attach several provider keys and control the order they’re tried, see Multi-key failover.

​

Timeouts and token limits

Configure account-level limits in Account Settings. Streaming responses get more time after the first byte: up to 15 minutes with the default total timeout, as long as the provider keeps sending data at least every 30 seconds. See Streaming responses. Request bodies are limited to 32 MiB.

​

Next steps