FAQ - ngrok documentation

Getting started

What URL do I send requests to?

Send model requests to https://gateway.ngrok.ai. If you are using an OpenAI-compatible SDK, set the base URL to:

https://gateway.ngrok.ai/v1

For example:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx"
});

See the Quickstart for a full example.

Can I use my existing OpenAI SDK?

Yes. The AI Gateway supports OpenAI-compatible requests, so you can usually keep your SDK and change the base URL and API key. Use your ngrok.ai access key as the API key, and set the base URL to https://gateway.ngrok.ai/v1. See SDK Integration for examples.

Do I need a provider account to use ngrok.ai?

Not always. For many providers, including OpenAI, Anthropic, Google, and DeepSeek, you can skip adding your own provider key and pay with credits instead. For other built-in providers, or if you want usage billed to your own provider account, you can add a provider key.

Do I need credits?

Yes. Every request through ngrok.ai require credits for the gateway processing fee. If ngrok.ai supplies the upstream provider credentials, credits also pay the upstream model cost. If you bring your own provider key, the provider bills you for model usage and credits only pay the gateway processing fee. See Credits.

Do I need a ngrok subscription plan?

No, not to use the AI Gateway itself. You can sign in at app.ngrok.ai, purchase AI Gateway credits, create an access key, and send requests to https://gateway.ngrok.ai. You may need a ngrok platform plan for platform features on the same account, such as team members or internal endpoints for custom providers. Manage those features from dashboard.ngrok.com. See Credits. Yes. app.ngrok.ai uses your ngrok account to sign in, so it uses your account’s single sign-on (SAML or OpenID Connect, with providers like Okta). SSO requires the Pay-as-you-go plan with the SSO & RBAC add-on. Set it up on your ngrok account settings. See Dashboard Single Sign-On. Role-based permissions inside app.ngrok.ai aren’t available yet.

Keys and credentials

Which key do I put in my app?

Use an access key. Your app sends the access key to https://gateway.ngrok.ai. Do not put provider keys, ngrok API keys, or AI Gateway API keys in your app’s model requests. See Access keys.

What is the difference between access keys, provider keys, AI Gateway API keys, and ngrok API keys?

Key	Used for
Access key	Authenticates model requests to `gateway.ngrok.ai`
Provider key	Authenticates ngrok.ai to an upstream provider
AI Gateway API key	Automates AI Gateway resources through `api.ngrok.ai`
ngrok API key	Manages ngrok platform resources through `api.ngrok.com`

Most applications only need an access key in runtime code.

What are access keys?

Access keys are what your app uses to authenticate model requests to https://gateway.ngrok.ai. It’s a good idea to create separate access keys for different applications, environments, or teams so you can easily track usage, control access, and revoke keys if needed.

Can I create multiple access keys?

Yes, you can. It’s a good idea to create separate access keys for each app, environment, client, or team. Doing this makes it easier to track usage, revoke access when needed, and apply different access key configurations.

What happens if I lose an access key?

Access key tokens are only shown once when they are created. If you lose one, delete the old access key, create a new one, and update your application with the new token.

Are access keys scoped to endpoints?

No. Access keys are account-scoped and work with https://gateway.ngrok.ai. You do not need to create or manage a custom endpoint URL to use the AI Gateway.

Can I use different keys for different teams?

Yes. Create separate access keys for each team or client. To control what each key can call, assign different access key configurations. Configurations can limit providers, models, and routing credentials.

Do I need provider keys?

It depends on the provider and how you want billing to work. For models you can pay for with credits, you don’t need to add your own provider key. That includes models from OpenAI, Anthropic, Google, DeepSeek, xAI, Moonshot AI, Z.ai, and others. For other built-in providers, or if you want upstream usage billed to your own provider account, add a provider key.

Can I bring my own key for every provider?

Yes. Every built-in provider accepts your own provider key, including OpenAI and Anthropic. ngrok-managed keys, where you pay with ngrok.ai credits instead of a provider account, are available for supported models from OpenAI, Anthropic, Google, DeepSeek, xAI, Moonshot AI, Z.ai, and others. The model catalog shows which providers need your own key. Add provider keys on the Providers page in app.ngrok.ai or with the Provider Keys API. See Bring your own provider key.

Are provider keys exposed to my app?

No. Your app sends an access key to the AI Gateway. Provider keys are stored in app.ngrok.ai and used server-side when ngrok.ai calls the upstream provider. They do not need to be stored in your application code.

Does ngrok.ai provide provider API keys?

No. ngrok.ai does not give you provider API keys to copy or use outside the gateway. For models you can pay for with credits, ngrok.ai supplies the provider credentials. For other providers, or when you want to use your own provider account, add your own provider key.

How do I rotate keys?

For access keys, create a new access key, update your application, then delete the old access key. For provider keys, rotate the stored provider key value so configurations that reference the key can keep using the same key ID. See Bring your own provider key.

Can I manage the AI Gateway with an API?

Yes. The AI Gateway API at https://api.ngrok.ai manages access keys, access key configurations, providers, provider keys, gateway settings, credits, models, and usage. Authenticate with an AI Gateway API key, which you create on the Settings page. The full OpenAPI spec is public, so you can generate a client in any language. See the AI Gateway API reference.

Credits and billing

What do credits pay for?

Credits always pay the AI Gateway processing fee. When you pay with credits, credits also pay the provider’s model cost. If you bring your own provider key, the provider bills you for model usage and credits only pay the gateway processing fee.

How much is the processing fee?

The ngrok.ai processing fee is $0.05 per million tokens.

Do credits expire?

Yes. Credits expire 365 days after purchase.

What happens when credits run out?

The gateway rejects new requests with HTTP 402 and ERR_NGROK_27206 until you purchase more credits from the Billing page. That’s true even if you use your own provider keys, since every request pays a small processing fee from your credits. To avoid this, turn on automatic top-up so your balance refills before it runs out.

What’s the minimum credit purchase?

The minimum credit purchase is $5.00. You can purchase credits from the Billing page. Your balance can’t go above $500.00.

Can credits top up automatically?

Yes. Turn on automatic top-up on the Billing page. When your balance falls to an amount you choose, ngrok.ai charges your saved card and tops it back up. You can also set a maximum to spend each billing period. See Automatic top-up.

Are AI Gateway credits the same as ngrok platform billing?

No. AI Gateway credits are purchased and used in app.ngrok.ai. ngrok platform billing for products managed through dashboard.ngrok.com is separate, and certain features such as team members and internal endpoints for self-hosted models require a ngrok subscription.

Models and providers

Which providers are supported?

The AI Gateway includes built-in support for:

OpenAI
Anthropic
Google
Groq
DeepSeek
OpenRouter
Hyperbolic
InceptionLabs
Inference.net

You can also connect self-hosted or private OpenAI-compatible and Anthropic-compatible endpoints as custom providers. See the Model Catalog for the full list.

Which providers can I call without adding a provider key?

You can see which providers require a provider key in the Model Catalog by checking the “needs provider keys” column. Other built-in providers require you to add your own provider key before sending traffic.

How do I choose a model?

Set the model field in your request. Use a model ID when you want ngrok.ai to resolve the provider:

{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Hello"}]
}

Use provider:model when you want to choose the provider explicitly:

{
  "model": "openai:gpt-4o",
  "messages": [{"role": "user", "content": "Hello"}]
}

See Choose a model.

How do I know which models are available?

Built-in providers and models are listed in the Model Catalog. The catalog includes provider IDs, model IDs, context windows, output token limits, modalities, and provider key requirements. Custom providers declare their own models when you create them.

Can clients use models not in the catalog?

Yes, if the request uses a provider-qualified model name and the provider allows pass-through for that model. For example:

{
  "model": "openai:gpt-5-preview",
  "messages": [{"role": "user", "content": "Hello"}]
}

To restrict an access key to approved models only, configure a model allowlist in the access key configuration.

Can I use self-hosted models?

Yes. Create a custom provider for a model running on your machine, private network, or cloud GPU. See Use a model you run yourself.

What is a custom provider?

A custom provider is an upstream endpoint you define, such as Ollama, vLLM, LM Studio, a private deployment, or another OpenAI-compatible or Anthropic-compatible API. Custom providers let you call models you run yourself through the same gateway URL as built-in providers. See Custom Providers.

Why isn’t my custom model working?

Check that:

The upstream exposes a supported OpenAI or Anthropic API surface.
The custom provider has the correct base URL.
The custom provider declares the model ID you are calling.
The endpoint is reachable from ngrok.ai.
The access key configuration allows the provider and model.
Provider credentials are attached if the upstream requires authentication.

See Use a model you run yourself.

Failover and timeouts

How does failover work?

The gateway tries candidates in order. It can fail over between provider keys for the same provider, then between fallback models listed in the request. The gateway does not retry the same model and provider key combination. It moves to the next available candidate. See Error Handling.

How do I configure fallback models?

Use model for the primary model and models for fallback models.

{
  "model": "openai:gpt-4o",
  "models": ["anthropic:claude-sonnet-4-6"],
  "messages": [{"role": "user", "content": "Hello"}]
}

The gateway tries the primary model first, then each fallback model in order until one succeeds. See Configure fallback models.

Can I set fallbacks on the gateway instead of in each request?

Not yet. Fallback models are set per request, in the models list. On the gateway side today, you can:

Attach multiple provider keys for the same provider, so the gateway fails over between keys on rate limits, quota limits, or auth errors.
Use access key configurations to control which providers and models each access key can reach.

To change a default model for many clients without changing their code, put the model and models values in a shared client wrapper or config your apps already read.

Can the gateway fail over to a different provider?

Yes, if the request includes fallback models from different providers. For example, you can try OpenAI first and fall back to Anthropic:

{
  "model": "openai:gpt-4o",
  "models": ["anthropic:claude-sonnet-4-6"],
  "messages": [{"role": "user", "content": "Hello"}]
}

Each provider and model must be allowed by the access key configuration.

How long does the gateway try before giving up?

By default, each upstream attempt can take up to 60 seconds, and the whole request, including every failover attempt, up to 120 seconds. The gateway makes at most 10 upstream attempts per request.

Bound	Default	Configurable
Per-request timeout (one upstream attempt)	60s	Yes
Total timeout (all attempts)	120s	Yes
Upstream attempts per request	10	No

Change the timeouts in Account Settings or with the gateway configuration API. You can also set account-wide input and output token limits there. If the total timeout is reached, the gateway stops trying candidates and returns an error. See Handle gateway errors.

Can I disable automatic failover?

Not directly. To narrow failover behavior, use a single provider key per provider, send requests with a single model, avoid fallback models, and set shorter timeouts in Account Settings.

Why am I getting rate limited even with multiple keys?

Check that multiple provider keys are attached to the relevant provider in the access key configuration. The gateway only fails over between keys that are available to the request. See Key selection and failover.

Security and privacy

Does ngrok.ai store request and response bodies?

The gateway processes request and response bodies in memory by default and doesn’t store them unless you turn on request capture. To turn on capture, open Settings > Data Retention in app.ngrok.ai and enable Request Capture. Capture stores the request and response headers and bodies for your request and for each attempt the gateway sends to a provider, including streamed responses. Captures are kept for 30 days and then deleted. See Request capture. The gateway always records request metadata for usage events and debugging, such as token counts, latencies, status codes, providers, and models. See Observability for more details.

How long does ngrok.ai keep my data?

Data	Retention
Captured request and response headers and bodies (only if you turn capture on)	30 days
Usage events (request metadata: tokens, latency, status, provider, model, cost)	90 days

Can I see individual request and response bodies?

Yes, if you’ve turned on request capture. Captures are available for 30 days on the Logs page. Without capture, usage records include only metadata such as token counts and latency, not full request or response bodies. See Observability for what is captured.

Can I use the gateway for sensitive data?

Yes, but review your own security and compliance requirements. The AI Gateway doesn’t store request and response bodies unless you turn on capture. If you do, captured bodies are kept for 30 days. Requests still go to the upstream provider you call, so review the data handling terms for any provider you use.

How do I redact PII automatically?

Automatic redaction of request and response bodies isn’t available on ngrok.ai just yet. Captures do redact credential headers, such as Authorization, x-api-key, and cookies. Until then, redact any sensitive data in your application before sending it to the gateway.

How do I secure my gateway when using my own provider keys?

You can store your provider keys safely in app.ngrok.ai, and have your application send only access keys. You can use access key configurations to gently control which providers and models each key can call. If an access key is ever exposed, simply revoke it and create a new one. See Securing Your Gateway.

Monitoring and debugging

How do I view usage and request metrics?

You can check out the Usage page in app.ngrok.ai, or query usage events with the AI Gateway API. Usage events include helpful details like request time, provider, model, token counts, latency, and status. For more, see Observability.

How do I debug which provider was used?

Take a look at usage events in app.ngrok.ai, or query them with the AI Gateway API to see what’s happening behind the scenes. If you want more predictable routing while debugging, try using provider:model in your request, like openai:gpt-4o.

Why are my requests failing silently?

Start with the response status code and error message. Then check:

The access key is valid.
The access key configuration allows the requested provider and model.
The provider key is present if the provider requires one.
The model name is valid.
The account has credits.
The upstream provider is available.

See Debugging and Error Codes.

Does the gateway add latency?

Yes, the gateway can add a small amount of latency for parsing, token counting, routing, and failover decisions. Provider response time usually dominates total latency.

How are tokens counted?

The gateway estimates token counts with tiktoken and uses provider-reported counts when available. Token counts appear in usage events.

Can I cache responses to reduce costs?

Not yet. Response caching isn’t currently available on ngrok.ai, but it’s under consideration for a future release.

Troubleshooting

Why am I getting “provider not allowed”?

The requested provider isn’t currently allowed by the access key configuration tied to your access key. To fix this, you can update the configuration to allow that provider, or send the request using an access key that already has permission to call it. See Access Key Configurations and Error Codes.

Why am I getting “model unknown”?

The gateway could not resolve the model for the request. Common causes include:

The model is not in the catalog.
The model name has a typo.
The request uses a non-catalog model without a provider prefix.
The access key configuration does not allow the model.

For non-catalog models, use the provider:model format:

{
  "model": "openai:gpt-5-preview",
  "messages": [{"role": "user", "content": "Hello"}]
}

Why is my provider key not working?

Check that:

The provider key was added to the correct provider.
The key is valid and has not expired.
The provider account has access to the requested model.
The provider account has enough balance or quota.
The access key configuration routes requests through that provider key.

If usage is not appearing in your provider account, the request may be paid with credits instead of using your provider key.

Why is failover not working?

Start by thinking about the kind of failover you want. If you’re looking for provider key failover, make sure you’ve attached multiple provider keys for the same provider in your access key configuration. If you want model or provider failover, include fallback models in your request:

{
  "model": "openai:gpt-4o",
  "models": ["anthropic:claude-sonnet-4-6"],
  "messages": [{"role": "user", "content": "Hello"}]
}

Every fallback provider and model must be allowed by the access key configuration.

Why is the gateway timing out?

The total request time may be hitting the timeout set in Account Settings. Try increasing the total timeout, checking upstream provider latency, or reducing the number of fallback candidates.

Why was my request rejected before reaching a provider?

The gateway can reject a request before upstream routing when:

The access key is missing or invalid.
The access key configuration does not allow the provider or model.
The model cannot be resolved.
The account has insufficient credits.
Required provider credentials are missing.

Check the response error code and see Error Codes.

Routing behavior

Can I route based on request content?

Not at the gateway layer today. For now, you can use separate access keys and configurations for each client, or handle routing in your application before sending requests to the gateway.

How do I prioritize certain models or providers?

List fallback models in preference order in the request body.

{
  "model": "openai:gpt-4o",
  "models": ["anthropic:claude-sonnet-4-6"],
  "messages": [{"role": "user", "content": "Hello"}]
}

For different clients or teams, create separate access keys with different configurations.

Does the gateway support streaming?

Yes. Streaming is supported, and the gateway forwards SSE streams transparently.

What happens if a stream fails partway through?

A stream that fails partway through is billed, because its first bytes were already delivered to your client. If the provider doesn’t report usage for that request, the gateway estimates token counts, and the usage event is marked usageEstimated: true. To see what happened, look up the request by its Ngrok-AIG-Request-Id with the request attempts endpoint. See Handle gateway errors.

​Getting started

​What URL do I send requests to?

​Can I use my existing OpenAI SDK?

​Do I need a provider account to use ngrok.ai?

​Do I need credits?

​Do I need a ngrok subscription plan?

​Can my team sign in with SSO?

​Keys and credentials

​Which key do I put in my app?

​What is the difference between access keys, provider keys, AI Gateway API keys, and ngrok API keys?

​What are access keys?

​Can I create multiple access keys?

​What happens if I lose an access key?

​Are access keys scoped to endpoints?

​Can I use different keys for different teams?

​Do I need provider keys?

​Can I bring my own key for every provider?

​Are provider keys exposed to my app?

​Does ngrok.ai provide provider API keys?

​How do I rotate keys?

​Can I manage the AI Gateway with an API?

​Credits and billing

​What do credits pay for?

​How much is the processing fee?

​Do credits expire?

​What happens when credits run out?

​What’s the minimum credit purchase?

​Can credits top up automatically?

​Are AI Gateway credits the same as ngrok platform billing?

​Models and providers

​Which providers are supported?

​Which providers can I call without adding a provider key?

​How do I choose a model?

​How do I know which models are available?

​Can clients use models not in the catalog?

​Can I use self-hosted models?

​What is a custom provider?

​Why isn’t my custom model working?

​Failover and timeouts

​How does failover work?

​How do I configure fallback models?

​Can I set fallbacks on the gateway instead of in each request?

​Can the gateway fail over to a different provider?

​How long does the gateway try before giving up?

​Can I disable automatic failover?

​Why am I getting rate limited even with multiple keys?

​Security and privacy

​Does ngrok.ai store request and response bodies?

​How long does ngrok.ai keep my data?

​Can I see individual request and response bodies?

​Can I use the gateway for sensitive data?

​How do I redact PII automatically?

​How do I secure my gateway when using my own provider keys?

​Monitoring and debugging

​How do I view usage and request metrics?

​How do I debug which provider was used?

​Why are my requests failing silently?

​Does the gateway add latency?

​How are tokens counted?

​Can I cache responses to reduce costs?

​Troubleshooting

​Why am I getting “provider not allowed”?

​Why am I getting “model unknown”?

​Why is my provider key not working?

​Why is failover not working?

​Why is the gateway timing out?

​Why was my request rejected before reaching a provider?

​Routing behavior

​Can I route based on request content?

​How do I prioritize certain models or providers?

​Does the gateway support streaming?

​What happens if a stream fails partway through?

​See also