Skip to main content
​
General Availability
Automatic top-up

​
Never run out of credits

Turn on automatic top-up and ngrok.ai refills your balance from your saved card whenever it runs low. You choose when it tops up and how much, and you can cap how much it spends each billing period. See Automatic top-up.
​
General Availability
Card approvals and a request size limit

​
Approve card payments that need an extra check

Some banks ask you to confirm a payment before it goes through. When that happens while you buy credits, you now get a link to approve the payment, and your credits arrive once you do. See Credits.

​
Requests can be up to 32 MiB

Requests larger than 32 MiB are now turned away with a clear error, before they reach a provider. Most requests are far below this. If you hit it, trim long conversation history or large embedded documents.
​
General Availability
See what was left out of your request

​
Know when a setting didn’t reach the model

Not every model supports every setting. When the gateway has to leave a setting out because the model can’t use it, the response now tells you which ones. See Response headers.
​
General Availability
Longer streaming responses

​
Long answers no longer get cut off

Streaming responses can now keep going for up to 15 minutes, which helps with long answers and slow reasoning models. If you’ve set your own total timeout, it still applies. See Streaming responses.
​
General Availability
Pay with credits for more providers

​
Use more providers without signing up for them

You can now pay with credits for models from Google, DeepSeek, xAI, Moonshot AI, Z.ai, Qwen Cloud, MiniMax, and more, in addition to OpenAI and Anthropic. You don’t need an account or API key with those providers. The model catalog shows which models work this way.
​
General Availability
Easier request troubleshooting

​
See why a fallback model didn’t run

If a model you listed as a fallback couldn’t be used, for example because your access key isn’t allowed to call it, the response now names it. See Response headers.

​
Follow every retry for a request

You can now look up a single request and see each attempt the gateway made, what happened, and why it moved on. See Why a request failed over.
​
General Availability
Response headers

​
Request ID, attempt count, and served model on every response

Gateway responses now state how the request was handled:
  • Ngrok-AIG-Request-Id: the request’s ID, on every response including rejections and errors.
  • Ngrok-AIG-Attempts: how many upstream attempts the gateway made.
  • Ngrok-AIG-Model: the upstream model that produced the response.
All three are exposed through CORS, so browser clients can read them. They are ngrok’s own headers, so managed-key and Bring Your Own Keys callers see the same values. See Response headers.The native Gemini generateContent and streamGenerateContent surface returns these headers too, where it previously returned only Content-Type. Bodies on that surface are still forwarded byte-for-byte.
​
Breaking Change
Traffic Policy action retired

​
The ai-gateway Traffic Policy action is retired

The AI Gateway no longer runs as a Traffic Policy action on your own endpoint. A policy that still contains the ai-gateway action fails validation with ERR_NGROK_3820.
  • Send LLM traffic to https://gateway.ngrok.ai with an access key, and manage the gateway at app.ngrok.ai.
  • Existing credits were migrated to app.ngrok.ai automatically. No action is needed to keep your balance.
  • Remove the ai-gateway action from any endpoint that still carries it.
  • The action’s result variables, such as actions.ngrok.ai_gateway, are gone. Read the response headers and the Usage API instead.
Email support@ngrok.com with questions about a migrated account.
​
General Availability
app.ngrok.ai launch

​
app.ngrok.ai and gateway.ngrok.ai

Manage your AI Gateway from app.ngrok.ai and send traffic to a shared gateway URL:
​
Early Access
EA v0.0.5

​
AI Gateway API keys & credits

You can now authenticate requests to your AI Gateway with AI Gateway API Keys—no provider accounts needed. The AI Gateway manages the provider keys for OpenAI and Anthropic on your behalf. Purchase prepaid credits to fund usage.
​
Early Access
EA v0.0.4

​
Native Anthropic SDK support

Use Anthropic’s official Python and TypeScript SDKs directly with the AI Gateway. The gateway now supports the native /v1/messages API format alongside the OpenAI-compatible format. See the Anthropic SDK guide for setup instructions.
​
Early Access
EA v0.0.3

​
Groq provider support

Groq is now a built-in provider. Access Llama and Mixtral models running on Groq’s LPU hardware with automatic failover and observability.
​
Early Access
EA v0.0.2

​
Expanded model catalog

Added new providers to the model catalog: Hyperbolic, InceptionLabs, Inference.net, and OpenRouter. Updated model aliases and provider-specific model IDs for improved compatibility.
​
Early Access
EA v0.0.1

​
Early access release

Initial Early Access release of the AI Gateway.

​
Features

  • OpenAI-compatible API for routing requests to multiple AI providers
  • Support for OpenAI, Anthropic, Google, and other providers
  • Multi-provider failover across models, providers, and API keys
  • Multi-key failover per provider
  • CEL-based model selection strategies
  • Custom provider support via internal endpoints
  • Token counting and usage metrics