Use a model you run yourself

​

What you’ll need

1. Expose a supported API surface, such as OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages.
2. Be reachable from the AI Gateway.
3. Have at least one model ID configured on the provider.

​

Connect a local model with an internal endpoint

ngrok http 11434 --url https://my-ollama.internal

​

Create a custom provider

1. Go to Providers.
2. Open the Custom tab.
3. Select Add provider.
4. Choose External URL or Local.
5. Enter a provider ID, base URL, API format, and model IDs.
6. Add a provider key if the upstream requires authentication.

​

Create a custom provider with the API

curl -X POST https://api.ngrok.ai/providers \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "providerId": "my-ollama",
    "name": "Local Ollama",
    "baseUrl": "https://my-ollama.internal",
    "supportedApiSurfaces": [
      { "format": "openai", "surface": "chat-completions" }
    ],
    "models": [{ "modelId": "llama3.2" }]
  }'

​

Allow the provider on an access key

1. Add a provider key if the upstream requires authentication.
2. Create or update an access key configuration.
3. Allow the custom provider ID.
4. Assign the configuration to an access key.

​

Call the model

{
  "model": "my-ollama:llama3.2",
  "messages": [{"role": "user", "content": "Hello"}]
}

​

URL requirements

​

Next steps

• Ollama guide: Connect Ollama
• vLLM guide: Connect vLLM
• Provider keys: Store upstream credentials
• Restrict providers and models: Allow the custom provider on a key