Skip to main content
Azure OpenAI Service provides OpenAI models on Microsoft Azure. Connect your deployment to the AI Gateway as a custom provider.

What you’ll need

Overview

Azure OpenAI uses deployment-specific URLs and requires an api-version query parameter. The gateway injects your Azure API key server-side.

Getting started

1

Gather Azure details

From the Azure Portal, collect:
  1. Resource endpoint: https://your-resource.openai.azure.com
  2. API key: from Keys and Endpoint
  3. Deployment name: the name of your model deployment
  4. API version: a supported version (for example 2024-02-15-preview)
2

Create the custom provider

Azure requires the deployment path and api-version in the base URL. See Create a custom provider with:
  • Provider ID: azure-openai
  • Base URL: https://your-resource.openai.azure.com/openai/deployments/your-deployment?api-version=2024-02-15-preview
  • API format: OpenAI Chat Completions
  • Models: the model name for this deployment (for example gpt-4o)
Include api-version as a query parameter on the base URL. Azure rejects requests without it.
3

Store your Azure API key

Add a provider key for your Azure deployment.
4

Configure access

Create an access key configuration that:
  1. Allows the azure-openai provider in the access scope
  2. Adds a routing rule for azure-openai with Bring your own API key and your stored key attached
Assign the configuration to your access key.
5

Send requests

Multiple deployments and regions

Create a separate custom provider for each Azure deployment or region. Give each a unique provider ID (for example azure-gpt4, azure-eastus) with its own deployment URL and model list. Attach the same or different provider keys per deployment. Pin a region with azure-eastus:gpt-4o, or list multiple models for failover: models: ["azure-eastus:gpt-4o", "openai:gpt-4o"].

Troubleshooting

Next steps