Azure Aigateway

Azure AI Gateway

Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.

> To deploy APIM, use the azure-prepare skill. See APIM deployment guide.

When to Use This Skill

| Category | Triggers | |----------|----------| | Model Governance | "semantic caching", "token limits", "load balance AI", "track token usage" | | Tool Governance | "rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP" | | Agent Governance | "content safety", "jailbreak detection", "filter harmful content" | | Configuration | "add Azure OpenAI backend", "configure my model", "add AI Foundry model" | | Testing | "test AI gateway", "call OpenAI through gateway" |

---

Quick Reference

| Policy | Purpose | Details | |--------|---------|---------| | azure-openai-token-limit | Cost control | [Model Policies](references/policies.md#token-rate-limiting) | | azure-openai-semantic-cache-lookup/store | 60-80% cost savings | [Model Policies](references/policies.md#semantic-caching) | | azure-openai-emit-token-metric | Observability | [Model Policies](references/policies.md#token-metrics) | | llm-content-safety | Safety & compliance | [Agent Policies](references/policies.md#content-safety) | | rate-limit-by-key | MCP/tool protection | [Tool Policies](references/policies.md#request-rate-limiting) |

---

Get Gateway Details

# Get gateway URL
az apim show --name  --resource-group  --query "gatewayUrl" -o tsv

# List backends (AI models)
az apim backend list --service-name  --resource-group  \
  --query "[].{id:name, url:url}" -o table

# Get subscription key
az apim subscription keys list \
  --service-name  --resource-group  --subscription-id 

---

Test AI Endpoint

GATEWAY_URL=$(az apim show --name  --resource-group  --query "gatewayUrl" -o tsv)

curl -X POST "${GATEWAY_URL}/openai/deployments//chat/completions?api-version=2024-02-01" \
  -H "Content-Type: application/json" \
  -H "Ocp-Apim-Subscription-Key: " \
  -d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'

---

Common Tasks

Add AI Backend

See [references/patterns.md](references/patterns.md#pattern-1-add-ai-model-backend) for full steps.

# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table

# Create backend
az apim backend create --service-name  --resource-group  \
  --backend-id openai-backend --protocol http --url "https://.openai.azure.com/openai"

# Grant access (managed identity)
az role assignment create --assignee  \
  --role "Cognitive Services User" --scope 

Apply AI Governance Policy

Recommended policy order in ``:

1. Authentication - Managed identity to backend
2. Semantic Cache Lookup - Check cache before calling AI
3. Token Limits - Cost control
4. Content Safety - Filter harmful content
5. Backend Selection - Load balancing
6. Metrics - Token usage tracking

See [references/policies.md](references/policies.md#combining-policies) for complete example.

---

Troubleshooting

| Issue | Solution | |-------|----------| | Token limit 429 | Increase tokens-per-minute or add load balancing | | No cache hits | Lower score-threshold to 0.7 | | Content false positives | Increase category thresholds (5-6) | | Backend auth 401 | Grant APIM "Cognitive Services User" role |

See [references/troubleshooting.md](references/troubleshooting.md) for details.

---

References

• [Detailed Policies](references/policies.md) - Full policy examples
• [Configuration Patterns](references/patterns.md) - Step-by-step patterns
• [Troubleshooting](references/troubleshooting.md) - Common issues
• AI-Gateway Samples
• GenAI Gateway Docs

SDK Quick References

• Content Safety: [Python](references/sdk/azure-ai-contentsafety-py.md) | [TypeScript](references/sdk/azure-ai-contentsafety-ts.md)
• API Management: [Python](references/sdk/azure-mgmt-apimanagement-py.md) | [.NET](references/sdk/azure-mgmt-apimanagement-dotnet.md)

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: microsoft
• Source: microsoft/azure-skills
• License: MIT

Install and usage instructions live in the source repository linked above.