Ai Gateway Examples

AI Gateway Examples

Examples demonstrating how to use Ferro Labs AI Gateway — an open-source AI gateway written in Go with a unified API for OpenAI, Anthropic, Bedrock, Azure, and 100+ LLMs.

Directory Structure

.
├── basic/                    # Direct provider request — the simplest use case
├── streaming/                # Streaming chat completions (real-time token output)
├── custom-plugin/            # Write and register a custom plugin
├── embedded/                 # Embed the gateway inside an existing HTTP server
├── fallback/                 # Automatic fallback between providers on failure
├── loadbalance/              # Weighted load balancing across multiple providers
├── with-circuit-breaker/     # Per-provider circuit breaker configuration
├── with-guardrails/          # Built-in word-filter and token-limit guardrails
├── with-hooks/               # Event hooks for telemetry and audit logging
├── with-mcp/                 # MCP (Model Context Protocol) tool-call integration
└── shared/                   # Shared provider helpers (used by all examples)

Quick Start

Prerequisites

• Go 1.22+ — install
• At least one LLM provider API key (OpenAI, Anthropic, Groq, Gemini, Mistral, Together, Cohere, or DeepSeek)

Run an Example

# Set a provider API key
export OPENAI_API_KEY=sk-...
# or
export ANTHROPIC_API_KEY=sk-ant-...
# or any other supported provider key

# Run the basic example
go run ./basic

Examples

basic

Sends a single chat completion request to the first provider for which an API key is configured. Great starting point.

OPENAI_API_KEY=sk-... go run ./basic

streaming

Streams chat completion tokens in real-time via gw.RouteStream(). Tokens are printed as they arrive instead of waiting for the full response.

OPENAI_API_KEY=sk-... go run ./streaming

custom-plugin

Demonstrates writing a plugin.Plugin implementation (a request-ID injector) and registering it at the before_request stage.

OPENAI_API_KEY=sk-... go run ./custom-plugin

embedded

Shows how to embed the gateway as an HTTP handler inside your own net/http server, mounting the AI route alongside your existing application routes.

OPENAI_API_KEY=sk-... go run ./embedded
# Then: curl -X POST http://localhost:8080/ai/v1/chat/completions -d '...'

fallback

Configures ModeFallback: the gateway tries providers in order, retrying on the primary before switching to the next. Requires at least one provider key; works better with two.

OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... go run ./fallback

loadbalance

Configures ModeLoadBalance with weighted targets. The first provider receives 70% of requests, the rest share the remaining 30%. Requires at least two provider keys.

OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... go run ./loadbalance

with-circuit-breaker

Attaches a CircuitBreakerConfig to the primary provider target. After two consecutive failures the circuit opens for 10 s, routing all traffic to the fallback.

OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... go run ./with-circuit-breaker

with-guardrails

Loads the built-in word-filter and max-token guardrail plugins via PluginConfig. Requests containing blocked words are rejected before reaching the provider.

OPENAI_API_KEY=sk-... go run ./with-guardrails

with-hooks

Registers an event hook via gw.AddHook(fn). The hook receives gateway.request.completed and gateway.request.failed events asynchronously after each request — useful for telemetry, cost tracking, or audit logging.

OPENAI_API_KEY=sk-... go run ./with-hooks

with-mcp

Starts a local mock MCP server, wires it into the gateway, and sends a tool-calling request. The gateway injects tool definitions, drives the agentic tool-call loop, and returns the final answer.

OPENAI_API_KEY=sk-... go run ./with-mcp
# Works best with models that support tool/function calling (gpt-4o, claude-3-5-sonnet, etc.)

Supported Provider Environment Variables

| Variable | Provider | |----------|----------| | OPENAI_API_KEY | OpenAI | | ANTHROPIC_API_KEY | Anthropic | | GROQ_API_KEY | Groq | | GEMINI_API_KEY | Google Gemini | | MISTRAL_API_KEY | Mistral | | TOGETHER_API_KEY | Together AI | | COHERE_API_KEY | Cohere | | DEEPSEEK_API_KEY | DeepSeek |

More Information

• AI Gateway repository
• Documentation

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: ferro-labs
• Source: ferro-labs/ai-gateway-examples
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.