Awesome Ai Gateway

Awesome AI Gateway [](https://awesome.re)

[](https://github.com/cuihuan/awesome-ai-gateway/stargazers) [](BENCHMARKS.md) [](.github/workflows/daily-update.yml)

> Pick the right AI gateway for your need in ~10 seconds — then trust the answer. A decision tree, a reproducible cost benchmark, and independent evidence for what we exclude. Organized by what you actually need, not by vendor.

Built the hard way: I burned $788 on AI coding in a single day — one flagship model ate 78% of it, just because I'd defaulted everything to the priciest option. So I mapped the whole gateway landscape. → [the story](#why-this-exists)

Languages: English · [简体中文](README.zh-CN.md)

  🧭 Pick a gateway       🚀 Live interactive site       📊 Cost & scorecard  

📑 Full contents — pick fast · browse by need · reference

[](CONTRIBUTING.md) [](LICENSE) [](https://github.com/cuihuan/awesome-ai-gateway/commits/main)

Pick fast · [Which gateway should I use?](#which-gateway-should-i-use) · [📊 Latest evaluations](#-latest-evaluations) · [Quick comparison](#quick-comparison)

Browse by need · [💰 Cost-first](#-cost-first-cheapest-multi-model-access) · [🔓 Self-hosted](#-self-hosted-open-source) · [🏢 Enterprise & compliance](#-enterprise--compliance) · [☁️ First-party clouds](#️-first-party-gateways-cloud--model-vendors) · [🇨🇳 China ecosystem](#-china-ecosystem) · [🤖 MCP & agent gateways](#-mcp--agent-gateways)

Reference · [📊 Evaluation set](BENCHMARKS.md) · [How to choose safely](#how-to-choose-safely) · [FAQ](#faq) · [📚 Essential reading](#-essential-reading) · [📰 What's new](#-whats-new) · [Glossary](#glossary) · [Why this exists](#why-this-exists) · [Contributing](#contributing)

📊 Latest evaluations

A running digest of fresh model, pricing and gateway evals — newest first, every entry dated and sourced. This is the fast-moving signal layer; for our own reproducible cost tables and model scorecard, see the [full evaluation set](BENCHMARKS.md). Spotted a new eval worth tracking? [Add it](CONTRIBUTING.md).

| Date | Category | Finding | Source | |---|---|---|---| | 2026-06-21 | 💰 Pricing | The API pricing market now spans 123 models across 12 providers, with a >400× price spread over the full input/output range — cheapest flagship DeepSeek V4 Flash ($0.14/M input) vs priciest GPT-5.5 Pro ($30.00/M input) is already ~214× on input alone. Tiering has hardened: top reasoning (o3) runs ~20× a nano-tier model on input, wider on output. | aipricing.guru | | 2026-06 | 📈 Adoption | ChatGPT hit ~900M weekly active users and >2.5B queries/day — demand scaling about as fast as the price spread. | DemandSage |

💸 Same ¥100 (≈ $14.66) — how much can each model read? The 400× spread, made concrete.

How many input+output tokens ¥100 buys, by model (blended estimate · snapshot 2026-06-21 · aipricing.guru):

| Tier | Model | Tokens / ¥100 | ≈ Chinese chars | |---|---|---|---| | 🥇 Rock-bottom | DeepSeek V4 Flash | 35.2M | ~26.4M | | 🥇 Rock-bottom | GPT-4.1 nano | 29.6M | ~22.2M | | 🥇 Rock-bottom | GPT-5.4 nano | 10.2M | ~7.7M | | 💚 Value | GPT-5.4 mini | 2.83M | ~2.1M | | 💚 Value | DeepSeek V4 Pro | 2.83M | ~2.1M | | 🧠 Reasoning | o3 | 1.48M | ~1.1M | | 🏁 Flagship | Gemini 2.5 Pro | 1.31M | ~0.98M | | 🏁 Flagship | GPT-5.5 | 0.42M | ~0.32M | | 🏁 Flagship | GPT-5.5 Pro | 0.07M | ~0.05M |

> One line: ¥100 reads ~26M Chinese characters on DeepSeek V4 Flash — roughly **52× the Three-Body trilogy — but only ~50K** on GPT-5.5 Pro, about one short story. Choosing a model is choosing the scale factor on your money; the [Cost-first](#-cost-first-cheapest-multi-model-access) gateways exist to exploit exactly this spread.

Which gateway should I use?

⚡ Fast answer — one sane default per need (alternatives in each linked section):

| I need… | Start with | Drill into | |---|---|---| | Cheapest access to many models, zero ops | OpenRouter | [Cost-first](#-cost-first-cheapest-multi-model-access) | | Zero markup on my own keys | Vercel / Cloudflare | [Cost-first](#-cost-first-cheapest-multi-model-access) | | Self-host, broadest features | LiteLLM | [Self-hosted](#-self-hosted-open-source) | | Self-host, lowest overhead | Bifrost (Go) | [Self-hosted](#-self-hosted-open-source) | | China models + team key billing | new-api | [China ecosystem](#-china-ecosystem) | | Enterprise K8s + audit | Kong / Higress | [Enterprise](#-enterprise--compliance) | | Strongest compliance (HIPAA/FedRAMP) | Azure / Bedrock | [First-party](#️-first-party-gateways-cloud--model-vendors) | | Govern agents / MCP traffic | agentgateway | [MCP & agents](#-mcp--agent-gateways) |

📋 The full decision tree — every branch, copy-pasteable

Do you want to self-host?
│
├─ NO — hosted, minimal ops
│   ├─ Cheapest access to many models ──────────▶ OpenRouter · Vercel AI Gateway (0% markup)
│   ├─ Free control plane over your own keys ───▶ Cloudflare AI Gateway
│   ├─ EU data residency matters ───────────────▶ Requesty · Eden AI · nexos.ai
│   └─ Already on one cloud ────────────────────▶ AWS Bedrock · Azure APIM · Vertex AI
│
└─ YES — self-hosted / open source
    ├─ Python stack, broadest features ─────────▶ LiteLLM
    ├─ Raw performance (Go/Rust/TS) ────────────▶ Bifrost · Portkey Gateway
    ├─ Built-in evals + observability ──────────▶ Helicone · Portkey Gateway
    ├─ Key distribution / billing / CN models ──▶ new-api · one-api · GPT-Load
    ├─ Enterprise K8s, audit, guardrails ───────▶ Kong · Higress · APISIX · Envoy AI Gateway
    └─ Governing AI agents & MCP traffic ───────▶ agentgateway · Lunar.dev

✅ Why trust this list

• Independent — no vendor money, no affiliate links, CC0. Unlike affiliate-driven relay "rankings," nobody pays to appear here.
• Reproducible, not asserted. Every cost cell is computed from [open pricing data](data/models.json) by a [unit-tested script](scripts/cost_calc.py); stars refresh daily via CI.
• Honest about risk. We disclose CVEs, label archived/stale projects, and [exclude gray-market relays](#how-to-choose-safely) — with the research to back it.

---

> Why this matters: the same task can cost 100× more depending on the model behind your gateway. An AI gateway sits between your code and LLM providers — one endpoint, one key, many models — handling routing, failover, caching, rate limits, cost tracking and guardrails, so you change a base_url instead of rewriting your app. Pick the gateway here, then the [evaluation set](BENCHMARKS.md) shows which model to route to.

⭐ Found this useful? Star it — that's how the next engineer choosing a gateway finds it. CC0, no signup, no tracking, no vendor money.

Quick comparison

Stars auto-refresh daily. ✅ built-in · ➕ via plugin/paid tier · ❌ not available.

| Project | Type | Stars | License | Multi-provider | Fallback / LB | Caching | Guardrails | Cost tracking | |---|---|---|---|---|---|---|---|---| | LiteLLM | OSS proxy + SDK | ⭐ 51.9k | MIT¹ | ✅ 100+ | ✅ | ✅ | ✅ | ✅ | | new-api | OSS relay/billing | ⭐ 40.4k | AGPL-3.0 | ✅ | ✅ | ➕ | ➕ | ✅ | | one-api | OSS relay/billing | ⭐ 35.3k | MIT | ✅ | ✅ | ❌ | ❌ | ✅ | | Kong AI Gateway | OSS API gateway | ⭐ 43.7k | Apache-2.0 | ✅ | ✅ | ✅ semantic | ✅ | ✅ | | Apache APISIX | OSS API gateway | ⭐ 16.8k | Apache-2.0 | ✅ | ✅ | ➕ | ➕ | ➕ | | Portkey Gateway | OSS gateway + SaaS | ⭐ 12.2k | MIT | ✅ 1600+ | ✅ | ✅ | ✅ 50+ | ➕ SaaS | | TensorZero | OSS LLMOps · ⚠️ archived '26 | ⭐ 11.7k | Apache-2.0 | ✅ | ✅ | ✅ | ➕ | ✅ | | Higress | OSS AI-native gateway | ⭐ 8.7k | Apache-2.0 | ✅ | ✅ | ✅ | ✅ | ✅ | | GPT-Load | OSS key-pool proxy | ⭐ 6.2k | MIT | ✅ | ✅ key rotation | ❌ | ❌ | ➕ | | Bifrost | OSS gateway (Go) | ⭐ 6.1k | Apache-2.0 | ✅ | ✅ adaptive | ✅ | ✅ | ✅ | | Helicone | OSS observability + gateway | ⭐ 5.9k | Apache-2.0 | ✅ | ✅ | ✅ | ➕ | ✅ | | Envoy AI Gateway | OSS K8s gateway | ⭐ 1.8k | Apache-2.0 | ✅ | ✅ | ➕ | ➕ | ✅ | | OpenRouter | SaaS marketplace | — | Commercial | ✅ 400+ | ✅ | ✅ | ➕ | ✅ | | Vercel AI Gateway | SaaS (0% markup) | — | Commercial | ✅ 100s | ✅ | ❌ | ❌ | ✅ | | Cloudflare AI Gateway | SaaS control plane | — | Commercial (free tier) | ✅ | ✅ dynamic | ✅ | ✅ | ✅ budgets |

¹ LiteLLM core is MIT; the repo contains a separately licensed enterprise directory.

> 📂 Browse the raw data (machine-readable, CC0): [models & pricing JSON](data/models.json) · [cost table CSV](data/costtable.csv) · [gateway scorecard CSV](data/gatewaysscorecard.csv). Every cost cell is regenerated from this data by a [unit-tested script](scripts/cost_calc.py).

> The full directory at a glance — browse the sections below by your need.

💰 Cost-first: cheapest multi-model access

Pain point: "I want many models for the least money and zero ops."

• OpenRouter — The dominant model marketplace: 400+ models behind one OpenAI-compatible API, pay-as-you-go with automatic failover; ~5.5% fee when buying credits. $113M Series B (May 2026), ~8M users.
• Vercel AI Gateway — Hundreds of models at provider list price (0% markup), $5/month free credits, zero-data-retention option; pairs naturally with the AI SDK.
• Cloudflare AI Gateway — Free control plane in front of your own provider keys: caching, dynamic routing, unified billing, and dollar-denominated spend limits (2026 beta).
• Requesty — EU-friendly OpenRouter alternative: 400+ models, sub-20ms failover, ~5% markup.
• Eden AI — Unified API for 500+ models plus vision/OCR/speech; EU-based, ~5.5% platform fee.
• Helicone AI Gateway (cloud) — Passthrough billing at 0% markup with observability bundled.
• GPT-Load ⭐ 6.2k — High-performance Go proxy that rotates pools of API keys across channels to maximize quota usage.
• Loop Gateway — OpenAI-compatible proxy that meters every request in Bitcoin sats instead of dollars. 311 models via OpenRouter at a 15% markup. No accounts, no email, no card; top up over Lightning, get a bearer token. Three auth rails (prepaid bearer, L402, Cashu). Self-hostable in Go via docker-compose, live at api.loopxxi.com. New & unverified (anonymous 1★ repo) — it resells frontier models through the operator's own OpenRouter account at a 15% markup, and account-less + crypto-prepaid means no recourse if it swaps models or vanishes; confirm fidelity with [canarycheck.py](scripts/canarycheck.py) and only top up what you can afford to lose.
• nullsink (repo) — Account-less, metered proxy for frontier-model APIs, paid in Monero or Bitcoin. No accounts, no email, no card; mint a bearer token, prepay on-chain, and point the official SDKs at one base URL. ~10% markup taken once at top-up; no IP logging, no request logs; payment and token kept unlinkable. Self-hostable single binary (TypeScript/Bun, AGPL-3.0), live at nullsink.is. New & unverified (repo created 2026-06, 3★) — account-less + crypto-prepaid + no logs means no recourse if it swaps models or vanishes; confirm fidelity with [canarycheck.py](scripts/canary_check.py) and only top up what you can afford to lose.
• AIMLAPI — One OpenAI/Anthropic-compatible endpoint fronting 400+ models (chat, image, video, audio, embeddings); prepaid, OpenRouter-style aggregator.
• Novita AI — Unified API to 200+ open-source models (DeepSeek/Qwen/Llama…) with load balancing, autoscaling and failover; also a GPU cloud.
• FlintAPI (repo) — Hosted OpenAI-compatible gateway aggregating 25+ Chinese LLMs (DeepSeek, Qwen, Kimi, GLM, MiniMax) with $2 free credits. New and unverified — confirm model fidelity (e.g. with [canarycheck.py](scripts/canarycheck.py)) before relying on it in production.
• FlowBar — Hosted OpenAI-compatible relay reselling 50+ models (GPT, Claude, Gemini, DeepSeek, Qwen, GLM, Kimi) below OpenRouter, with USD/CNY/crypto payment. New and unverified — confirm model fidelity (e.g. with [canarycheck.py](scripts/canary_check.py)) before relying on it in production.
• Meshs One — Hosted OpenAI-compatible relay fronting Chinese frontier models (DeepSeek-V4, Qwen3.7-Max, MiniMax-M3) under one key, per-token pay-as-you-go (its /v1 endpoint returns a new_api_error, so it appears to run on new-api). New & unverified — closed-source and brand-new; confirm model fidelity (e.g. with [canarycheck.py](scripts/canarycheck.py)) before relying on it in production.
• lxg2it ModelRouter (repo) — Solo-built, OpenAI-compatible router over 7+ providers (Anthropic, OpenAI, Google, Cerebras, Groq, Grok, GLM) with tiered automatic fallback that selects the cheapest available model. Free tier plus a paid tier advertised at 0% markup on Anthropic (a deposit fee may apply — verify current pricing). New and unverified — the public repo is now a deprecated reference stub (routing moved to the closed hosted service; still no license file) — confirm model fidelity (e.g. with [canarycheck.py](scripts/canarycheck.py)) before relying on it in production.
• OpenPaths (repo) — Hosted OpenAI-compatible router auto-routing across 15+ providers (OpenAI, Anthropic, Gemini, Groq, xAI, DeepSeek, Mistral) under one API, spanning chat, image, video, music, speech, embeddings and transcription. New & unverified — despite the "open source" framing the GitHub repo is a no-code, unlicensed marketing mirror (canonical code lives on the third-party Codex Infinity platform and is agent-maintained), so treat it as a closed hosted relay and confirm model fidelity (e.g. with [canarycheck.py](scripts/canary_check.py)) before relying on it in production.
• Glama Gateway — OpenAI-compatible gateway to 100+ models with consolidated billing, caching and logging (OSS core glama-ai/lightport).

> 💡 Squeeze more from any gateway: enable semantic caching (Kong, Bifrost, Zuplo), set spend limits (Cloudflare, Zuplo, Pydantic/Logfire), and route easy prompts to cheap models (see [Smart routing](#-smart-routing--model-selection)).

🔓 Self-hosted open source

Pain point: "My keys, my infra, no per-token middleman fee."

• LiteLLM ⭐ 51.9k — The default choice: Python SDK + proxy server speaking OpenAI format to 100+ providers, with virtual keys, budgets, load balancing and guardrails.
• Portkey Gateway ⭐ 12.2k — Fast TypeScript gateway (1,600+ models, 50+ guardrails) that also powers Portkey's com

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: cuihuan
• Source: cuihuan/awesome-ai-gateway
• License: CC0-1.0
• Homepage: https://cuihuan.github.io/awesome-ai-gateway/

Install and usage instructions live in the source repository linked above.