# Awesome Ai Gateway > ⚡ Awesome AI Gateway — curated comparison of 100+ AI gateways & LLM proxies (LiteLLM, OpenRouter, Portkey, Kong, Higress, new-api, Bifrost) by cost, security, compliance & self-hosting. Decision tree + reproducible benchmarks. Open source, bilingual, updated daily. - **Type:** MCP server - **Install:** `agentstack add mcp-cuihuan-awesome-ai-gateway` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [cuihuan](https://agentstack.voostack.com/s/cuihuan) - **Installs:** 0 - **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml) - **Latest version:** 0.1.0 - **License:** CC0-1.0 - **Upstream author:** [cuihuan](https://github.com/cuihuan) - **Source:** https://github.com/cuihuan/awesome-ai-gateway - **Website:** https://cuihuan.github.io/awesome-ai-gateway/ ## Install ```sh agentstack add mcp-cuihuan-awesome-ai-gateway ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # Awesome AI Gateway [](https://awesome.re) [](https://github.com/cuihuan/awesome-ai-gateway/stargazers) [](BENCHMARKS.md) [](.github/workflows/daily-update.yml) > **Pick the right AI gateway for your need in ~10 seconds — then trust the answer.** A decision tree, a reproducible cost benchmark, and independent evidence for what we exclude. Organized by what you actually need, not by vendor. _Built the hard way: **I burned $788 on AI coding in a single day** — one flagship model ate 78% of it, just because I'd defaulted everything to the priciest option. So I mapped the whole gateway landscape. → [the story](#why-this-exists)_ **Languages:** English · [简体中文](README.zh-CN.md)   🧭 Pick a gateway       🚀 Live interactive site       📊 Cost & scorecard   📑 Full contents — pick fast · browse by need · reference [](CONTRIBUTING.md) [](LICENSE) [](https://github.com/cuihuan/awesome-ai-gateway/commits/main) **Pick fast** · [Which gateway should I use?](#which-gateway-should-i-use) · [📊 Latest evaluations](#-latest-evaluations) · [Quick comparison](#quick-comparison) **Browse by need** · [💰 Cost-first](#-cost-first-cheapest-multi-model-access) · [🔓 Self-hosted](#-self-hosted-open-source) · [🏢 Enterprise & compliance](#-enterprise--compliance) · [☁️ First-party clouds](#️-first-party-gateways-cloud--model-vendors) · [🇨🇳 China ecosystem](#-china-ecosystem) · [🤖 MCP & agent gateways](#-mcp--agent-gateways) **Reference** · [📊 Evaluation set](BENCHMARKS.md) · [How to choose safely](#how-to-choose-safely) · [FAQ](#faq) · [📚 Essential reading](#-essential-reading) · [📰 What's new](#-whats-new) · [Glossary](#glossary) · [Why this exists](#why-this-exists) · [Contributing](#contributing) ## 📊 Latest evaluations *A running digest of fresh model, pricing and gateway evals — **newest first, every entry dated and sourced.** This is the fast-moving signal layer; for our own **reproducible** cost tables and model scorecard, see the [full evaluation set](BENCHMARKS.md). Spotted a new eval worth tracking? [Add it](CONTRIBUTING.md).* | Date | Category | Finding | Source | |---|---|---|---| | 2026-06-21 | 💰 Pricing | The API pricing market now spans **123 models across 12 providers**, with a **>400× price spread** over the full input/output range — cheapest flagship **DeepSeek V4 Flash ($0.14/M input)** vs priciest **GPT-5.5 Pro ($30.00/M input)** is already ~214× on input alone. Tiering has hardened: top reasoning (o3) runs ~20× a nano-tier model on input, wider on output. | [aipricing.guru](https://aipricing.guru) | | 2026-06 | 📈 Adoption | ChatGPT hit **~900M weekly active users** and **>2.5B queries/day** — demand scaling about as fast as the price spread. | [DemandSage](https://www.demandsage.com/chatgpt-statistics/) | 💸 Same ¥100 (≈ $14.66) — how much can each model read? The 400× spread, made concrete. How many **input+output tokens** ¥100 buys, by model (blended estimate · snapshot **2026-06-21** · [aipricing.guru](https://aipricing.guru)): | Tier | Model | Tokens / ¥100 | ≈ Chinese chars | |---|---|---|---| | 🥇 Rock-bottom | **DeepSeek V4 Flash** | 35.2M | ~26.4M | | 🥇 Rock-bottom | GPT-4.1 nano | 29.6M | ~22.2M | | 🥇 Rock-bottom | GPT-5.4 nano | 10.2M | ~7.7M | | 💚 Value | GPT-5.4 mini | 2.83M | ~2.1M | | 💚 Value | DeepSeek V4 Pro | 2.83M | ~2.1M | | 🧠 Reasoning | o3 | 1.48M | ~1.1M | | 🏁 Flagship | Gemini 2.5 Pro | 1.31M | ~0.98M | | 🏁 Flagship | GPT-5.5 | 0.42M | ~0.32M | | 🏁 Flagship | GPT-5.5 Pro | 0.07M | ~0.05M | > **One line:** ¥100 reads **~26M Chinese characters** on DeepSeek V4 Flash — roughly **52× the *Three-Body* trilogy** — but only **~50K** on GPT-5.5 Pro, about one short story. Choosing a model is choosing the scale factor on your money; the [Cost-first](#-cost-first-cheapest-multi-model-access) gateways exist to exploit exactly this spread. ## Which gateway should I use? **⚡ Fast answer** — one sane default per need (alternatives in each linked section): | I need… | Start with | Drill into | |---|---|---| | Cheapest access to many models, zero ops | **OpenRouter** | [Cost-first](#-cost-first-cheapest-multi-model-access) | | Zero markup on my own keys | **Vercel** / **Cloudflare** | [Cost-first](#-cost-first-cheapest-multi-model-access) | | Self-host, broadest features | **LiteLLM** | [Self-hosted](#-self-hosted-open-source) | | Self-host, lowest overhead | **Bifrost** (Go) | [Self-hosted](#-self-hosted-open-source) | | China models + team key billing | **new-api** | [China ecosystem](#-china-ecosystem) | | Enterprise K8s + audit | **Kong** / **Higress** | [Enterprise](#-enterprise--compliance) | | Strongest compliance (HIPAA/FedRAMP) | **Azure** / **Bedrock** | [First-party](#️-first-party-gateways-cloud--model-vendors) | | Govern agents / MCP traffic | **agentgateway** | [MCP & agents](#-mcp--agent-gateways) | 📋 The full decision tree — every branch, copy-pasteable ```text Do you want to self-host? │ ├─ NO — hosted, minimal ops │ ├─ Cheapest access to many models ──────────▶ OpenRouter · Vercel AI Gateway (0% markup) │ ├─ Free control plane over your own keys ───▶ Cloudflare AI Gateway │ ├─ EU data residency matters ───────────────▶ Requesty · Eden AI · nexos.ai │ └─ Already on one cloud ────────────────────▶ AWS Bedrock · Azure APIM · Vertex AI │ └─ YES — self-hosted / open source ├─ Python stack, broadest features ─────────▶ LiteLLM ├─ Raw performance (Go/Rust/TS) ────────────▶ Bifrost · Portkey Gateway ├─ Built-in evals + observability ──────────▶ Helicone · Portkey Gateway ├─ Key distribution / billing / CN models ──▶ new-api · one-api · GPT-Load ├─ Enterprise K8s, audit, guardrails ───────▶ Kong · Higress · APISIX · Envoy AI Gateway └─ Governing AI agents & MCP traffic ───────▶ agentgateway · Lunar.dev ``` ### ✅ Why trust this list - **Independent — no vendor money, no affiliate links, CC0.** Unlike affiliate-driven relay "rankings," nobody pays to appear here. - **Reproducible, not asserted.** Every cost cell is computed from [open pricing data](data/models.json) by a [unit-tested script](scripts/cost_calc.py); stars refresh daily via CI. - **Honest about risk.** We disclose CVEs, label archived/stale projects, and [exclude gray-market relays](#how-to-choose-safely) — with the research to back it. --- > **Why this matters:** the same task can cost **100× more** depending on the model behind your gateway. An **AI gateway** sits between your code and LLM providers — one endpoint, one key, many models — handling routing, failover, caching, rate limits, cost tracking and guardrails, so you change a `base_url` instead of rewriting your app. Pick the gateway here, then the [evaluation set](BENCHMARKS.md) shows which model to route to. ⭐ **Found this useful? [Star it](https://github.com/cuihuan/awesome-ai-gateway)** — that's how the next engineer choosing a gateway finds it. CC0, no signup, no tracking, no vendor money. ## Quick comparison Stars auto-refresh daily. ✅ built-in · ➕ via plugin/paid tier · ❌ not available. | Project | Type | Stars | License | Multi-provider | Fallback / LB | Caching | Guardrails | Cost tracking | |---|---|---|---|---|---|---|---|---| | [LiteLLM](https://github.com/BerriAI/litellm) | OSS proxy + SDK | ⭐ 51.9k | MIT¹ | ✅ 100+ | ✅ | ✅ | ✅ | ✅ | | [new-api](https://github.com/QuantumNous/new-api) | OSS relay/billing | ⭐ 40.4k | AGPL-3.0 | ✅ | ✅ | ➕ | ➕ | ✅ | | [one-api](https://github.com/songquanpeng/one-api) | OSS relay/billing | ⭐ 35.3k | MIT | ✅ | ✅ | ❌ | ❌ | ✅ | | [Kong AI Gateway](https://github.com/Kong/kong) | OSS API gateway | ⭐ 43.7k | Apache-2.0 | ✅ | ✅ | ✅ semantic | ✅ | ✅ | | [Apache APISIX](https://github.com/apache/apisix) | OSS API gateway | ⭐ 16.8k | Apache-2.0 | ✅ | ✅ | ➕ | ➕ | ➕ | | [Portkey Gateway](https://github.com/Portkey-AI/gateway) | OSS gateway + SaaS | ⭐ 12.2k | MIT | ✅ 1600+ | ✅ | ✅ | ✅ 50+ | ➕ SaaS | | [TensorZero](https://github.com/tensorzero/tensorzero) | OSS LLMOps · ⚠️ archived '26 | ⭐ 11.7k | Apache-2.0 | ✅ | ✅ | ✅ | ➕ | ✅ | | [Higress](https://github.com/higress-group/higress) | OSS AI-native gateway | ⭐ 8.7k | Apache-2.0 | ✅ | ✅ | ✅ | ✅ | ✅ | | [GPT-Load](https://github.com/tbphp/gpt-load) | OSS key-pool proxy | ⭐ 6.2k | MIT | ✅ | ✅ key rotation | ❌ | ❌ | ➕ | | [Bifrost](https://github.com/maximhq/bifrost) | OSS gateway (Go) | ⭐ 6.1k | Apache-2.0 | ✅ | ✅ adaptive | ✅ | ✅ | ✅ | | [Helicone](https://github.com/Helicone/helicone) | OSS observability + gateway | ⭐ 5.9k | Apache-2.0 | ✅ | ✅ | ✅ | ➕ | ✅ | | [Envoy AI Gateway](https://github.com/envoyproxy/ai-gateway) | OSS K8s gateway | ⭐ 1.8k | Apache-2.0 | ✅ | ✅ | ➕ | ➕ | ✅ | | [OpenRouter](https://openrouter.ai) | SaaS marketplace | — | Commercial | ✅ 400+ | ✅ | ✅ | ➕ | ✅ | | [Vercel AI Gateway](https://vercel.com/ai-gateway) | SaaS (0% markup) | — | Commercial | ✅ 100s | ✅ | ❌ | ❌ | ✅ | | [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/) | SaaS control plane | — | Commercial (free tier) | ✅ | ✅ dynamic | ✅ | ✅ | ✅ budgets | ¹ LiteLLM core is MIT; the repo contains a separately licensed enterprise directory. > 📂 **Browse the raw data** (machine-readable, CC0): [models & pricing JSON](data/models.json) · [cost table CSV](data/cost_table.csv) · [gateway scorecard CSV](data/gateways_scorecard.csv). Every cost cell is regenerated from this data by a [unit-tested script](scripts/cost_calc.py). > _The full directory at a glance — browse the sections below by your need._ ## 💰 Cost-first: cheapest multi-model access *Pain point: "I want many models for the least money and zero ops."* - [OpenRouter](https://openrouter.ai) — The dominant model marketplace: 400+ models behind one OpenAI-compatible API, pay-as-you-go with automatic failover; ~5.5% fee when buying credits. $113M Series B (May 2026), ~8M users. - [Vercel AI Gateway](https://vercel.com/ai-gateway) — Hundreds of models at **provider list price (0% markup)**, $5/month free credits, zero-data-retention option; pairs naturally with the AI SDK. - [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/) — Free control plane in front of your own provider keys: caching, dynamic routing, unified billing, and dollar-denominated spend limits (2026 beta). - [Requesty](https://requesty.ai) — EU-friendly OpenRouter alternative: 400+ models, sub-20ms failover, ~5% markup. - [Eden AI](https://www.edenai.co) — Unified API for 500+ models plus vision/OCR/speech; EU-based, ~5.5% platform fee. - [Helicone AI Gateway (cloud)](https://www.helicone.ai) — Passthrough billing at **0% markup** with observability bundled. - [GPT-Load](https://github.com/tbphp/gpt-load) ⭐ 6.2k — High-performance Go proxy that rotates pools of API keys across channels to maximize quota usage. - [Loop Gateway](https://github.com/Loop-XXI/loop-gateway) — OpenAI-compatible proxy that meters every request in Bitcoin sats instead of dollars. 311 models via OpenRouter at a 15% markup. No accounts, no email, no card; top up over Lightning, get a bearer token. Three auth rails (prepaid bearer, L402, Cashu). Self-hostable in Go via docker-compose, live at [api.loopxxi.com](https://api.loopxxi.com). **New & unverified** (anonymous 1★ repo) — it resells frontier models *through the operator's own OpenRouter account* at a 15% markup, and account-less + crypto-prepaid means no recourse if it swaps models or vanishes; confirm fidelity with [canary_check.py](scripts/canary_check.py) and only top up what you can afford to lose. - [nullsink](https://nullsink.is) ([repo](https://github.com/nullsink/nullsink)) — Account-less, metered proxy for frontier-model APIs, paid in Monero or Bitcoin. No accounts, no email, no card; mint a bearer token, prepay on-chain, and point the official SDKs at one base URL. ~10% markup taken once at top-up; no IP logging, no request logs; payment and token kept unlinkable. Self-hostable single binary (TypeScript/Bun, AGPL-3.0), live at [nullsink.is](https://nullsink.is). **New & unverified** (repo created 2026-06, 3★) — account-less + crypto-prepaid + no logs means no recourse if it swaps models or vanishes; confirm fidelity with [canary_check.py](scripts/canary_check.py) and only top up what you can afford to lose. - [AIMLAPI](https://aimlapi.com) — One OpenAI/Anthropic-compatible endpoint fronting 400+ models (chat, image, video, audio, embeddings); prepaid, OpenRouter-style aggregator. - [Novita AI](https://novita.ai) — Unified API to 200+ open-source models (DeepSeek/Qwen/Llama…) with load balancing, autoscaling and failover; also a GPU cloud. - [FlintAPI](https://flintapi.ai) ([repo](https://github.com/moozechen/flintapi)) — Hosted OpenAI-compatible gateway aggregating 25+ Chinese LLMs (DeepSeek, Qwen, Kimi, GLM, MiniMax) with $2 free credits. New and unverified — confirm model fidelity (e.g. with [canary_check.py](scripts/canary_check.py)) before relying on it in production. - [FlowBar](https://flowbarai.com) — Hosted OpenAI-compatible relay reselling 50+ models (GPT, Claude, Gemini, DeepSeek, Qwen, GLM, Kimi) below OpenRouter, with USD/CNY/crypto payment. New and unverified — confirm model fidelity (e.g. with [canary_check.py](scripts/canary_check.py)) before relying on it in production. - [Meshs One](https://api.meshs.one) — Hosted OpenAI-compatible relay fronting Chinese frontier models (DeepSeek-V4, Qwen3.7-Max, MiniMax-M3) under one key, per-token pay-as-you-go (its `/v1` endpoint returns a `new_api_error`, so it appears to run on [new-api](https://github.com/QuantumNous/new-api)). New & unverified — closed-source and brand-new; confirm model fidelity (e.g. with [canary_check.py](scripts/canary_check.py)) before relying on it in production. - [lxg2it ModelRouter](https://api.lxg2it.com) ([repo](https://github.com/lxg2it/modelrouter-core)) — Solo-built, OpenAI-compatible router over 7+ providers (Anthropic, OpenAI, Google, Cerebras, Groq, Grok, GLM) with tiered automatic fallback that selects the cheapest available model. Free tier plus a paid tier advertised at 0% markup on Anthropic (a deposit fee may apply — verify current pricing). New and unverified — the public repo is now a **deprecated reference stub** (routing moved to the closed hosted service; still no license file) — confirm model fidelity (e.g. with [canary_check.py](scripts/canary_check.py)) before relying on it in production. - [OpenPaths](https://openpaths.io) ([repo](https://github.com/lee101/openpaths)) — Hosted OpenAI-compatible router auto-routing across 15+ providers (OpenAI, Anthropic, Gemini, Groq, xAI, DeepSeek, Mistral) under one API, spanning chat, image, video, music, speech, embeddings and transcription. New & unverified — despite the "open source" framing the GitHub repo is a no-code, unlicensed marketing mirror (canonical code lives on the third-party [Codex Infinity](https://codex-infinity.com/lee101/openpaths) platform and is agent-maintained), so treat it as a closed hosted relay and confirm model fidelity (e.g. with [canary_check.py](scripts/canary_check.py)) before relying on it in production. - [Glama Gateway](https://glama.ai/ai/gateway) — OpenAI-compatible gateway to 100+ models with consolidated billing, caching and logging (OSS core [glama-ai/lightport](https://github.com/glama-ai/lightport)). > 💡 Squeeze more from any gateway: enable **semantic caching** (Kong, Bifrost, Zuplo), set **spend limits** (Cloudflare, Zuplo, Pydantic/Logfire), and route easy prompts to cheap models (see [Smart routing](#-smart-routing--model-selection)). ## 🔓 Self-hosted open source *Pain point: "My keys, my infra, no per-token middleman fee."* - [LiteLLM](https://github.com/BerriAI/litellm) ⭐ 51.9k — The default choice: Python SDK + proxy server speaking OpenAI format to 100+ providers, with virtual keys, budgets, load balancing and guardrails. - [Portkey Gateway](https://github.com/Portkey-AI/gateway) ⭐ 12.2k — Fast TypeScript gateway (1,600+ models, 50+ guardrails) that also powers Portkey's com … ## Source & license This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [cuihuan](https://github.com/cuihuan) - **Source:** [cuihuan/awesome-ai-gateway](https://github.com/cuihuan/awesome-ai-gateway) - **License:** CC0-1.0 - **Homepage:** https://cuihuan.github.io/awesome-ai-gateway/ Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/mcp-cuihuan-awesome-ai-gateway - Seller: https://agentstack.voostack.com/s/cuihuan - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.