# Caduceus > Provider-agnostic transport and routing for LLMs: normalize OpenAI, Anthropic, and Gemini, route by cost, quality, or latency with per-model circuit breakers, USD budgets, automatic failover, and an auditable route trace. Zero dependencies, node-free core, TypeScript-first. - **Type:** MCP server - **Install:** `agentstack add mcp-davccavalcante-caduceus` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [davccavalcante](https://agentstack.voostack.com/s/davccavalcante) - **Installs:** 0 - **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml) - **Latest version:** 0.1.0 - **License:** Apache-2.0 - **Upstream author:** [davccavalcante](https://github.com/davccavalcante) - **Source:** https://github.com/davccavalcante/caduceus - **Website:** https://davccavalcante.github.io/caduceus/ ## Install ```sh agentstack add mcp-davccavalcante-caduceus ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # Caduceus [](./CHANGELOG.md) [](https://www.npmjs.com/package/@takk/caduceus) [](./LICENSE) [](./SPEC.md) [](./package.json) **One shape for every model.** Caduceus is the open, local-first, provider-agnostic transport layer for Massive Intelligence (IM). It normalizes OpenAI, Anthropic, and Gemini, routes each request across a pool of heterogeneous models by cost, quality, or latency, fails over when a provider is down, holds every call to a hard USD ceiling, and emits an auditable trace that says which model answered. Zero runtime dependencies, a node-free core, TypeScript-first. It is the peripheral nervous system of the [@takk/coryphaeus](https://www.npmjs.com/package/@takk/coryphaeus) orchestrator, and it works perfectly well on its own. Where the market moved from prompt to agent, the next category is the non-human entity: MAIC, HIMs, and NHEs. Caduceus is the transport such entities route over, Massive Intelligence (IM) infrastructure for automation beyond OpenClaw, Hermes Agent, and Claude Code, run on your own infrastructure with your own keys, fully auditable. > Caduceus is the package (display name). `caduceus` is the npm scope token, the CLI binary, and the import path. The core never speaks HTTP; the network call lives in an injectable dispatch, so the routing logic is provider-agnostic, runs on the edge, and is testable offline. ## Why Every provider has a different request body, a different streaming format, and a different tool-call shape. Routing dynamically by task, cost, latency, and quality, with automatic failover when one falls over, is usually done ad hoc and is fragile. The gateways that promise to solve it (a hosted single endpoint) hide which model answered, can't be configured, price at the ceiling of the pool they route, and are blocked in some regions. Caduceus is the opposite: it runs on your infrastructure with your keys, every routing decision is visible, the strategies are yours to choose, and dollar ceilings are hard. - **Provider-agnostic.** OpenAI, Anthropic, and Gemini request, response, streaming, and tool calls normalized to one typed vocabulary. An OpenAI-compatible shape covers Ollama, vLLM, Together, Groq, and any `/v1/chat/completions` gateway. - **Resilient.** Per-model circuit breakers (closed, open, half-open), rate-limit recovery that honors `Retry-After` and retries before failover, and automatic failover, so one provider going down or rate-limiting you does not take your request with it. - **Auditable.** A route trace names every model tried, its breaker state, its cost and latency, and which one answered. It renders to DOT or Mermaid and seals with SHA-256. - **Cost-controlled.** USD forecasts before every call and a hard per-request ceiling that raises a typed error instead of a surprise bill. - **Portable.** A node-free core runs on Cloudflare Workers, Vercel Edge, Bun, Deno, and the browser. The one bundled real transport (`fetch`) lives in the Node entry. ## Install ```sh npm install @takk/caduceus # or: pnpm add @takk/caduceus ``` Node 20+. Ships ESM and CJS with types for every subpath. ## Quickstart ```ts import { Caduceus, defineProvider, defineModel } from "@takk/caduceus"; import { createFetchDispatch } from "@takk/caduceus/node"; const providers = [ defineProvider({ id: "openai", shape: "openai", baseUrl: "https://api.openai.com/v1" }), defineProvider({ id: "anthropic", shape: "anthropic", baseUrl: "https://api.anthropic.com/v1" }), defineProvider({ id: "google", shape: "gemini", baseUrl: "https://generativelanguage.googleapis.com/v1beta" }), ]; const models = [ defineModel({ id: "gpt", provider: "openai", model: "gpt-5", cost: { inputPerMTok: 5, outputPerMTok: 15 }, quality: 0.9 }), defineModel({ id: "sonnet", provider: "anthropic", model: "claude-sonnet-4-6", cost: { inputPerMTok: 3, outputPerMTok: 15 }, quality: 0.85 }), defineModel({ id: "flash", provider: "google", model: "gemini-flash", cost: { inputPerMTok: 0.3, outputPerMTok: 1.2 }, quality: 0.6 }), ]; // The dispatch carries your credentials; the core never sees a key. const dispatch = createFetchDispatch({ auth: (call) => { if (call.shape === "anthropic") return { "x-api-key": process.env.ANTHROPIC_API_KEY!, "anthropic-version": "2023-06-01" }; if (call.shape === "gemini") return { "x-goog-api-key": process.env.GEMINI_API_KEY! }; return { authorization: `Bearer ${process.env.OPENAI_API_KEY}` }; }, }); const caduceus = new Caduceus({ dispatch, providers, models, strategy: "cost-then-quality", budgetUsd: 0.05 }); const result = await caduceus.complete({ messages: [{ role: "user", content: "Name one routing strategy." }], }); console.log(result.winner); // { model: "flash", provider: "google" } console.log(result.response.text); // the answer, normalized console.log(result.totalCostUsd); // priced in USD console.log(result.attempts); // every model tried, with breaker state and cost ``` No keys to hand? Swap `createFetchDispatch(...)` for `createEchoDispatch()` from `@takk/caduceus/codec` and the whole thing runs offline and deterministically, the same client the tests and the CLI `--mock` use. ## Streaming ```ts const route = await caduceus.stream({ messages: [{ role: "user", content: "Stream a haiku." }] }); for await (const event of route.stream) { if (event.type === "text-delta") process.stdout.write(event.text); } ``` OpenAI chunk deltas, Anthropic content-block events, and Gemini partial candidates all arrive as the same typed `StreamEvent` union (`start`, `text-delta`, `tool-call-delta`, `finish`, `error`). ## The seven strategies A strategy orders the candidate models; the router then tries them in order, skipping open breakers and failing over on error. | Strategy | Orders by | | --- | --- | | `cost-then-quality` | cheapest first, ties broken by higher quality (the default) | | `cost-first` | cheapest first | | `quality-first` | highest quality first, ties broken by cost | | `latency-first` | lowest latency hint first | | `weighted` | highest selection weight first | | `round-robin` | rotates the starting model each request | | `sequential-cascade` | cheapest-and-weakest first, escalating to stronger models | Pick a default at construction and override per request with `complete(request, { strategy })`. Inspect the ordering and forecast without spending anything: ```ts const plan = caduceus.plan(request, { strategy: "quality-first" }); // plan.candidates -> ordered, each with an estimatedCostUsd; plan.estimatedCostUsd is the first attempt ``` ## Circuit breakers and budgets ```ts const caduceus = new Caduceus({ dispatch, providers, models, breaker: { failureThreshold: 3, cooldownMs: 30_000 }, // trip after 3 failures, probe after 30s budgetUsd: 0.05, // a hard per-request ceiling }); ``` A model that fails three times in a row has its breaker tripped open; the router skips it (recording `skipped-open`) until the cooldown elapses, then lets a single probe through. A request whose cheapest affordable candidate is still over the ceiling raises a typed `BUDGET_EXCEEDED` rather than dispatching. `caduceus.breakerSnapshot()` reports the live state of every model. ## Rate-limit recovery In production a provider rarely goes down; it returns a 429. Caduceus does not fail over blindly on a rate-limit: it waits the provider's `Retry-After` (or an exponential backoff, both capped) and retries the same model before failing over, recording the retry count and wait in the trace. ```ts const caduceus = new Caduceus({ dispatch, providers, models, retry: { maxRetries: 2, baseDelayMs: 500, maxDelayMs: 30_000 }, }); ``` Pair it with key rotation so a 429 on one key retries with the next, the [@takk/keymesh](https://www.npmjs.com/package/@takk/keymesh) seam without a hard dependency: ```ts import { withKeyRotation } from "@takk/caduceus/adapter"; const dispatch = withKeyRotation((key) => createFetchDispatch({ auth: () => ({ authorization: `Bearer ${key}` }) }), keyPool); ``` ## The auditable trace ```ts import { renderTraceDot, provenanceOf, sealTrace } from "@takk/caduceus/trace"; provenanceOf(result); // { model, provider } that answered renderTraceDot(result); // a Graphviz transport graph of every attempt await sealTrace(result); // a SHA-256 integrity seal over the canonical trace ``` This is the answer to the gateway black box: you can see, render, and seal which model responded and why, on every request. ## Drop it into what you already have ```ts import { dispatchFromVercel } from "@takk/caduceus/adapter"; import { generateText } from "ai"; // the Vercel AI SDK const dispatch = dispatchFromVercel(({ model, messages }) => generateText({ model: yourModel(model), messages })); const caduceus = new Caduceus({ dispatch, providers, models }); ``` Expose the router as a framework-agnostic tool for an MCP server or any tool-calling API: ```ts import { createTransportTool } from "@takk/caduceus/adapter"; const tool = createTransportTool(caduceus); // name: "caduceus_route", with a JSON Schema and a JSON-safe handler ``` Feed the [@takk/coryphaeus](https://www.npmjs.com/package/@takk/coryphaeus) orchestrator's agent pool with no hard dependency: ```ts import { agentClientFromCaduceus } from "@takk/caduceus/adapter"; const client = agentClientFromCaduceus(caduceus); // satisfies the @takk/coryphaeus AgentClient contract ``` ## Does failover actually help? The benchmark (`benchmarks/transport-benchmark.mjs`) runs the built router over a controlled failure model on a fixed seed, 4000 requests per regime. It is a mechanism demonstration, not an LLM quality claim. | Regime | Without Caduceus | With Caduceus | | --- | --- | --- | | Independent 40% provider failure | single provider 59.4% success | three-way failover 93.5% success (bound 1 − p³ = 93.6%) | | One provider fully down | 8000 dispatch calls | 4005 with the breaker (49.9% of wasted calls cut) | Failover converts independent provider unreliability into near-certain success; the circuit breaker stops paying to call a provider that is down. The numbers above come from running the benchmark end to end, never invented. ## CLI ```sh caduceus providers --providers providers.json --models models.json caduceus plan request.json --providers providers.json --models models.json --strategy quality-first caduceus route request.json --providers providers.json --models models.json --mock --out trace.json caduceus trace trace.json --format mermaid ``` Exit codes: `0` ok, `2` usage or input error, `21` no route, `30` budget exceeded. The CLI carries no provider credentials; `route` requires `--mock`, and real runs use the library API. ## Subpath exports | Import | What it gives you | | --- | --- | | `@takk/caduceus` | the barrel: `Caduceus`, providers, codec, strategies, breakers, budget, trace, adapters, errors | | `@takk/caduceus/transport` | the `Caduceus` router | | `@takk/caduceus/provider` | `defineProvider`, `defineModel`, `ProviderRegistry` | | `@takk/caduceus/codec` | the compatibility matrix and `createEchoDispatch` | | `@takk/caduceus/strategy` | the seven strategies | | `@takk/caduceus/breaker` | `CircuitBreaker`, `BreakerRegistry` | | `@takk/caduceus/budget` | token estimation, USD pricing, `BudgetTracker` | | `@takk/caduceus/trace` | provenance, DOT and Mermaid rendering, the SHA-256 seal | | `@takk/caduceus/adapter` | Vercel and OpenAI bridges, the @takk/coryphaeus AgentClient, the routing tool | | `@takk/caduceus/node` | the real `fetch` dispatch and the JSON loaders (the only entry touching a Node built-in) | | `@takk/caduceus/edge` | the node-free core for Workers, Edge, Bun, Deno, and the browser | See [SPEC.md](./SPEC.md) for the full surface and [CHANGELOG.md](./CHANGELOG.md) for what shipped. ## Not in 1.0 A learned, quality-measured router (the 1.0 strategies are transparent and deterministic; a measured strategy with evaluators and bandits is planned behind the same `Strategy` seam); bundled provider SDKs; a hosted gateway; a runnable MCP server binary; multimodal content parts; and signed, timestamped trace seals. Each is planned for a later release. ## License [Apache-2.0](./LICENSE), © David C Cavalcante, [Takk Innovate Studio](https://takk.ag). See [NOTICE](./NOTICE). ## Source & license This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [davccavalcante](https://github.com/davccavalcante) - **Source:** [davccavalcante/caduceus](https://github.com/davccavalcante/caduceus) - **License:** Apache-2.0 - **Homepage:** https://davccavalcante.github.io/caduceus/ Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** yes - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/mcp-davccavalcante-caduceus - Seller: https://agentstack.voostack.com/s/davccavalcante - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.