# Llmrouter

> Simple router to aggregate multiple LLM providers with optional smart routing and MCP aggretation.

- **Type:** MCP server
- **Install:** `agentstack add mcp-paularlott-llmrouter`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [paularlott](https://agentstack.voostack.com/s/paularlott)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [paularlott](https://github.com/paularlott)
- **Source:** https://github.com/paularlott/llmrouter

## Install

```sh
agentstack add mcp-paularlott-llmrouter
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# LLM Router

A unified gateway that aggregates multiple LLM providers behind a single endpoint. Clients use their preferred protocol (OpenAI or Anthropic Messages) and the gateway handles routing, protocol translation, and load balancing.

## Features

- **Multi-Provider**: OpenAI, Claude, Gemini, Ollama, Mistral, ZAi — configure once, route by model name
- **Protocol Translation**: Clients speak OpenAI or Anthropic Messages; the gateway translates as needed, and also exposes a full Ollama-compatible API under `/ollama`
- **Weight-Based Load Balancing**: Distribute load across providers with configurable weights
- **Smart Routing**: Request the `auto` model and a [Scriptling](https://scriptling.dev/) script picks the best provider/model based on tags, load, and request content
- **MCP Aggregator**: Combine tools, resources, and prompts from multiple remote MCP servers with namespace isolation, OAuth support, and per-tool visibility / allow / deny filtering
- **Chat UI**: Built-in interface at `/chat` with personas, conversation history, slash commands, `@prompt` / `@resource` menus, live MCP tool calling, markdown rendering, and `skill://` resources auto-surfaced to the model
- **Personas**: System prompts, default models, and generation parameters — defined in the config file or managed through the admin UI
- **Admin UI**: Optional web interface at `/admin` to manage providers, personas, and MCP servers (create / edit / delete, enable / disable), browse models (with rescan) and tools, and test-run MCP tools directly from the browser
- **Responses API**: OpenAI-compatible responses storage (emulated for all providers)
- **Conversations API**: n8n-compatible conversation management
- **Optional Auth**: Bearer token protection for all endpoints

## Installation

### Homebrew (macOS & Linux)

```bash
brew tap paularlott/tap
brew install llmrouter
```

### Download from GitHub Releases

Download the latest release for your platform from [github.com/paularlott/llmrouter/releases](https://github.com/paularlott/llmrouter/releases):

| Platform | Architecture          | Download                      |
| -------- | --------------------- | ----------------------------- |
| macOS    | Intel (AMD64)         | `llmrouter-darwin-amd64.zip`  |
| macOS    | Apple Silicon (ARM64) | `llmrouter-darwin-arm64.zip`  |
| Linux    | AMD64                 | `llmrouter-linux-amd64.zip`   |
| Linux    | ARM64                 | `llmrouter-linux-arm64.zip`   |
| Windows  | AMD64                 | `llmrouter-windows-amd64.zip` |
| Windows  | ARM64                 | `llmrouter-windows-arm64.zip` |

Extract the archive and place the binary in your PATH.

### Build from Source

```bash
git clone https://github.com/paularlott/llmrouter.git
cd llmrouter
go build -o llmrouter .
./llmrouter server
./llmrouter -config /path/to/config.toml server
```

## Configuration

```toml
[server]
host = "0.0.0.0"
port = 12345
token = "your-secret-token"   # Optional bearer token
admin_password = "admin123"   # Optional: enables admin UI at /admin
storage_path = "./data"  # Omit for memory-only storage (under [server])

[logging]
level = "info"    # trace | debug | info | warn | error
format = "console" # console | json

[responses]
ttl_days = 30

[conversations]
ttl_days = 30

[[providers]]
name = "openai"
provider = "openai"           # openai | claude | gemini | ollama | mistral | zai
token = "sk-..."
enabled = true
weight = 1.0                  # 0.0-2.0, default 1.0; higher = preferred
tags = ["capable", "expensive"]  # optional tags for smart routing

[providers.model_tags]        # optional per-model tags
"gpt-4o"      = ["capable", "expensive"]
"gpt-4o-mini" = ["fast", "cheap"]

[providers.model_aliases]     # optional alias -> real model name, resolved per-provider
"gpt4"     = "gpt-4o"
"gpt4-mini" = "gpt-4o-mini"

[[providers]]
name = "anthropic"
provider = "claude"
token = "sk-ant-..."
enabled = true
model_allowlist = ["claude-opus-4-5", "claude-sonnet-4-5"]  # Required for Claude
tags = ["capable"]

[providers.model_aliases]     # same alias "fast", different real model on this provider
"fast" = "claude-sonnet-4-5"

[providers.model_tags]
"claude-opus-4-5"   = ["capable", "expensive"]
"claude-sonnet-4-5" = ["capable", "fast"]

[[providers]]
name = "google"
provider = "gemini"
token = "your-google-key"
enabled = true
model_allowlist = ["gemini-2.5-flash-lite"]  # Optional: restrict to specific models
models = ["gemini-2.5-flash-lite"]           # Optional: override model list entirely (still health-checks the API)

[[providers]]
name = "local"
provider = "ollama"
base_url = "http://localhost:11434/v1"
enabled = true

[smart_routing]
enabled = false
script = "router.py"  # Scriptling script for routing decisions
default_model = "mistralai/ministral-3-3b"  # Fallback if script returns nothing
libdir = "./router_libs"  # Optional: directory of .py script libraries

[mcp]
[[mcp.remote_servers]]
namespace = "tools"
url = "https://tools.example.com/mcp"
token = "secret"
tool_visibility = "native"    # native | discoverable
tool_allowlist = ["search", "query"]  # Optional: only these tools are enabled
tool_denylist = ["delete"]           # Optional: these tools are disabled
```

### Provider Types

| Provider  | Default Base URL                              | Embeddings | Model Discovery  |
| --------- | --------------------------------------------- | ---------- | ---------------- |
| `openai`  | https://api.openai.com/v1                     | Yes        | Auto             |
| `claude`  | https://api.anthropic.com/v1                  | No         | **Must specify** |
| `gemini`  | https://generativelanguage.googleapis.com/... | Yes        | Auto             |
| `ollama`  | https://ollama.com/v1/                        | Yes        | Auto             |
| `mistral` | https://api.mistral.ai/v1                     | Yes        | Auto             |
| `zai`     | https://api.z.ai/api/paas/v4/                 | Yes        | Auto             |

`base_url` is optional — each provider has a built-in default. Set it to override (e.g. local LM Studio).

`model_allowlist` restricts the provider to only the listed models. For Claude this is required (no discovery API). For other providers it is optional.

`model_denylist` excludes specific models from auto-discovery. Ignored when `model_allowlist` is set.

`models` overrides the model list entirely — the provider's `/models` API is still called (for health checks) but its response is discarded and the configured list is used instead. Useful for providers that return no models or an incomplete list.

`model_aliases` maps short or friendly names to real model IDs. Aliases appear in `/v1/models` alongside real models and support the same weight-based load balancing and round-robin behaviour. When multiple providers define the same alias, each provider maps it to its own real model — requests via that alias are distributed across all of them, and each provider receives its own real model name.

### Smart Routing

When a client requests the model name `auto`, the router runs a [Scriptling](https://scriptling.dev/) script to pick the best provider and model. If the script returns nothing or fails, `default_model` is used.

`libdir` is an optional directory of `.py` files. Each file is registered as a script library (named after the file without the `.py` extension) and is available for `import` in the routing script. The directory is watched for changes — any modification triggers a full VM pool rebuild.

```toml
[smart_routing]
enabled = true
script = "router.py"  # Scriptling script for routing decisions
default_model = "mistralai/ministral-3-3b"

[smart_routing.vars]  # Optional key-value pairs exposed to the script
openai_key = "sk-..."
my_endpoint = "https://api.example.com"
```

The `auto` model appears in `/v1/models` so clients can discover it.

#### Provider and Model Tags

Tags are arbitrary strings assigned to providers and individual models. The routing script uses them to select the right provider/model for each request.

```toml
[[providers]]
name = "mistral"
provider = "mistral"
token = "..."
enabled = true
tags = ["fast", "cheap"]           # provider-level tags

[providers.model_tags]
"mistralai/ministral-3-3b"      = ["small", "fast", "cheap"]
"mistralai/mistral-small-latest" = ["small", "fast"]
```

The routing script can then query by tag:

```python
import router

req = router.get_request()

# Route tool-heavy requests to a capable model
if router.is_chat_completion() and len(req["tools"]) > 0:
    models = router.models_by_tag("capable")
else:
    models = router.models_by_tag("cheap")

if models:
    router.set_model(models[0])
```

Use `router.model_tags(model_id)` to narrow a candidate list by a secondary tag:

```python
import router

# Start broad: all "capable" models
candidates = router.models_by_tag("capable")

# Narrow: prefer "super_fast" within that set
fast = [m for m in candidates if "super_fast" in router.model_tags(m)]
cheap = [m for m in candidates if "cheap" in router.model_tags(m)]

models = fast or cheap or candidates
if models:
    router.set_model(models[0])
```

See [docs/scriptling-router-library.md](docs/scriptling-router-library.md) for the full script API reference.

#### Script Libraries

Every routing script has access to all Scriptling standard libraries (`json`, `re`, `math`, `random`, `hashlib`, `base64`, `uuid`, `datetime`, `time`, `urllib`, etc.) plus the following extended and Scriptling-specific libraries:

| Library                    | Description                                   |
| -------------------------- | --------------------------------------------- |
| `requests`                 | HTTP client                                   |
| `secrets`                  | Cryptographically strong random numbers       |
| `html.parser`              | HTML/XHTML parser                             |
| `logging`                  | Logging to the router log                     |
| `yaml`                     | YAML parsing                                  |
| `toml`                     | TOML parsing                                  |
| `sys`                      | System parameters                             |
| `scriptling.ai`            | AI/LLM client for OpenAI-compatible APIs      |
| `scriptling.ai.agent`      | Agentic AI loop with automatic tool execution |
| `scriptling.mcp`           | MCP tool interaction                          |
| `scriptling.toon`          | TOON encoding/decoding                        |
| `scriptling.similarity`    | String matching and similarity utilities      |
| `scriptling.net.resolve`   | DNS resolution (IP, SRV, srv+http URLs)       |
| `scriptling.template.html` | HTML template rendering                       |
| `scriptling.template.text` | Text template rendering                       |
| `scriptling.runtime`       | Background tasks, KV store, sync primitives   |

Filesystem access (`os`, `pathlib`, `glob`), subprocess execution, and `wait_for` are not available in routing scripts.

#### Script Variables

Use `[smart_routing.vars]` to pass tokens or other config to the script without hard-coding them:

```toml
[smart_routing.vars]
openai_key = "sk-..."
```

```python
import vars
import scriptling.ai as ai

client = ai.Client("", api_key=vars.openai_key)
```

### Weight-Based Load Balancing

When multiple providers serve the same model, the router selects using `score = active_completions / weight`. Lower score wins.

| Weight | Effect                                |
| ------ | ------------------------------------- |
| `0.0`  | Last resort only                      |
| `1.0`  | Normal (default)                      |
| `2.0`  | Preferred — gets 2× the traffic share |

### MCP Tool Visibility

| Mode           | Behavior                                            |
| -------------- | --------------------------------------------------- |
| `native`       | Tools appear in `tools/list`, directly callable     |
| `discoverable` | Hidden from list, searchable via `tool_search` only |

### MCP Tool Filtering

Tools from remote MCP servers can be filtered using `tool_allowlist` or `tool_denylist`:

- **`tool_allowlist`**: When defined, only the listed tools are enabled. All other tools are disabled.
- **`tool_denylist`**: When defined, all tools are enabled except those in the list.

Note: If both are defined, `tool_allowlist` takes precedence and `tool_denylist` is ignored.

```toml
[[mcp.remote_servers]]
namespace = "github"
url = "https://github.example.com/mcp"
token = "secret"
tool_visibility = "native"
tool_allowlist = ["search_repos", "get_issue", "create_issue"]  # Only these 3 tools are enabled

[[mcp.remote_servers]]
namespace = "slack"
url = "https://slack.example.com/mcp"
token = "secret"
tool_visibility = "native"
tool_denylist = ["delete_message", "ban_user"]  # All tools enabled except these 2
```

### Scripting Tools

Define custom MCP tools using `.toml` (metadata) and `.py` (implementation) file pairs. Tools are loaded from a configurable directory and automatically reloaded when files change.

```bash
./llmrouter server --tools-dir ./tools --plugin-dir ./plugins --libpath ./libs
```

**Directory structure:**

```
tools/
├── calculator.toml
├── calculator.py
├── weather.toml
└── weather.py
```

**Tool metadata (`.toml`):**

```toml
description = "Calculate the sum of two numbers"
keywords = ["math", "add", "sum"]

[[parameters]]
name = "a"
type = "int"
description = "First number"
required = true

[[parameters]]
name = "b"
type = "int"
description = "Second number"
required = true
```

**Tool implementation (`.py`):**

```python
import scriptling.mcp.tool as tool

a = tool.get_int("a")
b = tool.get_int("b")

result = a + b
tool.return_string(f"{a} + {b} = {result}")
```

**Configuration:**

```toml
[scripting]
tools_dir = "./tools"              # Directory containing .toml/.py tool pairs
plugin_dirs = ["./plugins"]        # Directories containing plugin executables
lib_paths = ["./libs"]             # Additional directories for scriptling libraries
```

Tools can be marked as `discoverable = true` to hide them from `tools/list` and make them searchable via `tool_search` only.

### Chat UI

When `admin_password` is set and a personas directory is configured, a built-in chat interface is available at `/chat` with conversation history, personas, slash commands, `@prompt` / `@resource` menus, MCP tool calling, and markdown rendering.

```bash
./llmrouter server --personas-dir ./personas --commands-dir ./commands --resources-dir ./resources
```

**Skills:** Resources with a `skill://` URI prefix are automatically surfaced to the LLM. On every chat request, the router queries the MCP server for `skill://` resources and appends their names and descriptions to the persona's system prompt. A virtual tool (`lmchatkit__get_skill`) is injected into the tool list — the model calls it to retrieve a skill's full instructions on demand. The tool routes to the standard MCP `ReadResource` API, so skills work from any source (files, remote servers, or custom providers). The tool is auto-approved (no user prompt) since it's a read-only context fetch. Skills are transient — the stored conversation is not modified; the augmentation is recomputed on each request.

Example skill resource directory structure:
```
resources/
└── skill/
    ├── golang.md    → skill://golang.md
    └── testing.md   → skill://testing.md
```

### MCP OAuth Authentication

For MCP servers that require OAuth authentication, configure the OAuth fields instead of `token`:

```toml
[[mcp.remote_servers]]
namespace = "my-oauth-service"
url = "https://api.example.com/mcp"
auth_type = "oauth"
oauth_client_id = "your-client-id"
oauth_token_url = "https://auth.example.com/token"
oauth_access_token = "current-access-token"
oauth_refresh_token = "refresh-token"  # Optional: for token refresh
```

## API Endpoints

All endpoints are available under both `/v1/`

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [paularlott](https://github.com/paularlott)
- **Source:** [paularlott/llmrouter](https://github.com/paularlott/llmrouter)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** yes
- **Shell / process execution:** yes
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-paularlott-llmrouter
- Seller: https://agentstack.voostack.com/s/paularlott
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.