ElBruno ModelContextProtocol

ElBruno.ModelContextProtocol

[](https://github.com/elbruno/ElBruno.ModelContextProtocol/actions/workflows/build.yml) [](https://github.com/elbruno/ElBruno.ModelContextProtocol/actions/workflows/publish.yml) [](https://opensource.org/licenses/MIT) [](https://github.com/elbruno/ElBruno.ModelContextProtocol)

Semantic routing for MCP tools 🔀

ElBruno.ModelContextProtocol is a .NET library that makes it easy to find the right tools from Model Context Protocol (MCP) tool definitions. It uses semantic search powered by local embeddings to route prompts to the most relevant tools, enabling intelligent tool selection without external API calls. By routing prompts to only relevant tools before sending to an LLM, you can reduce token costs by 70–85%.

Packages

| Package | NuGet | Downloads | Description | |---------|-------|-----------|-------------| | ElBruno.ModelContextProtocol.MCPToolRouter | [](https://www.nuget.org/packages/ElBruno.ModelContextProtocol.MCPToolRouter) | [](https://www.nuget.org/packages/ElBruno.ModelContextProtocol.MCPToolRouter) | Semantic tool routing for MCP |

MCPToolRouter

A high-performance semantic search engine for Model Context Protocol tools. MCPToolRouter indexes your MCP tool definitions and returns the most relevant tools for any prompt using vector similarity search.

Installation

dotnet add package ElBruno.ModelContextProtocol.MCPToolRouter

TL;DR

// Mode 1: Embeddings only — fast, no LLM needed
var results = await ToolRouter.SearchAsync(prompt, tools, topK: 3);

// Mode 2: LLM-assisted — best for verbose/complex prompts
var results = await ToolRouter.SearchUsingLLMAsync(prompt, tools, topK: 5);

---

How It Works — Two Modes

MCPToolRouter supports two distinct modes for finding the right tools. Choose based on your prompt complexity and speed requirements:

The Pipeline

┌─────────────────────────────────────────────────────────────────┐
│  Mode 1: Embeddings Filter (Fast, Simple)                       │
│                                                                  │
│  User Prompt ──► Embed ──► Cosine Similarity ──► Top-K Tools    │
│                                                                  │
│  "What's the weather?" → [0.89 get_weather, 0.45 send_email]   │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│  Mode 2: Hybrid Search (Precise, Complex)                        │
│                                                                  │
│  User Prompt ──┬──► Embed (baseline) ────────────────┐           │
│                └──► LLM Distill ──► Split Phrases ──►├─► Merge   │
│                     ──► Embed Each ──────────────────┘  ──► TopK │
│                                                                  │
│  "Hey, I was thinking about my trip and need to know if it's    │
│   going to rain in Tokyo..." → "check weather Tokyo,            │
│   set reminder" → search each + merge with baseline             │
└─────────────────────────────────────────────────────────────────┘

---

Mode 1: Embeddings Filter (No LLM Needed) — One-Liner

Use when: Your prompt is clear and single-intent.

What it does: Embeds your prompt locally using ONNX and finds the top matching tools via cosine similarity.

Speed: ~1–5ms per query Dependencies: Local embeddings only (~90MB, auto-downloaded)

Static API (Recommended):

using ElBruno.ModelContextProtocol.MCPToolRouter;
using ModelContextProtocol.Protocol;

var tools = new[]
{
    new Tool { Name = "get_weather", Description = "Get current weather for a location" },
    new Tool { Name = "send_email", Description = "Send an email message to a recipient" },
    new Tool { Name = "search_files", Description = "Search for files by name or content" },
    new Tool { Name = "calculate", Description = "Perform mathematical calculations" },
    new Tool { Name = "translate_text", Description = "Translate text between languages" }
};

// One-liner: route and get results immediately
var results = await ToolRouter.SearchAsync("What's the temperature outside?", tools, topK: 3);

foreach (var r in results)
    Console.WriteLine($"  {r.Tool.Name}: {r.Score:F3}");
// Output:
//   get_weather: 0.847
//   calculate: 0.312
//   translate_text: 0.201

Advanced: Reusable Instance (for performance-critical scenarios)

For servers, agents, or batch operations, build and reuse an index to avoid re-embedding:

// Build once, reuse many times (e.g., in a web API or agent loop)
await using var index = await ToolIndex.CreateAsync(tools);

var results = await index.SearchAsync("What's the temperature outside?", topK: 3);

---

Mode 2: Hybrid Search (LLM + Multi-Query) — One-Liner

Use when: Your prompt is verbose, multi-part, or conversational.

What it does: Uses a small local LLM (e.g., Qwen 2.5 0.5B) to distill the prompt into comma-separated action phrases, then runs a hybrid search: the original prompt is searched as a baseline, each extracted phrase is searched individually, and results are merged. Baseline tools keep full scores; phrase-only tools get an 85% discount. This means Mode 2 can only improve over Mode 1, never degrade.

Speed: ~50–200ms (LLM inference + embedding) — faster on GPU Dependencies: Local embeddings (~90MB) + local LLM (~1GB, auto-downloaded)

Static API — Zero Setup (Recommended):

using ElBruno.ModelContextProtocol.MCPToolRouter;
using ModelContextProtocol.Protocol;

var tools = new Tool[] 
{
    new Tool { Name = "get_weather", Description = "Get current weather for a location" },
    new Tool { Name = "send_email", Description = "Send an email message to a recipient" },
    new Tool { Name = "search_files", Description = "Search for files by name or content" },
    new Tool { Name = "calculate", Description = "Perform mathematical calculations" },
    new Tool { Name = "translate_text", Description = "Translate text between languages" }
};

// One-liner: the local LLM is downloaded and managed automatically
var results = await ToolRouter.SearchUsingLLMAsync(
    "Hey, I was thinking about my trip next week and I need to know " +
    "if it's going to rain in Tokyo. Also remind me to call the dentist.",
    tools, topK: 5);

// The LLM distills this to action phrases: "check weather Tokyo, set reminder call dentist"
// Then each phrase + the original prompt are searched and results merged
foreach (var r in results)
    Console.WriteLine($"  {r.Tool.Name}: {r.Score:F3}");
// Output:
//   get_weather: 0.912
//   calculate: 0.287
//   translate_text: 0.156

Advanced: Bring Your Own IChatClient

Use any IChatClient (Azure OpenAI, Ollama, etc.) instead of the built-in local LLM:

// Pass your own IChatClient for prompt distillation
var results = await ToolRouter.SearchUsingLLMAsync(
    "Complex prompt here...",
    tools, chatClient, topK: 5);

Advanced: Reusable Instance (for performance-critical scenarios)

For servers or agents running multiple queries, build and reuse a router instance:

await using var router = await ToolRouter.CreateAsync(tools, chatClient);
var results = await router.RouteAsync("Complex prompt here...", topK: 5);

Model Metadata (v0.7.1+)

When using ElBruno.LocalLLMs v0.7.1+, the library exposes model metadata with accurate runtime limits:

using var client = await LocalChatClient.CreateAsync();
var info = client.ModelInfo;
Console.WriteLine($"Model: {info?.ModelName}");
Console.WriteLine($"  Effective Max Sequence Length: {info?.MaxSequenceLength} tokens");
Console.WriteLine($"  Config Max Sequence Length: {info?.ConfigMaxSequenceLength} tokens");

In v0.7.1+, MaxSequenceLength returns the effective runtime limit (e.g., 128 tokens for Phi-3.5 mini ONNX), while ConfigMaxSequenceLength preserves the raw config value (e.g., 131072). This ensures metadata accuracy for proper input validation and context window management. Mode 2 (SearchUsingLLMAsync) uses these values automatically to validate prompts fit within the model's capacity.

GPU Acceleration (Optional)

The samples default to the CPU-only ONNX runtime (Microsoft.ML.OnnxRuntimeGenAI), which works everywhere. GPU acceleration is optional — if your machine supports it, swap the runtime package for a potential 2–5x speedup on Mode 2 inference:

| Hardware | Package | Command | |----------|---------|---------| | CPU only (default) | Microsoft.ML.OnnxRuntimeGenAI | dotnet add package Microsoft.ML.OnnxRuntimeGenAI | | Windows GPU (DirectML) | Microsoft.ML.OnnxRuntimeGenAI.DirectML | dotnet add package Microsoft.ML.OnnxRuntimeGenAI.DirectML | | NVIDIA GPU (CUDA) | Microsoft.ML.OnnxRuntimeGenAI.Cuda | dotnet add package Microsoft.ML.OnnxRuntimeGenAI.Cuda |

Do not mix CPU and GPU variants — add exactly one. Note: DirectML and CUDA require compatible hardware and drivers; if your machine doesn't support them you'll get a "Specified provider is not supported" error. When in doubt, use the CPU package.

---

When to Use Which Mode

| | Mode 1: Embeddings | Mode 2: Hybrid Search | |---|---|---| | Best for | Clear, single-intent prompts | Verbose, multi-part, conversational prompts | | Speed | ~1–5ms per query | ~50–200ms on CPU (~20–100ms with GPU) | | Dependencies | Local embeddings only (~90MB) | Local embeddings + local LLM (~1GB) + optional GPU runtime | | Static API | ToolRouter.SearchAsync() | ToolRouter.SearchUsingLLMAsync() | | Instance API | ToolIndex.CreateAsync() + SearchAsync() | ToolRouter.CreateAsync() + RouteAsync() | | Example prompt | "Send an email" | "I need to email Alice about the deadline and check the weather" | | Accuracy | High for clear intents | Higher for ambiguous/complex intents (never worse than Mode 1) |

---

Using Filtered Tools with Azure OpenAI

Both modes work seamlessly with Azure OpenAI — route first, then send only the filtered tools to reduce token costs.

using Azure;
using Azure.AI.OpenAI;
using OpenAI.Chat;

// Create Azure OpenAI ChatClient
var chatClient = new AzureOpenAIClient(
    new Uri("https://your-resource.openai.azure.com/"),
    new AzureKeyCredential("your-api-key"))
    .GetChatClient("gpt-5-mini");

// Route to relevant tools only (using Mode 1 as example)
var relevant = await ToolRouter.SearchAsync(userPrompt, allTools, topK: 3);

// Add only filtered tools to the chat call — saving tokens!
var chatOptions = new ChatCompletionOptions();
foreach (var r in relevant)
    chatOptions.Tools.Add(ChatTool.CreateFunctionTool(r.Tool.Name, r.Tool.Description ?? ""));

var response = await chatClient.CompleteChatAsync([new UserChatMessage(userPrompt)], chatOptions);

How It Works — Technical Details

Core mechanism: MCPToolRouter embeds tool descriptions and user prompts into a vector space, then finds the top-K tools via cosine similarity — all using local embeddings (ONNX, no external APIs).

Hybrid pipeline for Mode 2: When using LLM distillation, the local LLM extracts comma-separated action phrases from verbose prompts. The original prompt is searched as a baseline (Mode 1), then each extracted phrase is searched individually. Results are merged: baseline tools keep full scores, phrase-only tools get an 85% discount. Post-processing automatically cleans up trailing word repetitions and duplicate phrases (common with small local LLMs). This hybrid approach guarantees Mode 2 can only improve over Mode 1.

Token savings: By routing prompts to only the relevant tools before sending to external LLMs, you can reduce token usage by 70–85% while maintaining accuracy.

Advanced Features

ToolIndexOptions

Configure the index behavior with ToolIndexOptions:

var options = new ToolIndexOptions
{
    QueryCacheSize = 20,                          // LRU cache for repeated queries (0 = disabled)
    EmbeddingTextTemplate = "{Name}: {Description}" // Customize how tools are embedded
};

await using var index = await ToolIndex.CreateAsync(tools, options);

Save / Load Index

Persist a pre-built index to avoid re-embedding on startup:

// Save
using var file = File.Create("tools.bin");
await index.SaveAsync(file);

// Load (instant warm-start — no re-embedding)
using var stream = File.OpenRead("tools.bin");
await using var loaded = await ToolIndex.LoadAsync(stream);

Dynamic Index (Add / Remove Tools)

Modify the index at runtime without rebuilding:

await index.AddToolsAsync(new[] { new Tool { Name = "new_tool", Description = "..." } });
index.RemoveTools(new[] { "obsolete_tool" });

Dependency Injection

Register IToolIndex as a singleton in ASP.NET Core or any DI container:

builder.Services.AddMcpToolRouter(tools, opts =>
{
    opts.QueryCacheSize = 20;
});

// Inject IToolIndex anywhere
app.MapGet("/search", async (IToolIndex index, string query)
    => await index.SearchAsync(query, topK: 3));

Custom Embedding Generator

Bring your own IEmbeddingGenerator> from the Microsoft.Extensions.AI ecosystem:

IEmbeddingGenerator> myGenerator = /* your provider */;
await using var index = await ToolIndex.CreateAsync(tools, myGenerator, options);

---

⚡ Performance Guide

MCPToolRouter is designed for both one-off queries and high-throughput scenarios. Understand the trade-offs to choose the right API for your use case.

Static vs Instance API

The static API (SearchAsync, SearchUsingLLMAsync) is ideal for simple scripts and one-off calls — just pass your prompt and tools, get results immediately. Under the hood, static methods now reuse shared singleton ONNX sessions by default (UseSharedResources = true), so repeated static calls are much faster than before.

The instance API (ToolIndex.CreateAsync + SearchAsync, or ToolRouter.CreateAsync + RouteAsync) is optimized for servers, agents, and batch operations where you make many queries. By reusing the same index, you avoid re-embedding tools and re-initializing the LLM — this provides 15–35x speedup on subsequent queries compared to creating a new index each time.

Pro tip: For cleanup, call await ToolRouter.ResetSharedResourcesAsync() to release singleton resources when your application shuts down.

Query Cache

Set QueryCacheSize > 0 in ToolIndexOptions to cache query embeddings. Identical prompts skip embedding generation entirely (~0ms vs ~10–20ms per query):

var options = new ToolIndexOptions { QueryCacheSize = 20 };
await using var index = await ToolIndex.CreateAsync(tools, options);

var result1 = await index.SearchAsync("weather in Tokyo");     // ~10ms: embedding generated
var result2 = await index.SearchAsync("weather in Tokyo");     // ~0ms: cache hit

The cache is automatically cleared when tools are added or removed.

Quick Recommendation

| Scenario | Recommended API | Why | |----------|----------------|-----| | CLI tool, one-off query | ToolRouter.SearchAsync() | Simplest, shared resources handle perf | | Server/agent, many queries | ToolRouter.CreateAsync() + RouteAsync() | Full control, best performance | | Verbose or complex prompts | ToolRouter.SearchUsingLLMAsync() | Hybrid search: LLM distills + multi-query merge |

---

🔒 Security Considerations

MCPToolRouter runs entirely locally — models and searches never leave your machine. However, a few security practices are worth considering:

Prompt Injection

Mode 2 uses a local LLM for prompt distillation. While the LLM cannot execute tools directly, adversarial prompts could theoretically influence which tools are selected. Validate tool selection downstream in your application before executing any tool calls. Example: if a tool requires authentication or permission c

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: elbruno
• Source: elbruno/ElBruno.ModelContextProtocol
• License: MIT

Install and usage instructions live in the source repository linked above.