# Langchainrust

> A Rust framework for building LLM applications with support for 8+ LLM providers, agents, RAG, BM25/hybrid search, LangGraph workflows, and multiple vector/document storage backends.

- **Type:** MCP server
- **Install:** `agentstack add mcp-atliliw-langchainrust`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [atliliw](https://agentstack.voostack.com/s/atliliw)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [atliliw](https://github.com/atliliw)
- **Source:** https://github.com/atliliw/langchainrust

## Install

```sh
agentstack add mcp-atliliw-langchainrust
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# langchainrust

[](https://www.rust-lang.org/)
[](LICENSE)
[](https://crates.io/crates/langchainrust)
[](https://docs.rs/langchainrust)
[](https://github.com/atliliw/langchainrust/actions/workflows/ci.yml)
[](https://crates.io/crates/langchainrust)

A LangChain-inspired Rust framework for building LLM applications.

**What it solves**: Build Agents, RAG, BM25 keyword search, Hybrid retrieval, LangGraph workflows, MCP tools, A2A agent-to-agent protocols, Guardrails, multi-agent Handoffs — all in pure Rust.

---

## Design Principles

The framework is engineered around a few hard rules that come out of its own design reviews. These are what make it feel different from hand-rolled LLM glue code:

| Principle | What it means in practice |
|-----------|---------------------------|
| **Explicit over silent** | No silent degradation. If an API promises X, it either delivers X or fails loudly. Batch-embedding alignment returns explicit errors instead of empty/shifted vectors; the keyword fallback for `EmbeddingMatcher` was removed; routing failures surface as errors, not swallowed `Err(_) => {}`. |
| **Consumed or deleted abstractions** | Every trait either has real implementors or is removed. `BaseChatMemory` is now implemented by all four memories, all retrievers implement `RetrieverTrait`, and `PairwiseJudge` plugs into the unified `Evaluator` pipeline — so users write against one abstraction, not five ad-hoc names. |
| **Structured output over text parsing** | Models that support `tool_calls` go through structured output (JSON schema / function calling). Regex-parsing model output is the last resort, not the default — it is the most common source of silent fragility. |
| **Production hardening first** | Tool execution has timeouts, LLM calls retry with exponential backoff, agent loops are capped (`max_iterations` clamped to `[1, 100]`), parallel actions are concurrency-limited, and `CancellationToken` propagates through every runnable. Eliminate the deterministic failure paths before adding features. |
| **Type system enforces safety** | Security properties live in types, not comments. Guardrails split `InputGuardrailResult` / `OutputGuardrailResult` so "Modify only applies to output" is enforced at compile time. |
| **Composition first** | Everything can be `pipe`d. Since v0.16.0, prompts, memory, native providers, parsers and RAG are all `Runnable` — `prompt.pipe(llm).pipe(parser)` compiles and runs. |
| **Honest implementations** | No fake backends. Empty-shell sandboxes (Wasm/E2B) were deleted; unsupported operations (e.g. Pinecone fetch-by-ID) return explicit `StorageError`s instead of pretending to work. |

---

## Core Features

### LLM & Providers

| Component | Description |
|-----------|-------------|
| **Unified LLM access** | 11 providers behind one `BaseChatModel` trait: OpenAI, Ollama, Anthropic Claude, Gemini, Azure, Cohere, DeepSeek, Qwen, Moonshot, Zhipu, Mistral. `LLMClient::from_env()` auto-detects any of the 11 from environment variables. |
| **OpenAI-compatible endpoint (v0.22.4)** | One generic `OpenAICompatibleChat` covers any `base_url` endpoint — keyless vLLM / LM Studio / SGLang / Ollama / internal gateways (no `Authorization` header is sent without a key), with `GroqChat` / `OpenRouterChat` / `XaiChat` presets, env constructors and `extra_headers`. |
| **OpenAI-compatible thin wrappers** | DeepSeek / Qwen / Moonshot / Zhipu / Mistral reuse the OpenAI request path; each keeps its own error variant (`ProviderError::DeepSeek`, etc.) so you can tell which vendor failed. New vendors are cheap to add. |
| **Chat & Streaming** | `chat()` (one full reply) and `stream_chat()` (first token in ~1s). Streaming chunks carry token usage (`StreamChunk`), so budget gates get real usage on the streaming path (v0.18). `config.streaming = true` makes `chat()` stream internally then aggregate. |
| **Function Calling** | `bind_tools()` + `result.tool_calls`, the native path for tool-capable models. |
| **Multimodal Vision** | `Message::human_with_image` / `human_with_audio` / `human_with_file` via schema `ImageContent` / `AudioContent` / `FileContent`. |
| **Thinking models** | Reasoning is kept in `LLMResult.thinking_content` and never leaked into `content` (DeepSeek-R1, GLM-5.2, Claude Extended Thinking). |
| **OpenAI Assistants API** | Stateful assistants with `requires_action` tool dispatch. |
| **OpenAI Responses API** | Typed Responses-endpoint client (`openai/responses/`, ~1.4k lines): `web_search` / `file_search` / code-interpreter tooling. Chat Completions remains the portable default. |
| **Anthropic Extended Thinking** | `with_thinking` for Claude reasoning. |
| **Model Routing** | `RouterLLM` with 6 strategies — Fallback / RoundRobin / LeastLatency / `LatencyWeighted(beta)` (EMA-latency weighted draw, deterministic SplitMix64) / LowestCost (registry-priced, blended 0.75-in/0.25-out) / InputDirected — plus per-slot `ModelRateLimit` admission and a shared `RouterBudget` USD circuit breaker (v0.22.4). Remaining models always act as fallback. |
| **Cost & budgets (v0.22.4)** | `ModelPrice` / `PricingTable` / shared `CostTracker` (run/session scopes, unknown models bill $0 and never break the loop), JSON-fetchable `ModelRegistry` of capabilities/prices, agent-level `BudgetConfig.max_cost_usd` hard gate on both invoke and stream paths, costs exported as `ObsEvent::Cost`. |
| **Batch API** | `BatchClient` for OpenAI / Anthropic batch inference (~50% cost reduction). |
| **LLM Cache** | `LLMCache` with TTL + true LRU eviction (hits refresh recency). |
| **Structured Output** | `with_structured_output` + `StructuredOutputExt` trait, `JsonOutputParser` fallback, and streaming structured output via `PartialJsonParser`. |
| **Native JSON Schema (v0.21)** | `OpenAIChat::with_json_schema_output::()` sends `response_format: {type: "json_schema"}` (strict mode) generated from schemars 1.0 — schema-constrained decoding on the provider side; `make_strict_schema` enforces `additionalProperties: false` + full `required`. `with_structured_output` remains the portable path. |
| **Token Counter** | `TiktokenCounter` (precise) / `CharRatioCounter` (Chinese-friendly estimate) + `TokenTrackingLLM` usage stats + `ModelPricing` cost estimation. |

### Embeddings

| Component | Description |
|-----------|-------------|
| **Unified `Embeddings` trait** | `embed_query` / `embed_documents` / `dimension` / `model_name`, with empty-input and batch-alignment checks enforced in the trait default path. |
| **Providers** | OpenAI (ada-002 / 3-small / 3-large), DeepSeek, Qwen, Cohere (embed-v3.0, 4 input types), FastEmbed (local ONNX), Mock (deterministic, for tests), BagOfWords (local, always available). |
| **Qwen3-Embedding (v0.21)** | `qwen3-embedding-0.6b` / `4b` / `8b` (dims 1024 / 2560 / 4096) + matryoshka output via `QwenEmbeddingsConfig::with_dimensions(32..=4096)`; the configured `dimensions` is passed through to the DashScope request. |
| **Reliability** | Exponential-backoff retries (429/5xx, max 3, 4xx not retried) with jitter and `Retry-After` header support (v0.21), concurrent batching (OpenAI: 2048 docs/batch, concurrency 8), and unified normalization so downstream similarity is provider-independent. |
| **Local ONNX** | `LocalEmbeddings` via the `local-embeddings` feature (ort). |
| **Token-level embeddings (v0.21)** | Optional `TokenLevelEmbeddings` capability trait (native `async fn`, statically dispatched): `embed_tokens` returns per-token `TokenEmbedding { span, vector }` with byte-offset spans. Local ONNX path shares the fastembed pipeline. |
| **Late chunking (v0.21, end-to-end in v0.23)** | `late_chunk(&embedder, text, &LateChunkConfig)` embeds the whole text once at token level, then mean-pools token vectors per chunk range into L2-normalized chunk vectors — better context retention than chunk-then-embed for long documents. `late_index_in(&vector_store, &embedder, parent_key, text, &config)` (v0.23) closes the loop: one token-level pass → pooled chunks → straight into any `VectorStore`, with deterministic `{parent}:{index}` ids. Runnable offline demo: `cargo run -p lc-rag --example late_chunking`. |
| **Candle backend (v0.21)** | `CandleEmbeddings` via the `local-candle` feature: pure-Rust CPU inference for BERT-family models (`from_hf_hub("BAAI/bge-small-en-v1.5")` or `from_dir`), masked mean-pooling, batch size 16. |
| **Vision embeddings (v0.22.4)** | `VisionEmbeddings` trait maps `(text/image)` into one shared vector space: Cohere Embed v4 (`CohereVisionEmbeddings`, 1536-d) and DashScope `multimodal-embedding-v1` (`QwenVisionEmbeddings`, 1024-d), plus `MockVisionEmbeddings` for tests. Feeds multimodal RAG (see below). |

### Composition: Chains & LCEL

| Component | Description |
|-----------|-------------|
| **LCEL** | `Runnable` with four base actions — `invoke` / `batch` / `stream` / `transform`. Operators: `pipe`, `RunnableLambda`, `RunnablePassthrough`, `RunnableParallel`, `RunnableBranch`, `RunnableBinding`, `RunnableWithFallbacks`, `RunnableAssign`, `with_retry`, `RunnableSequence`. Type-erased with `PhantomData` — dynamic composition with compiler-checked type matches. |
| **Unified composition (v0.15)** | Prompts, memory, native providers, parsers and RAG are all `Runnable`: `prompt.pipe(llm).pipe(StrOutputParser)` — no glue code. `RunnableWithMessageHistory` wraps "LLM + memory" as one runnable (auto read history → invoke → write back). `RagRunnable` makes retrieval-augmented generation one link of a chain. Native `OpenAIChat`/`QwenChat`/`DeepSeekChat` errors are unified into `LcelError`. |
| **Chains** | `BaseChain` with 9 implementations: `LLMChain`, `ConversationChain`, `SequentialChain`, `RouterChain`, `LLMRouterChain`, `RetrievalQA`, `ConversationRetrievalChain`, plus the 4 document chains — `Stuff` / `MapReduce` / `Refine` / `MapRerank`. Chain streaming per token, `ChainRunnable` bridges chains into LCEL. |
| **Prompts** | `PromptTemplate` (parsed once, cached segments), `ChatPromptTemplate` (Runnable, outputs `Vec`), `FewShotPromptTemplate` + `ExampleSelector`s (`LengthBasedExampleSelector`). `{{`/`}}` escapes, Chinese variable names, missing variables error loudly. v0.22.4 adds `PromptRegistry` — versioned, named prompt storage with render-by-name and fallback, so prompt text is managed centrally instead of scattered through call sites. |
| **Output Parsers** | `StrOutputParser`, `JsonOutputParser`, `CommaSeparatedListOutputParser`, `StructuredOutputParser`, `TypedOutputParser` — all tolerant of dirty model output (markdown fences, trailing commas, trailing junk). |
| **Retrieval & Sessions in LCEL (v0.17)** | `RetrieverRunnable` wraps any retriever as `Runnable>`; `SessionManagerRunnable` wraps persistent sessions as `Runnable` — both compose with `pipe` into a chain. |
| **Cancellation** | `CancellationToken` threads through `RunnableConfig` into every execution. |

### Agents & Multi-Agent

| Component | Description |
|-----------|-------------|
| **BaseAgent / AgentExecutor** | The "translator / butler" split: `BaseAgent` turns model output into a decision (Action / Actions / Finish), `AgentExecutor` is the one real loop — with tool timeouts, LLM retries, concurrency semaphore, and `max_iterations` clamped to `[1, 100]`. |
| **FunctionCallingAgent** | Recommended path — reads native `tool_calls` (requires model support). |
| **ReActAgent** | Text-regex thought/action loop, fallback for models without tool-calling. |
| **Plan-Execute** | Planner → per-step executor → replan on failure; the executor factory is configurable (no longer hardcoded to function calling). |
| **DeepResearch** | Multi-round research agent with sub-topic decomposition, parallel search, dedup, citation reporting. |
| **RAG Agents** | `CorrectiveRAGAgent` (self-correcting grade/rewrite/detect), `AdaptiveRAG` (LLM-routed retrieval), as standalone graphs. |
| **Handoffs** | Multi-agent handoff with `max_handoff_depth` (default 10) to stop A↔B ping-pong. |
| **Orchestrators** | `Supervisor` (v0.24.0 — one router LLM delegates each round to a named sub-agent or `FINISH`, with scratchpad feedback and a bounded-rounds one-level recursion guard), `FanOutFanIn` (parallel fan-out + aggregate), `SequentialPipeline` (serial), `OrchestratorRunnable` for LCEL integration. |
| **Parallel tool calls (v0.24.0)** | Multiple tool calls emitted in one model turn run concurrently, bounded by `.with_max_concurrency(n)` (default 8); observations zip back to actions in the model's call order on both invoke and stream paths. |
| **Agent Hooks** | Approval (`on_before_tool_call` allow/reject/skip), `PromptInjectionHook`, `TokenBudgetHook`, `ContentFilterHook`, logging. |
| **Agent Gates (v0.16)** | Async human-approval gate — `.with_approval()` (Allow / Deny / Modify; Deny feeds the reason back as an observation, Modify rewrites the arguments). Budget gate — `.with_budget()` with hard caps on tool calls / tokens / wall-clock duration / iterations, exceeding returns `AgentError::BudgetExceeded`. Both default off. |
| **Cross-process resume (v0.18)** | `FileResumeStore` persists the pending human-approval / budget-gate state to disk (atomic write); a restarted executor loads the pending point and re-enters approval instead of restarting the agent loop. |
| **Context compaction (v0.21)** | `.with_compaction(CompactionConfig)` — trigger on `TurnCount` / `TokenCount` / `Any` / `All`, compact with `SlidingWindow` or `TokenBudget` (turn-boundary truncation, no orphan tool results, `min_recent_turns` floor, default 2). Off by default; compaction count lands in `AgentMetrics.compactions`. |
| **Streaming** | Token-level streaming via `StreamingFunctionCallingAgent` + `AgentStreamEvent`; tool-level events via `AgentExecutor::stream`. |
| **Web SSE (v0.22.4)** | Optional axum SSE endpoint in `lc-agents` (`sse-server` feature) serving `AgentStreamEvent`s to browsers — see the `agent_sse_server` example. |
| **Durable checkpoints (v0.22.4)** | LangGraph persistence gains three production backends behind `checkpoint-sqlite` (rusqlite bundled, WAL) / `checkpoint-postgres` (tokio-postgres) / `checkpoint-redis` (Lua CAS) features. Optimistic concurrency: stale writes return `GraphError::CheckpointVersionConflict` instead of last-write-wins. |
| **Two-layer semantic memory (v0.22.4)** | Episodic layer (raw per-turn observations, vector-retrieved) + semantic layer (LLM-consolidated, deduped facts) with an async background extractor — the agent accumulates durable knowledge across sessions instead of only replaying recent chat. |
| **Dynamic interrupt/resume + time travel (v0.24.0)** | `InterruptibleNode` suspends *inside* a node — its closure runs with `resume: None`, raises an `InterruptRequest` payload into the checkpoint, then re-enters with `resume: Some(decision)` via `CompiledGraph::resume_with_value` (even from a fresh process over a durable checkpointer; `NodeInterrupt`/`Resumed` stream events). Agent approvals converge on this one path: `ApprovalGate` + `ApprovalDecision` (Allow / Deny / Modify), so the gated tool executes exactly once and Deny performs no side effect; the non-graph `ResumeStore` remains for the plain executor. `get_state_history()` lists `CheckpointInfo`s and `fork_from(checkpoint, node, override_state)` branches a new forward-only lineage off any past state. |
| **Tool Policies** | `ToolPolicy` / `ToolRisk` risk classification for tool access control. |

### Retrieval & RAG

| Component | Description |
|-----------|-------------|
| **Unified `RetrieverTrait`** | All retrievers implement it: `SimilarityRetriever`, `BM25Retriever` / `ChunkedBM25Retriever`, `UnifiedHybridIndex`, `ParentDocumentRetriever` — so any retrieval strategy plugs into the RAG pipeline. |
| **RAGPipeline** | `RAGPipelineBuilder` (llm + embeddings + vector store + retriever) → `index_documents` / `query` / `query_with_sources` (citation tracing). |
| **Document Loaders** | Text / JSON / Markdown / PDF / CSV / HTML + WebScraper / Sitemap / D

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [atliliw](https://github.com/atliliw)
- **Source:** [atliliw/langchainrust](https://github.com/atliliw/langchainrust)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** yes
- **Shell / process execution:** yes
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-atliliw-langchainrust
- Seller: https://agentstack.voostack.com/s/atliliw
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.