KnowledgeForge

KnowledgeForge

Enterprise Agentic RAG Knowledge Agent

Multi-tenant · RBAC-secured · Graph-enhanced · Production-grade

---

About

KnowledgeForge is a production-grade, enterprise-ready AI Knowledge Agent that enables employees to query internal documents using natural language and receive accurate, cited answers — all governed by role-based access control.

What makes it special:

• 🏢 Multi-tenant RBAC — Department-level document isolation, admin/member/viewer roles
• 🤖 Agentic RAG — LangGraph pipeline with tool calling, intent routing, CRAG grading
• 🕸️ Graph-based RAG — LightRAG knowledge graph for multi-hop reasoning
• 📊 RAGAS-evaluated — Quantifiable quality metrics (Faithfulness 0.75, Precision 1.00)
• 🔌 Universal LLM Gateway — Zero-code switching to any provider via LiteLLM (Ollama, vLLM, Gemini, OpenRouter, Anthropic, Claude, DeepSeek, etc.)
• 🧠 Persistent Memory — mem0 cloud: auto fact extraction, semantic search, cross-session personalization
• 📡 Full Observability — LangSmith + Langfuse + Prometheus tracing & metrics
• 🐳 Docker-ready — 12-service production stack with health checks

---

System Architecture

KnowledgeForge is designed as an enterprise-grade AI knowledge platform built on a robust, scalable, and secure architecture. The system data flow and request processing pipeline is shown in the architecture diagram above.

1. The 5-Layer Context Engineering Engine

Managing long-context windows efficiently while preserving focus is handled by the context builder inside [src/memory/working.py](src/memory/working.py). It structures context into 5 distinct, budget-controlled layers to mitigate "Lost in the Middle" attention degradation:

• Layer 1: System Instructions & Safety Boundaries (L1): Static, cached system instructions, Markdown formatting schemas, and tool-calling constraints. Placed at the very top.
• Layer 2: Personalization & User Facts Profile (L2): Dynamically recalled user facts and preferences retrieved from mem0 memory. Prefixed with [User profile] delimiters.
• Layer 3: Document/RAG Retrieval Context (L3): Relevance-ranked passages from the hybrid retrieval pipeline. Chunks are reordered via _lost_in_middle_reorder (most relevant chunks placed at the start and end of the block, less relevant chunks in the middle) to maximize attention.
• Layer 4: Current User Query & Instruction (L4): Placed at the absolute bottom of the context, closest to the generation target, which ensures the model prioritizes the active task instructions.
• Layer 5: Conversation & Session History (L5): Sliding-window memory. Older history turns are automatically compressed into key bullets via LLM summarization (summarize_history), while the most recent turn-pairs are preserved verbatim to maintain local dialog coherence.

2. Multi-Tier Agent Memory System

KnowledgeForge features a tiered, multi-tier agent memory registry to emulate human cognitive recall:

| Memory Tier | Scope | Backend | Lifecycle | Key Capabilities | |-------------|-------|---------|-----------|------------------| | Working Memory | Session-level | In-Memory / Redis | Active thread session | Token counting, sliding window pruning, token usage warnings, history summarization. | | Semantic Memory | Cross-session | mem0 Cloud (Vector Index) | Persistent (survives restarts) | Fact extraction, user profile accumulation, contradiction resolution (e.g. updating old facts), relevance score thresholding ($\ge 0.15$). | | Episodic Memory| Historical overview | MongoDB / File Store | Background async consolidation | Parses conversation chains asynchronously to extract high-level episodes and store session summaries. |

---

Features

Advanced RAG Pipeline

KnowledgeForge contains a highly optimized, enterprise-grade RAG pipeline that implements state-of-the-art retrieval and generation techniques:

• Lexical & Semantic Ensemble (BM25 + Vector Search): Combines dense vector semantic search (Qdrant/Chroma) and lexical keyword matching (Rank-BM25), fusing candidate rankings with Reciprocal Rank Fusion (RRF, constant $k=60$) to balance exact keyword hits with conceptual understanding.
• Cross-Encoder Reranking (ms-marco-MiniLM-L-6-v2): Acts as a secondary relevance filter by scoring document chunks directly against the user query. This process discards low-quality retrieved passages, significantly compressing the prompt context size and reducing downstream token costs.
• Corrective RAG (CRAG): Employs a dedicated LLM relevance grader to review the retrieved document chunks. If chunk confidence falls below the acceptance threshold, the grader dynamically triggers external web search tool fallbacks to resolve knowledge gaps and eliminate hallucinations.
• Graph RAG Integration (LightRAG): Combines standard vector RAG with LightRAG. It indexes and queries entities and relationship graphs scoped by organization and department boundaries, enabling multi-hop reasoning over complex themes and structural connections that traditional vector chunking fails to locate.
• Hypothetical Document Embeddings (HyDE): Pre-generates a hypothetical answer paragraph matching the query, then uses its embedding to perform semantic searches, bridging the semantic gap between questions and raw document statements.
• Scope-Isolated Semantic Caching: Caches previous responses based on query cosine similarity $\ge 0.92$. To prevent data leakage, the cache is isolated by user role and department boundaries.
• Context-Aware Query Rewriting: Uses conversation history and the latest user prompt to rewrite query inputs into clear standalone questions, resolving coreferences, pronouns, and implicit context.
• Multi-Tenant & RBAC Filtered Retrieval: Ingests and queries documents with organization, department, and role metadata filters applied at the database level, guaranteeing absolute data isolation between departments.
• SSE streaming: Real-time token delivery via POST /api/v1/chat/stream.

Multi-Agent (LangGraph)

Set RAG_MODE=agent for the full agentic pipeline:

• Intent Router — semantic-router embedding classification (4 intents: direct/rag/tool/agent)
• Tool Caller — LLM function calling with bound tool schemas
• Tool Executor — LangGraph ToolNode (getdatetime, calculate, websearch)
• Retriever — RBAC-filtered vector search
• Reranker — Cross-encoder scoring
• Grader — CRAG chunk relevance filtering
• Synthesizer — Multi-provider LLM generation
• QA Checker — Hallucination guard + confidence scoring

Auth & Multi-Tenancy

• JWT authentication with refresh tokens
• Organization → Department → User hierarchy
• Role-based access: admin | member | viewer
• API versioning: /api/v1/* with backward compat
• Rate limiting (slowapi, configurable)

Observability

| Provider | Purpose | Toggle | |----------|---------|--------| | LangSmith | Full pipeline trace spans with I/O | OBSERVABILITY_PROVIDER=langsmith | | Langfuse | Self-hosted tracing + analytics | OBSERVABILITY_PROVIDER=langfuse | | Prometheus | 11 metric types (latency, tokens, cost, cache) | Always active at /metrics |

Document Understanding

• 10 formats: PDF, DOCX, PPTX, XLSX, TXT, MD, HTML, JPG, PNG, WEBP
• Docling (IBM) — Layout-aware parsing: tables, formulas, multi-column
• Vision LLM — Images without text → Gemini/OpenRouter describes content
• Async processing — Non-blocking with job tracking
• SHA-256 dedup — Skip re-ingestion of identical files

---

Advanced Capabilities & Architecture Deep-Dive

KnowledgeForge is built from the ground up for production stability, cost control, strict tenant security, and high retrieval accuracy.

1. Resiliency & Failure Recovery

• Async Circuit Breakers: Protects external LLM APIs from cascading failures using a custom async circuit breaker (AsyncCircuitBreaker in [src/core/errors.py](src/core/errors.py#L30)). If failures cross a threshold (e.g. 5 failed calls), the breaker trips to OPEN, immediately rejecting outgoing requests for a recovery timeout (30 seconds) to prevent server thread starvation.
• Exponential Backoff with Full Jitter: Transient network errors or rate limits (e.g., HTTP 429 Too Many Requests) are handled with exponential backoff capped at a maximum delay. It injects a randomized delay variable (Full Jitter algorithm) to eliminate the "thundering herd" concurrency problem on downstream servers.

2. Quota Management & Cost Control

• Complexity-Based Query Routing: Analyzes query structure, sentence clauses, and question triggers via the ModelComplexityRouter in [src/governance/cost.py](src/governance/cost.py). Simple chitchat or direct instructions are automatically routed to low-cost local models (e.g., Ollama / local Qwen), while complex multi-step analytical queries are sent to premium cloud models.
• USD & Token Budget Manager: The TokenBudgetTracker monitors daily usage. It tracks accumulated token consumption and dollar spend on a per-user basis. If a request would breach the daily threshold, it raises a TokenBudgetExceededError before calling downstream APIs, protecting host budgets.
• Custom Model Cost Handling: Wrapped all budgeting callbacks in defensive try-except blocks. If custom unmapped model gateways (like 9Router Cloudflare free pools) return completions without defined token costs, it logs a warning instead of throwing fatal errors.

3. Agentic Orchestration & Tool Harness

• LangGraph State Machine: Models the agent pipeline as a directed graph. Features intent classification, planning nodes, tool executor loops, synthesis nodes, and QA reflection nodes. Includes a loop-guard counter to prevent infinite reflection loops.
• Automatic Function-to-Tool Compiler: The ToolRegistry compiles standard Python functions into JSON tool schemas for LLM function calling by extracting docstrings, parameter type hints, and validation constraints at runtime.
• Model Context Protocol (MCP): Fully supports MCP. Interacts with external filesystem, database, or API tools over stdio and SSE transport protocols.

4. Enterprise Security, Privacy & Isolation

• Presidio PII Masking: Integrates Microsoft Presidio at the API gateway. Automatically detects and redacts emails, phone numbers, credit card numbers, SSNs, and IP addresses before sending context to public LLM APIs.
• Input & Output Guardrails: Blocks malicious prompt injection attempts (e.g., "ignore previous instructions") at the API boundary, returning HTTP 403 Forbidden. Outputs are scanned against Vietnamese/English harmful content lists.
• KMS Secret Wrapping: Protects sensitive adapter credentials (API keys, database connections) using secure KMS envelope decryption wrappers.
• RBAC Department Isolation: Department metadata filters are injected into all vector search and document ingestion queries, guaranteeing that users can never retrieve documents outside their authorized department boundary.

---

Project Structure

knowledgeforge/
├── src/                              # Application source code
│   ├── api/                          #   Delivery & API Presentation layer
│   │   ├── main.py                   #     FastAPI application factory + lifespan
│   │   ├── middleware.py             #     Rate limiters, CORS, security headers
│   │   └── deps.py                   #     Composition Root (Dependency Injection wiring)
│   │   └── routes/                   #     FastAPI endpoint routers
│   │       ├── chat.py               #       Synchronous & SSE stream chat endpoints
│   │       ├── ingest.py             #       Document upload & pipeline ingestion
│   │       ├── auth.py               #       JWT authentication, login, refresh tokens
│   │       ├── admin.py              #       Admin dashboards (metrics, jobs, logs)
│   │       ├── feedback.py           #       User thumbs up/down feedback collection
│   │       └── sessions.py           #       Conversation session history CRUD
│   ├── core/                         #   Shared Domain Core (entities & abstractions)
│   │   ├── models.py                 #     Common database and JSON models (User, ChatRequest)
│   │   ├── ports.py                  #     Ports (interface protocols for adapters)
│   │   ├── errors.py                 #     Circuit Breakers & application exceptions
│   │   ├── events.py                 #     Internal event bus for system communication
│   │   └── context_composer.py       #     Token-aware context window manager
│   ├── adapters/                     #   Concrete Infrastructure Adapters
│   │   ├── llm/                      #     LiteLLM universal adapter & cost trackers
│   │   ├── embedding/                #     Local (SentenceTransformer) & TEI clients
│   │   ├── reranker/                 #     Cross-encoder re-ranking client
│   │   ├── vectordb/                 #     ChromaDB (dev) and Qdrant (production) clients
│   │   ├── cache/                    #     Redis semantic cache adapter
│   │   └── stores/                   #     InMemory (dev) and MongoDB (production) storage
│   ├── agents/                       #   Domain: Agentic Planning & Specialists
│   │   ├── orchestrator/             #     LangGraph state machine graph execution & routing
│   │   └── specialists/              #     Choreographed domain specialist services
│   │       ├── landmark.py           #       LandmarkService (tourism catalog CRUD)
│   │       ├── lightrag.py           #       LightRAGService (Graph RAG queries)
│   │       └── rag.py                #       RAGService (Standard RAG & chat coordinator)
│   ├── rag/                          #   Domain: Retrieval-Augmented Generation algorithms
│   │   ├── ingestion/                #     Loader, chunker, and document hash managers
│   │   ├── pipeline/                 #     RAG steps (HyDE, CRAG grading, rewriters)
│   │   └── retrievers/               #     BM25 + vector hybrid fusion retrievers
│   ├── governance/                   #   Domain: Budgeting, Decryption, & Content Safety
│   │   ├── cost.py                   #     Daily budget trackers and token budget manager
│   │   ├── guardrails.py             #     Presidio PII maskers & input guardrails
│   │   └── kms.py                    #     KMS secret decryption tools
│   ├── memory/                       #   Domain: Multi-tier Personalization Memory
│   │   ├── working.py                #     Short-term thread memory assembly
│   │   ├── semantic.py               #     mem0 personalization and profile recall
│   │   └── episodic.py               #     Background episodic summarization tasks
│   ├── observability/                #   Domain: Monitoring & Telemetry
│   │   ├── metrics.py                #     Prometheus metrics definitions
│   │   └── tracing.py                #     LangSmith & Langfuse span tracing managers
│   └── tools/                        #   Domain: Tool Integrations & MCP
│       ├── registry.py               #     Unified function-to-tool compilers
│       ├── builtin.py                #     Internal tools (calculator, calendar)
│       └── mcp/                      #     Model Context Protocol servers & adapters
├── config/                           # External configuration files
│   └── routes.yaml                   #   Intent router utterances + thresholds
├── tests/                            # Comprehensive Test Suite
│   ├── unit/                         #   518 unit tests (mocked dependencies)
│   │   ├── test_prompts.py           #     Prompt hub and few-shot registry tests
│   │   ├── test_litellm_gateway...   #     Gateway observability & callback tests
│   │   ├── test_memory.py            #     mem0 multi-tier memory tests
│   │   └── ...                       #     + api, auth, vecto

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [vtnguyen04](https://github.com/vtnguyen04)
- **Source:** [vtnguyen04/KnowledgeForge](https://github.com/vtnguyen04/KnowledgeForge)
- **License:** MIT

Install and usage instructions live in the source repository linked above.