Contextweaver Mcp

ContextWeaver

🧵 A codebase context engine woven for AI agents

Semantic Code Retrieval for AI Agents — Hybrid Search • Graph Expansion • Token-Aware Packing

English · 简体中文

---

ContextWeaver is a semantic retrieval engine purpose-built for AI coding assistants. It combines hybrid search (vector + lexical), intelligent context expansion, and token-aware packing to deliver precise, relevant, and context-complete code snippets to LLMs.

✨ Core Features

🔍 Hybrid Retrieval Engine

• Vector Retrieval: deep semantic understanding via similarity
• Lexical Retrieval (FTS): exact matching for function names, class names, and other technical terms
• RRF Fusion (Reciprocal Rank Fusion): intelligently merges multiple recall channels

🧠 AST Semantic Chunking

• Tree-sitter parsing: supports TypeScript, JavaScript, Python, Go, Java, Rust, C, C++, C#, and more
• Dual-Text strategy: displayCode for presentation, vectorText for embedding
• Gap-Aware merging: handles code gaps intelligently while preserving semantic integrity
• Breadcrumb injection: vector text carries hierarchical paths to boost recall
• UTF-16 character-domain normalization: offsets are unified via SourceAdapter.toCharOffset before writing metadata, preventing multi-byte character slicing errors (v1.4.0+)

📊 Three-Stage Context Expansion

• E1 Neighbor expansion: adjacent chunks within the same file, preserving block completeness
• E2 Breadcrumb completion: sibling methods under the same class/function for structural understanding
• E3 Import resolution: cross-file dependency tracking (configurable toggle)

🎯 Smart TopK Cutoff

• Anchor & Floor: dynamic threshold plus an absolute floor as dual safeguards
• Delta Guard: prevents misjudgment in Top1-outlier scenarios
• Safe Harbor: the first N results only check the floor, guaranteeing baseline recall

🔌 Native MCP Support

• MCP Server mode: launch a Model Context Protocol server with one command
• Multi-tool granularity (v1.5.0+): beyond core semantic retrieval, adds dedicated tools for structure browsing, symbol references, symbol definitions, and statistics
• Intent/term separation: an LLM-friendly API design
• Auto-indexing: the first query triggers indexing automatically; incremental updates are transparent

⚡ Query Cache & File Watching (v1.5.0+)

• Query cache (QueryCache): in-process per-project LRU cache (50 entries by default); a hit skips the entire vector recall / rerank / expansion pipeline
• Automatic cache invalidation: the cache key is composed of normalized query + projectId + index version + search-config fingerprint, so it invalidates automatically after an index update or config change — stale results are never returned
• Watch mode: contextweaver watch watches the filesystem and triggers incremental indexing automatically, with debouncing (500ms by default) and scan de-duplication (no concurrent scans)

📈 Statistics & Observability (v1.5.0+)

• Three metric groups: indexing process, search quality/behavior, health/consistency
• Dual exits: contextweaver stats CLI (with --json) plus the MCP stats tool
• Consistency diagnostics: automatically detects abnormal migration state, pending_marks backlog, missing vector rows, and more — with suggested fixes

🛡️ Crash-Safe Data Architecture (v1.4.0+)

• Single source of truth for content: LanceDB stores only vectors and locating metadata; content is read back from files.content, reducing index size by 30–50%
• Cross-store transactional compensation: three-stage write LanceDB → FTS+outbox → SQLite mark, with automatic rollback or replay on any failure
• Migration state machine: pending/done/aborted persisted, auto-rebuilt on crash recovery
• Cross-process mutual exclusion: an advisory lock prevents the MCP server and CLI from triggering LanceDB migration concurrently
• chunk_id de-duplication: pre-delete before write to avoid duplicate rows on retry

📦 Quick Start

Requirements

• Node.js >= 20
• pnpm (recommended) or npm

Installation

# Global install
npm install -g @chiway/contextweaver

# Or with pnpm
pnpm add -g @chiway/contextweaver

Initialize Configuration

# Create the config file (~/.contextweaver/.env)
contextweaver init
# Or the short alias
cw init

Edit ~/.contextweaver/.env and fill in your API keys:

# Embedding API config (required)
EMBEDDINGS_API_KEY=your-api-key-here
EMBEDDINGS_BASE_URL=https://api.siliconflow.cn/v1/embeddings
EMBEDDINGS_MODEL=BAAI/bge-m3
EMBEDDINGS_MAX_CONCURRENCY=10
EMBEDDINGS_DIMENSIONS=1024

# Reranker config (required)
RERANK_API_KEY=your-api-key-here
RERANK_BASE_URL=https://api.siliconflow.cn/v1/rerank
RERANK_MODEL=BAAI/bge-reranker-v2-m3
RERANK_TOP_N=20

# Search parameters (optional, override built-in defaults)
CW_SEARCH_WVEC=0.6
CW_SEARCH_WLEX=0.4
CW_SEARCH_RERANK_TOP_N=10
CW_SEARCH_MAX_TOTAL_CHARS=48000
CW_SEARCH_VECTOR_TOP_K=80
CW_SEARCH_SMART_MAX_K=8
CW_SEARCH_IMPORT_FILES_PER_SEED=3

# Ignore patterns (optional, comma-separated)
# IGNORE_PATTERNS=.venv,node_modules

Index a Codebase

# Run from the codebase root
contextweaver index

# Specify a path
contextweaver index /path/to/your/project

# Force a full re-index
contextweaver index --force

Watch Mode (v1.5.0+)

# Watch for file changes and auto-index incrementally (Ctrl+C to stop)
contextweaver watch

# Specify a path and debounce window (ms)
contextweaver watch /path/to/project --debounce 800

watch runs one full incremental scan on startup, then listens to filesystem events; changes trigger a de-duplicated scan within the debounce window, and paths excluded by ignore rules never trigger a scan.

Local Search

# Semantic search
cw search --information-request "How is the user authentication flow implemented?"

# With exact terms
cw search --information-request "Database connection logic" --technical-terms "DatabasePool,Connection"

Structure Browsing & Symbol Lookup (v1.5.0+)

The following commands are CLI mirrors of MCP tools, with zero Embedding API cost:

# List indexed files (supports glob / language / count filters)
contextweaver list-files --glob "src/**/*.ts" --language typescript --max-results 100

# Look up a symbol definition
contextweaver definition SearchService --hint-path src/search

# Look up symbol references
contextweaver references handleStats --exclude-definition

Statistics (v1.5.0+)

# Human-readable stats report
contextweaver stats

# JSON output (for scripting)
contextweaver stats --json

# Specify a project path
contextweaver stats --path /path/to/project

Start the MCP Server

# Launch the MCP server (for use by Claude and other AI assistants)
contextweaver mcp

Index Management (v1.4.0+)

# Show LanceDB migration state
contextweaver migrate

# Clear the aborted state: wipe LanceDB and trigger a full rebuild
# Triggered when: the Indexer refuses to write after sampling validation fails;
# run this, then index again.
contextweaver migrate --reset

# Specify a project path
contextweaver migrate --path /path/to/project

🔧 MCP Integration

Claude Desktop Configuration

Add the following to your Claude Desktop config file:

{
  "mcpServers": {
    "contextweaver": {
      "command": "contextweaver",
      "args": ["mcp"]
    }
  }
}

MCP Tools Overview (v1.5.0+)

ContextWeaver exposes 5 MCP tools, following a layered design of "semantic retrieval first, structure browsing second":

| Tool | Purpose | Embedding cost | |------|---------|----------------| | codebase-retrieval | Primary tool: hybrid semantic + exact-match retrieval | Yes | | list-files | List indexed file structure (path/language/size) | No | | find-references | Find heuristic text references to a symbol | No | | get-symbol-definition | Find likely definition blocks for a symbol | No | | stats | Index/search/health statistics | No |

codebase-retrieval Parameters

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | repo_path | string | ✅ | Absolute path to the repository root | | information_request | string | ✅ | The semantic intent in natural language | | technical_terms | string[] | ❌ | Exact technical terms (class/function names, etc.) | | mode | string | ❌ | Retrieval profile: quick, balanced, or deep | | include_globs | string[] | ❌ | File glob allowlist applied after retrieval | | exclude_globs | string[] | ❌ | File glob denylist applied after retrieval | | language | string[] | ❌ | Language allowlist applied after retrieval | | max_total_chars | number | ❌ | Per-call output budget in characters | | max_files | number | ❌ | Maximum number of files returned after packing | | max_segments_per_file | number | ❌ | Maximum non-contiguous segments per file | | return_debug | boolean | ❌ | Include debug metadata in structured output | | low_confidence_behavior | string | ❌ | Low-confidence handling: return_top1, return_empty, or return_with_warning | | output_format | string | ❌ | Response format: markdown, json, or both |

list-files Parameters

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | repo_path | string | ✅ | Absolute path to the repository root | | glob | string | ❌ | Glob pattern to filter paths | | language | string | ❌ | Language filter (matched against files.language) | | max_results | number | ❌ | Max files to return (default 200) |

find-references Parameters

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | repo_path | string | ✅ | Absolute path to the repository root | | symbol | string | ✅ | Exact symbol name | | exclude_definition | boolean | ❌ | Exclude chunks whose breadcrumb tail matches the symbol name | | max_results | number | ❌ | Max references to return (default 50) |

get-symbol-definition Parameters

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | repo_path | string | ✅ | Absolute path to the repository root | | symbol | string | ✅ | Exact symbol name to resolve | | hint_path | string | ❌ | Preferred path to disambiguate same-name definitions | | max_results | number | ❌ | Max definitions to return (default 3) |

> Note: find-references and get-symbol-definition are heuristic text lookups over indexed chunks, not compiler-accurate navigation. For exhaustive raw text matching, use grep outside MCP.

Design Philosophy

• Intent/term separation: information_request describes "what to do", technical_terms filters "what it's called"
• Same-file context first: same-file context is provided by default; cross-file exploration is initiated by the agent
• Return to agent instincts: the tool only locates; cross-file exploration is triggered by the agent on demand

🏗️ Architecture

flowchart TB
    subgraph Interface["CLI / MCP Interface"]
        CLI[contextweaver CLI]
        MCP[MCP Server]
    end

    subgraph Search["SearchService"]
        QC[QueryCacheLRU]
        VR[Vector Retrieval]
        LR[Lexical Retrieval]
        RRF[RRF Fusion + Rerank]
        QC -.cache hit.-> CP
        VR --> RRF
        LR --> RRF
    end

    subgraph Expand["Context Expansion"]
        GE[GraphExpander]
        CP[ContextPacker]
        GE --> CP
    end

    subgraph Storage["Storage Layer"]
        VS[(VectorStoreLanceDB)]
        DB[(SQLiteFTS5)]
    end

    subgraph Index["Indexing Pipeline"]
        CR[Crawlerfdir] --> SS[SemanticSplitterTree-sitter] --> IX[IndexerBatch Embedding]
    end

    Interface --> Search
    RRF --> GE
    Search  Storage
    Expand  Storage
    Index --> Storage

Core Modules

| Module | Responsibility | |--------|----------------| | SearchService | Hybrid search core: coordinates vector/lexical recall, RRF fusion, rerank; integrates QueryCache | | QueryCache | Per-project in-process LRU cache (v1.5.0+); a hit skips the entire retrieval pipeline | | GraphExpander | Context expander: runs the E1/E2/E3 three-stage expansion strategy | | ContextPacker | Context packer: segment merging and token budget control | | ChunkContentLoader | Slices files.content by (path, start_index, end_index) (v1.4.0+) | | VectorStore | LanceDB adapter; exposes pure vector operations only | | Database (SQLite) | Metadata storage + FTS5 full-text index + statistics counters, schemaversion=3 | | Bootstrap | Cross-store init coordinator: pendingmarks replay + LanceDB schema migration (v1.4.0+) | | SemanticSplitter | AST semantic chunker (Tree-sitter); normalizes offsets to the UTF-16 character domain on write | | Watcher | File-watch coordinator (v1.5.0+): debounce + scan de-duplication + ignore filtering | | Stats | Statistics aggregation layer (v1.5.0+): combines index/search/health metrics |

Data Architecture (v1.4.0+)

~/.contextweaver//
├── index.db                 # SQLite
│   ├── files                # File metadata + full content (content column, the only source for text slicing)
│   ├── files_fts            # External-content table, inverted index pointing to files
│   ├── chunks_fts           # Chunk-level inverted index, per-file wholesale replacement
│   ├── metadata             # schema_version / lancedb_migration_state / lock
│   ├── stats                # Cumulative index/search counters (v1.5.0+)
│   └── pending_marks        # Outbox: replayed when a vector_index_hash mark failed
└── vectors.lance/           # LanceDB chunks table (vectors + locating metadata only, no content)

Key invariants:

• The single source of truth for content is files.content; ChunkContentLoader slices via start_index/end_index (same source as displayCode)
• All LanceDB offset fields live in the UTF-16 character domain; multi-byte files are never sliced incorrectly
• Cross-store write order: LanceDB → (FTS + outbox single transaction) → SQLite mark + clear outbox
• LanceDB migration state pending/done/aborted is persisted, with cross-process mutual exclusion via an advisory lock
• The query cache key is bound to the index version and search-config fingerprint; it invalidates on any index or config change

📁 Project Structure

contextweaver/
├── src/
│   ├── index.ts              # CLI entry (init / index / watch / search / mcp / migrate / stats)
│   ├── config.ts             # Config management (environment variables)
│   ├── defaultEnv.ts         # Default .env template
│   ├── cli/
│   │   └── mirrorCommands.ts # CLI mirrors of MCP tools (list-files / definition / references)
│   ├── api/                  # External API wrappers
│   │   ├── embedding.ts      # Embedding API
│   │   └── reranker.ts       # Reranker API
│   ├── chunking/             # Semantic chunking
│   │   ├── SemanticSplitter.ts   # AST semantic chunker
│   │   ├── SourceAdapter.ts      # Source adapter (UTF-16/UTF-8 domain normalization)
│   │   ├── LanguageSpec.ts       # Language spec definitions
│   │   ├── ParserPool.ts         # Tree-sitter parser pool
│   │   └── types.ts              # Chunking type definitions
│   ├── scanner/              # File scanning
│   │   ├── index.ts          # Scan orchestration
│   │   ├── crawler.ts        # Filesystem traversal
│   │   ├── processor.ts      # File processing
│   │   ├── watcher.ts        # File-watch coordinator (v1.5.0+)
│   │   ├── filter.ts         # Filter rules
│   │   ├── hash.ts           # File hash
│   │   └── language.ts       # Language detection
│   ├── indexer/              # Indexer
│   │   └── index.ts          # Three-stage transacti

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [wchiway](https://github.com/wchiway)
- **Source:** [wchiway/contextweaver-mcp](https://github.com/wchiway/contextweaver-mcp)
- **License:** MIT
- **Homepage:** https://contextweaver.work

Install and usage instructions live in the source repository linked above.