Mcp Cpp Project Indexer

mcp-cpp-project-indexer

mcp-cpp-project-indexer is a deterministic C++ source-range indexer for large, module-heavy projects and MCP-based AI code navigation.

It is not a compiler, LSP replacement, refactoring engine, semantic analyzer, or call-graph builder.

Its job is simple:

Find code. Read code. Do not guess code.

The indexer maps C++ symbols, files, and C++20 modules to exact source ranges so an AI can read only the code it needs.

Project and related AI orchestration work is documented on the MEF Programming homepage, including the ongoing relay/governance layer we are building around MCP tool use.

---

30-Second Overview

mcp-cpp-project-indexer builds a lightweight routing index over a C++ source tree. MCP clients can then ask deterministic questions such as:

• where is this function/class/data member?
• which exact source range should be read?
• which module imports or exports this partition?
• which changed hunk intersects which indexed symbol or data range?

The indexer returns metadata and original source ranges. It does not claim to understand the program. The AI still has to read the returned source and reason from that evidence.

Minimal workflow:

User asks about Widget::OnScroll
-> find_symbol("Widget::OnScroll")
-> read_symbol(symbolId)
-> AI explains only what was visible in that source range

This keeps large C++ projects out of the prompt until exact source evidence is needed.

---

5-Minute Quick Start

1. Clone this repository

git clone https://github.com/walti1972/mcp-cpp-project-indexer.git
cd mcp-cpp-project-indexer

2. Build an index for your C++ project

python \build_project_index.py `
  --root  `
  --output-root \.mcp-cpp-project-indexer

The generated index is written to:

\.mcp-cpp-project-indexer

3. Start the MCP server

python \code_index_mcp_server.py `
  --project-root  `
  --index-root \.mcp-cpp-project-indexer

For multiple MCP clients or a long-running shared process, use HTTP transport:

python \code_index_mcp_server.py `
  --project-root  `
  --index-root \.mcp-cpp-project-indexer `
  --transport http `
  --http-host 127.0.0.1 `
  --http-port 8765

4. Add the server to your MCP client

Minimal LM Studio-style config:

{
  "mcpServers": {
    "mcp-cpp-project-indexer": {
      "command": "python",
      "args": [
        "\\code_index_mcp_server.py",
        "--project-root",
        "",
        "--index-root",
        "\\.mcp-cpp-project-indexer"
      ]
    }
  }
}

5. Ask for exact source, not whole files

Good first request:

Find the symbol Widget::OnScroll, read its implementation, and explain only
what is visible in the source range.

Expected tool path:

find_symbol -> read_symbol -> source-grounded answer

For best results, give your AI the rules from [prompttemplate.md](prompttemplate.md). The short version is:

Use metadata to locate code. Read exact source ranges before explaining behavior.
Do not infer implementation behavior from metadata alone.

---

Contents

• [5-Minute Quick Start](#5-minute-quick-start)
• [Production Scale & Performance](#-production-scale--performance)
• [TUI Control Center](#tui-control-center)
• [Repository Layout](#repository-layout)
• [Why This Tool?](#-why-this-tool)
• [Before / After](#-before--after)
• [How It Works](#how-it-works)
• [Core Workflow](#core-workflow)
• [What It Does](#what-it-does)
• [Build And Update](#build-a-project-index)
• [Project Discovery Config](#project-discovery-config)
• [Control Center](#control-center)
• [Start The MCP Server](#start-the-mcp-server)
• [Client Configuration](#lm-studio-mcp-configuration)
• [Possible Workflow Setups](#possible-workflow-setups)
• [Command Line Reference](#command-line-reference)
• [Tool Overview](#tool-overview)
• [Recommended AI Usage Rules](#recommended-ai-usage-rules)
• [Example Workflows](#example-workflows)
• [Design Rules](#design-rules)
• [Development Backstory](#-development-backstory)
• [Smoke Tests](#smoke-tests)
• [Maintenance Checklist](#maintenance-checklist)

---

Repository Layout

The public root readme.md is human-facing project documentation. The actual Python implementation lives under src/:

src/
  README.md
  indexer/
    build_project_index.py
    update_project_index.py
    cpp_project_index.py
  server/
    code_index_mcp_server.py
    server_ui/
  ui/
    indexer_tui.py
    indexer_control.py

Root-level scripts such as build_project_index.py, code_index_mcp_server.py, indexer_tui.py, and update_project_index.py are compatibility wrappers. They keep existing command lines and MCP client configs working while routing execution to the implementation package.

Folder-local READMEs under src/ use the project-indexer orientation format so agents can discover where to start without turning the public root README into machine-only documentation.

---

🚀 Production Scale & Performance

This project is used on real C++ codebases, not only toy examples. Two recent scale runs show the intended range:

| Project | Files | Source lines | Lexer tokens | Symbols | Data declarations | C++20 modules | Full build | |---|---:|---:|---:|---:|---:|---:|---:| | Anonymized commercial C++20 project | 7,046 | 979,658 | 4,682,882 | 97,924 | 36,551 | 3,754 | 19.5s | | Chromium checkout | 137,622 | 30,792,607 | 137,365,399 | 2,327,255 | 818,188 | 0 | ~24m 32s |

These numbers are machine-dependent. The Chromium run used --jobs 60 on a high-core workstation with an Intel Xeon Silver 4316 system, 128 GB RAM, and enterprise NVMe SSD storage. It is a useful public stress test because it exercises a very large classic include-based C++ codebase, while the anonymized commercial project exercises dense C++20 module and partition metadata. The Chromium run also validated the data/member indexer at scale: after fixing nested-template >> depth handling, the public stress test surfaced 46,529 additional data declarations and 66,866 additional data-name aliases.

The SQLite-backed lookup index keeps server startup practical even at Chromium scale: the MCP server can start immediately and stay around 200 MB RAM after startup instead of loading millions of symbol/data/name entries into Python objects.

It is designed for workflows that combine the Codex desktop app or other MCP clients with Visual Studio navigation and, when needed, binary/decompiler evidence from tools such as IDA Pro.

In one measured workflow, exact source-range routing reduced source text read from roughly 2,000 lines to 283 lines, an 86% reduction.

---

TUI Control Center

For daily use, the indexer includes an optional mouse-capable TUI. It turns the project index into a small local control center:

• start the HTTP MCP server and watcher from one place
• run full builds, incremental updates, fast updates, and module-map rebuilds
• watch live server, watcher, lock, process, token, and index stats
• inspect build/update logs without switching tools
• toggle diagnostic file sections for deeper parser evidence when needed

Install the optional UI dependency and start the control center with explicit project/index paths:

pip install -r \requirements-ui.txt

python \indexer_tui.py `
  --root  `
  --index-root \.mcp-cpp-project-indexer `
  --jobs 20 `
  --http-url http://127.0.0.1:8765

The UI is optional; the core indexer remains dependency-light and can still be driven entirely from scripts or MCP clients. For setup and keyboard shortcuts, see [Control Center](#control-center).

---

💡 Why This Tool?

Large C++ projects are expensive to feed into an AI model when entire files are loaded just to find one function, class, import, or declaration. C++20 modules make this harder: many IDE/LSP-style tools still struggle with large module graphs, partitions, generated SDK headers, and build-specific configuration.

This indexer solves a narrower but very practical problem: it gives the AI a small, deterministic routing map. The AI can locate the relevant symbol, module, file, or changed hunk first, then read only the exact original source lines needed for the task.

Example from a real bug-finding workflow:

Whole file context:     ~2000 source lines
On-demand source reads:  ~283 source lines
--------------------------------------------
Reduction:               ~86% less source text

---

📊 Before / After

| Standard AI code navigation | With mcp-cpp-project-indexer | |-----------------------------|------------------------------| | ❌ AI reads whole files to find a symbol | ✅ AI asks for compact metadata, then reads the exact source range | | ❌ Context fills with unrelated declarations and implementations | ✅ Context stays focused on the lines that matter | | ❌ C++20 module consumers/imports are hard to route through | ✅ Module imports, re-exports, partitions, and consumers are exposed directly | | ❌ Reviews start by scanning changed files manually | ✅ Change hunks are mapped to indexed symbol/data ranges | | ❌ Tooling may imply semantic certainty it does not have | ✅ The indexer only returns routing facts and original source ranges |

The result is lower token usage, lower latency, less context drift, and more source-grounded analysis.

---

How It Works

The indexer deliberately avoids pretending to be a compiler.

1. Fast token/structure scan

Source files are scanned with a lightweight Python lexer and structural parser. The output is a deterministic table of contents: files, symbols, data declarations, lexical #include directives, source ranges, diagnostics, and module facts.

1. C++20 module map

Module interfaces, partitions, imports, export-imports, and consumers are indexed so the AI can route through module-heavy code without asking an LSP to solve the whole build.

1. Incremental update and watcher

The updater tracks content hashes and rewrites only changed index data where possible. The optional watcher can keep the MCP server cache fresh while you work in Visual Studio.

1. MCP tools with compact output controls

Tools expose exact routing metadata first. The AI escalates only when needed: compact symbol lookup, file/module/change overview, exact read_symbol or read_range, then deeper recursive source reads.

The indexer is only the table of contents. The AI performs recursive exploration and code review from the original source lines it explicitly reads.

---

Core Workflow

Instead of this:

Read Renderer.cpp completely: ~2000 lines

use this:

find_symbol("Renderer::Paint")
read_symbol(symbolId)
inspect visible calls
read only relevant project callees

For changed-code review:

list_changed_files
get_file_change_hunks(includeIndexedRangeSummary:true, includeSource:false)
get_file_change_hunks(symbolId/dataId, includeSource:true)
read_symbol/read_range only when current source behavior is needed

---

What it does

The scanner extracts routing facts from C++ source files:

• files and stable file IDs
• C++20 modules and partitions
• lexical #include directives
• imports and exports
• namespaces
• classes / structs / enums
• functions / methods
• constructors / destructors / operators
• declarations and inline definitions
• exact startLine / endLine
• diagnostics for structurally suspicious files

It is stream/token based, not regex based.

---

What it does not do

Intentionally not included:

• no compiler-accurate whole-program call graph
• no find_references
• no type resolution
• no template-instantiation resolution
• no compiler-accurate overload resolution
• no macro expansion
• no semantic summaries
• no bug analysis
• no analyze_symbol(symbolId)

The AI should read source ranges and reason from the original code.

---

Output layout

Default output directory:

/.mcp-cpp-project-indexer/

Generated files:

.mcp-cpp-project-indexer/
  manifest.json
  files/
    f_.json
  index.sqlite
  modules.json
  diagnostics.json
  update_state.json      # written by build/update; used for fast incremental updates
  module_map.json        # generated by build_module_map.py
  .watch_update_summary.json  # temporary watcher/update summary
  .update.lock           # process lock for index writers
  .watcher.lock          # process lock for one active watcher

Global symbol and data routing indexes are stored in index.sqlite. The per-file JSON indexes remain the source of truth for exact source ranges and incremental rebuilds.

Optional JSONL export:

python \export_index_jsonl.py --index-root \.mcp-cpp-project-indexer --kind symbols --output symbols.jsonl
python \export_index_jsonl.py --index-root \.mcp-cpp-project-indexer --kind data --output data.jsonl

Scanner diagnostic file-index fields are emitted only with --emit-diagnostics or --emit-diagnostic-file-indexes:

scopeIntervals
structuralEvents
functionBodyRanges

---

Index one file

From any directory:

python \build_file_index.py `
  --file \path\to\file.ixx `
  --project-root  `
  --output \.mcp-cpp-project-indexer\diagnostic_file.json

With scanner diagnostic data:

python \build_file_index.py `
  --file \path\to\file.ixx `
  --project-root  `
  --output \.mcp-cpp-project-indexer\diagnostic_file.json `
  --emit-diagnostics

If --project-root is omitted, the file's parent directory is used.

---

Build a project index

Recommended usage from the C++ project root:

cd 
python \build_project_index.py

This writes to:

/.mcp-cpp-project-indexer/

Explicit form:

python \build_project_index.py `
  --root  `
  --output-root \.mcp-cpp-project-indexer

Example summary:

Built cpp.project_index.v1
Root: 
Output: /.mcp-cpp-project-indexer
Files: 7076
Symbols: 97583
Names: 95674
Modules: 3774
Diagnostics: 7
Total code lines: 1750000
Total tokens: 14200000
SQLite index: /.mcp-cpp-project-indexer/index.sqlite

Total tokens is the indexer's lexer token count over the indexed source after comment blanking. It is a project-size metric, not an LLM billing-token count.

When the project root is inside a Git worktree and git is available, file discovery respects Git ignore rules by filtering candidates through git check-ignore --stdin. This excludes paths matched by .gitignore, .git/info/exclude, or the user's global Git ignore file. Non-Git projects, or systems without Git, fall back to the built-in excluded directory list. Dot-directories such as .git, .vs, .cache, .idea, or .folder are excluded by default.

Project discovery config

For large projects with mixed source layouts, place indexer_config.json in the project root or in any subdirectory. Config files are applied while walking the tree: the root config becomes the base, and subdirectory configs can override or extend it for that subtree.

Example:

{
  "addExtensions": [".mm"],
  "addExcludeDirs": ["generated", "third_party"],
  "includeExtensionlessHeaders": true,
  "useGitIgnore": false
}

Supported fields:

{
  "extensions": [".cpp", ".cc", ".h"],
  "addExtensions": [".mm"],
  "removeExtensions": [".c"],
  "excludeDirs": ["out", "build"],
  "addExcludeDirs": ["generated"],
  "removeExcludeDirs": ["third_party"],
  "includeExtensionlessHeaders": true,
  "useGitIgnore": false
}

extensions and excludeDirs replace the inherited values for that subtree. add* and remove* fields modify the inherited values. Extensionless header discovery is conservative and opt-in; it only accepts extensionless files whose first lines look like C/C++ headers. useGitIgnore:false disables the final git check-ignore --stdin pass for very large repositories

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: mef-programming
• Source: mef-programming/mcp-cpp-project-indexer
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.