Cairn

Cairn

> The DocsGraph for AI agents. CodeGraph helps agents navigate code; Cairn > helps them navigate docs. Install it as docsgraph; keep the cairn name > for the product and compatibility alias.

[](https://github.com/jokeuncle/cairn/actions/workflows/ci.yml) [](LICENSE) [](https://pypi.org/project/docsgraph/) [](https://jokeuncle.github.io/cairn/) [](https://www.python.org/) [](https://modelcontextprotocol.io/)

Cairn is a local-first, MCP-native DocsGraph for software repositories and large structured documents. It turns README files, specs, ADRs, docs folders, PDFs, and optional MarkItDown-converted Office/data/web files into a navigable map: document catalog, hierarchical sections, multi-granularity summaries, entity mentions, cross-reference edges, and a semantic vector overlay.

Instead of dumping whole docs into context or relying on anonymous chunks, an agent can ask Cairn to list_documents, search_documents, inspect an outline, and drill into exact sections with stable cairn:// anchors. The same engine also works for standalone handbooks, papers, and PDFs.

The result: better retrieval accuracy, lower token spend, and a practical MCP tool layer between your project documentation and every AI coding agent you use. Local-first. Vendor-neutral. Designed for open-source repos.

> 🚀 Alpha — 0.1.0a12. Markdown + PDF ingest, the core MCP tool set, > the full structure-aware index (tree + summaries + entities + xrefs + > vectors), repo-level init/sync/status, repo-scoped MCP tools with > list_documents, search_documents, repo_context, repo_graph, and > repo_impact, failure-isolated sync, static graph inspector, Doubao > multimodal embeddings, and a benchmark harness with headline numbers. See > [CHANGELOG.md](CHANGELOG.md) for what's in this > release and [ROADMAP.md](ROADMAP.md) for what's next.

---

Why Cairn?

| Today | With Cairn | |---|---| | AI coding agents guess from README snippets or grep. | Agent gets a repo-level documentation map with stable section anchors. | | Dump the whole document into context. Burns tokens, dilutes attention. | Agent fetches only what it needs, at the granularity it needs. | | Naive RAG splits structure into context-free chunks. | The document's own structure is the index. | | Cross-references and entities are lost in chunking. | They are first-class objects. | | Locked into one vendor's embeddings / vector DB. | Pluggable everything. Local-first defaults. | | Different tool stacks for Claude / Cursor / Cline / Goose. | One MCP server. Any compliant agent works. |

For the in-depth motivation, see [PRODUCT.md](PRODUCT.md). For the technical design, see [ARCHITECTURE.md](ARCHITECTURE.md). For the public documentation quality contract Cairn optimizes for, see [docs/golden-docs-standard.md](docs/golden-docs-standard.md). The product website is published at .

---

How It Works (90 seconds)

1. Discover. docsgraph init -y writes .cairn/config.toml; docsgraph sync

discovers README, Markdown docs, ADRs, specs, and PDFs from conservative repo globs.

1. Index. Each document becomes a normal Cairn index: structural tree (T),

multi-level summaries (S), entity index (E), cross-reference graph (X), and vector overlay (V). A bad source file is isolated instead of breaking the whole repo sync. Sync is incremental by default: unchanged source hashes and producer fingerprints are skipped unless --force is passed. Concurrent syncs for the same repo are serialized by .cairn/sync.lock, so parallel agent sessions wait instead of rebuilding the same documents.

1. Serve. docsgraph serve exposes repo-scoped MCP tools:

list_documents, search_documents, plus outline, get_section, expand, search_semantic, search_keyword, find_mentions, get_related, and read_range routed by optional doc.

1. Navigate. Your agent searches across the repo, picks a document, drills

into promising sections, and only fetches full text when justified. Every result carries stable anchors for verification.

A visual explainer comparing Cairn's approach to RAPTOR, BookRAG, and A-RAG lives at [docs/retrieval-architecture-canvas.html](docs/retrieval-architecture-canvas.html). The repository workflow map lives at [docs/repo-docs-workflow-canvas.html](docs/repo-docs-workflow-canvas.html). Open either file in any browser.

---

Quickstart

The fastest way to see Cairn work is to index this repo's own documentation. Zero API keys, zero model downloads — the --fake flag uses deterministic in-process plugins so the whole thing runs offline.

The PyPI distribution is docsgraph; the primary CLI command is docsgraph. The older cairn command is installed as a compatibility alias:

pip install docsgraph

Or run it without installing:

uvx docsgraph --help

AI agents that can run shell commands can install and wire Cairn into their own MCP config. Start with a dry run, then write the config once the target path looks right:

uvx docsgraph init -y
uvx docsgraph sync --fake
uvx docsgraph install --client codex --dry-run --fake
uvx docsgraph install --client codex --yes --fake

Use --client claude, --client cursor, or --client goose for other MCP clients. docsgraph install writes the same server config that docsgraph mcp config prints, with command = "docsgraph" and args = ["serve"]. The server resolves the current MCP workspace at call time, so a global agent config does not stay pinned to the repo where it was installed. Pass --repo /path/to/repo to docsgraph install or docsgraph mcp config only when you intentionally want a fixed-repo MCP server.

Repository Workflow

Inside any repository:

docsgraph init -y
docsgraph discover
docsgraph sync --dry-run
docsgraph sync --fake
docsgraph status
docsgraph query repo "where are docs indexed?" --fake
docsgraph doctor --fake
docsgraph mcp config --client claude --fake
docsgraph serve --fake
docsgraph client --fake

docsgraph discover and docsgraph sync --dry-run explain what will be indexed before Cairn writes any document indexes: included docs, candidates that did not match include rules, unsupported extensions, and excluded files with their matching rule. docsgraph doctor checks repo config, index freshness, primary-doc routing, model settings, and whether the query embedder matches the indexed embedder mode and dimension. If an index was built with --fake but the MCP server is running a real embedder (or the reverse), doctor and fixed-repo serve print direct recovery commands. docsgraph mcp config prints copy-pasteable stdio snippets for Claude, Cursor, Codex, and Goose. For hosted providers, pass explicit MCP environment variables with repeated --env KEY=VALUE, or use --env-from-current to copy the current CAIRN_* environment into the generated config. docsgraph client opens a local browser UI for init, sync, status, repo_context previews, and MCP config install flows:

docsgraph mcp config --client claude
docsgraph mcp config --client cursor
docsgraph mcp config --client codex
docsgraph mcp config --client goose
docsgraph mcp config --client codex --repo . --env-from-current

For local development from source:

git clone https://github.com/jokeuncle/cairn.git
cd cairn

python3.11 -m venv .venv
.venv/bin/pip install -e ".[dev]"

# 1. Create .cairn/config.toml with conservative documentation globs.
.venv/bin/docsgraph init -y

# 2. Index README, Markdown docs, and PDFs.
.venv/bin/docsgraph sync --fake

# 3. Inspect freshness and indexed document ids.
.venv/bin/docsgraph status

# 4. Search across all indexed repository docs.
.venv/bin/docsgraph query repo "where are docs indexed?" --fake

# 5. Start the repo-scoped MCP stdio server for Claude Code / Cursor / Cline / Goose.
.venv/bin/docsgraph serve --fake

Repo mode writes a shareable config plus ignored runtime data:

.cairn/
  config.toml       # commit this if you want a stable repo docs policy
  manifest.json     # generated
  documents/        # generated per-document Cairn indexes
    readme/
    architecture/
    docs-specs-mcp-tools/

Repo-scoped MCP adds:

| Tool | Use it for | |---|---| | list_documents | See every indexed doc, its source path, freshness, section count, and structured agent usage guidance. | | search_documents | Search across all indexed docs and get globally ranked, explainable section hits with doc ids, skipped docs, and stale-doc warnings. | | repo_context | Get a ready-to-read context pack: ranked hits, selected section text, hit explanations, and a relationship map. | | repo_graph | Inspect the repo documentation graph: document, section, entity, contains, xref, and mention edges. Cross-document links are exposed through shared entity nodes. | | repo_impact | Estimate documentation surfaces affected by a document or section change. | | search_keyword | Search all indexed docs when doc is omitted, or one selected doc when doc is provided. | | normal Cairn tools + doc | Drill into a chosen document with outline, get_section, search_semantic, get_related, etc. |

Cairn is intentionally a docs graph, not a universal first step. Prefer repo_context / search_documents for documentation, product, architecture, setup, naming, protocols, business workflows, and durable decision records. Prefer CodeGraph or native project tools for source-code symbol tracing, small known-file edits, test output, build logs, and exact literal search. list_documents returns the same usage boundary as structured usage_guidance so MCP clients can route tasks without parsing README prose.

Repo behavior is intentionally configurable in .cairn/config.toml:

| Setting | Default | Impact | |---|---|---| | include | README, top-level Markdown/PDF, docs/**, guides/**, specs/**, ADR/decision/design/product/requirements/RFC/proposal dirs, one-level nested README | Expands or narrows what Cairn treats as repository documentation. Broader globs improve coverage but can index noisy generated files. | | exclude | .git, .cairn, .codegraph, caches, virtualenvs, package stores, build/site/coverage output, node_modules | Keeps generated or tool-owned docs out of search. Simple name/** directory excludes match at any depth, so frontend/node_modules/... and apps/web/dist/... are skipped. Add project-specific generated doc folders here. | | enable_markitdown | false | Enables non-Markdown/PDF conversion when the markitdown extra is installed. Useful for DOCX/PPTX/XLSX/HTML-heavy repos, slower and less deterministic than native Markdown/PDF parsing. | | primary_doc | readme | Chooses the default document for normal tools when doc is omitted in repo mode. | | search_sections_per_doc | 1 | Default diversity for search_documents. 1 helps agents find the right doc first; raise it when a repo has a few long docs and you want deeper hits from each doc by default. | | preferred_locales | [] | Optional locale preference for repo search, for example ["en"] or ["zh"]. When omitted, English queries prefer English or locale-neutral docs without hiding other languages. |

Markdown frontmatter is treated as generic document metadata. Cairn indexes common fields such as title, description, tags, date, and draft, uses them in keyword and semantic evidence, and marks metadata matches in search explanations. Empty parent headings remain useful navigation nodes, but they are down-ranked as primary evidence when a non-empty child section better answers the query.

MarkItDown integration is local-file only and optional. Cairn uses it as a conversion layer, then feeds the generated Markdown into the same canonical Markdown parser. This expands coverage to formats such as DOCX, PPTX, XLSX, HTML, CSV, JSON, XML, and EPUB without making the base install heavy:

pip install "docsgraph[markitdown]"
.venv/bin/docsgraph init -y --force --markitdown
.venv/bin/docsgraph sync --fake

Generate a standalone graph inspector for the primary repo doc:

docsgraph inspect --out /tmp/cairn-repo-inspector.html

Single Document Workflow

Cairn still works as a focused index for one large document:

# Index Cairn's own architecture document.
.venv/bin/docsgraph index ARCHITECTURE.md --out /tmp/cairn-arch --fake

# Get the map — gists only, never full text.
.venv/bin/docsgraph outline /tmp/cairn-arch --depth 2

# Keyword search: every section that mentions "LanceDB".
.venv/bin/docsgraph query keyword /tmp/cairn-arch LanceDB

# Multi-term keyword search with mode=all.
.venv/bin/docsgraph query keyword /tmp/cairn-arch progressive disclosure --mode all

# Generate a standalone graph inspector for the built index.
.venv/bin/docsgraph inspect /tmp/cairn-arch --out /tmp/cairn-arch/inspector.html

# Start a single-document MCP stdio server.
.venv/bin/docsgraph serve /tmp/cairn-arch --fake

A walkthrough with full output and an MCP-client config snippet is in [examples/hero-demo.md](examples/hero-demo.md).

Benchmarks

Cairn ships with cairn-bench, a small framework that compares Cairn against a naive 512-word-chunk vector-RAG baseline (both backed by LanceDB and the same embedder, so the comparison is apples-to-apples).

Running the starter suite (10 hand-curated questions over Cairn's own ARCHITECTURE.md) with deterministic in-process plugins:

docsgraph bench benchmarks/architecture.toml --fake

| metric | naive vector RAG | Cairn | |---|---:|---:| | mean recall@8 | 25% | 25% | | mean tokens returned | 3,677 | 1,386 (37.7% of naive) |

Caveat — these numbers come from the deterministic FakeEmbedder (a bag-of-words hash with no semantic understanding). Recall ties because neither system has semantics; the 2.6× token efficiency win is independent of the embedder: it comes from progressive disclosure and section-aware retrieval, not from vector quality. Cairn now returns a short evidence snippet with every semantic hit by default, which raises the token count but makes ranking errors easier to inspect. Reproduce these numbers in under a second on any machine — and re-run with Ollama (nomic-embed-text) or Doubao for the real-semantics version. See [benchmarks/README.md](benchmarks/README.md) for caveats and how to author your own suites.

Repo-level smoke tests are also public and reproducible:

python scripts/eval_repos.py --repo all --refresh --strict
python scripts/smoke_many_repos.py --limit 37 --strict

The labeled eval set covers astral-sh/uv, pydantic/pydantic-ai, modelcontextprotocol/python-sdk, and fastapi/full-stack-fastapi-template. The broad smoke matrix currently spans 37 public repositories across Python, JavaScript/TypeScript, Rust, and Go ecosystems. It is not an accuracy leaderboard; it verifies clone/discovery/sync/search/drilldown robustness and latency across different documentation shapes.

Latest fake-plugin runs on this machine:

| suite | result | |---|---| | pydantic-ai labeled eval | 178/178 docs indexed, 8/8 top1, 8/8 top5, 8/8 drilldown | | uv labeled eval | 89/89 docs indexed, 15/16 top1, 16/16 top3/top5, 16/16 drilldown | | mcp-python-sdk labeled eval | 17/17 docs indexed, 4/4 top1, 4/4 drilldown | | fastapi-template labeled eval | 7/7 docs indexed, 4/4 top1, 4/4 drilldown | | 37-repo smoke matrix | 2931 docs indexed, 0 sync failures, 185/185 searches with hits, 185/185 drilldowns |

search_documents uses a general hybrid ranker: dense vector similarity, BM25-style sparse evidence, structure-aware field support, weighted query-term coverage, path/title identity prior, and local graph-neighborhood propagation. Repo search builds a process-local cache and scores dense vectors in batches so large documentation sets stay warm-query friendly. On large section sets it uses a

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: jokeuncle
• Source: jokeuncle/cairn
• License: Apache-2.0
• Homepage: https://jokeuncle.github.io/cairn/

Install and usage instructions live in the source repository linked above.