# Codebase Agent Skill

> Tree-sitter + pgvector = semantic │ code search for AI agents

- **Type:** MCP server
- **Install:** `agentstack add mcp-dr34dl10n-codebase-agent-skill`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [dr34dl10n](https://agentstack.voostack.com/s/dr34dl10n)
- **Installs:** 0
- **Category:** [Cloud & Infrastructure](https://agentstack.voostack.com/c/cloud-infrastructure)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [dr34dl10n](https://github.com/dr34dl10n)
- **Source:** https://github.com/dr34dl10n/Codebase-Agent-Skill

## Install

```sh
agentstack add mcp-dr34dl10n-codebase-agent-skill
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# codebase-skill

**Semantic Code Search for AI Agents**

*Stop feeding entire repos to your context window. Search surgically instead.*

  

[](https://github.com/dr34dl10n/Codebase-Agent-Skill/stargazers)
[](https://github.com/dr34dl10n/Codebase-Agent-Skill/fork)
[](https://www.python.org)
[](https://tree-sitter.github.io)
[](https://github.com/pgvector/pgvector)
[](https://huggingface.co/nomic-ai/modernbert-embed-base)
[](https://modelcontextprotocol.io)
[](LICENSE)

---

## Why?

AI agents waste tokens reading entire files they don't need. A 500k-LOC repo becomes a 30-minute scroll fest. This skill turns your codebase into a **searchable vector database** — the agent asks a question, gets 8–15 precise chunks back, and skips the noise.

**Before:** "Let me read all 47 files in `src/auth/` to find how JWT validation works..."
**After:** One semantic query → the 3 functions that matter, with line numbers.

This isn't a search engine for humans. It's a **RAG backbone for AI agents** — exposed as MCP tools your agent calls like any other tool.

---

## Benchmark: Context Loading with ModernBERT

Real benchmark on a mid-size project (**AIssistant**: 406 chunks, 83 source files, ~312K tokens).
Eight representative developer queries, compared across models and strategies.

### Headline Numbers

| Strategy | Avg Time | Avg Context Tokens | vs Naive | vs Smart |
|----------|----------|--------------------|----------|----------|
| **Naive Traditional** (grep → read all) | ~16ms | ~44M | 1× | — |
| **Smart Traditional** (grep → top-5 files) | ~18ms | ~3.8M | 0.1× | 1× |
| **ModernBERT-base + pgvector** ⭐ | **66ms** | **~3.2K** | **13,700×** | **1,185×** |
| **Nomic-embed + pgvector** *(optional)* | 59ms | ~2.8K | 15,700× | 1,352× |

> **13,700× less context** than naive grep. That's the difference between sending an entire codebase and sending 3 chunks to your LLM.

### Per-Query Detail

```
Query                                      ModernBERT-base          Nomic-embed-text
                                            time/tokens/score       time/tokens/score
────────────────────────────────────────────────────────────────────────────────────────────
how does authentication work                83ms/3.2Ktok/0.556     66ms/1.8Ktok/0.610
send an email via gmail                     61ms/2.5Ktok/0.619     60ms/2.9Ktok/0.671
calendar event creation                    54ms/3.1Ktok/0.705     51ms/3.2Ktok/0.745
telegram bot message handler                64ms/38Ktok*/0.735     52ms/3.6Ktok/0.741
error handling and retries                 66ms/1.2Ktok/0.601     61ms/1.6Ktok/0.595
how is the agent run loop structured       88ms/5.8Ktok/0.619     69ms/3.3Ktok/0.550
memory and context management              59ms/3.2Ktok/0.579     48ms/3.5Ktok/0.623
Google Workspace OAuth flow                54ms/3.6Ktok/0.716     66ms/2.2Ktok/0.618
────────────────────────────────────────────────────────────────────────────────────────────
AVERAGE                                    66ms/3.2Ktok            59ms/2.8Ktok
```

*\* The "telegram" outlier for ModernBERT returns a large class chunk (34K tokens) — a chunking granularity issue, not an embedding quality issue.*

### Model Comparison: What This Means

| | ModernBERT-base (default) | Nomic-embed-text (optional) |
|--|:--:|:--:|
| **Top-1 similarity** | Wins 3/8 queries | Wins 5/8 queries |
| **Avg query latency** | ~66ms | ~59ms |
| **Avg context returned** | ~3.2K tokens | ~2.8K tokens |
| **Context window** | 8,192 tokens | ~8,192 tokens |
| **Dimensions** | 768 | 768 |
| **Backend** | sentence-transformers (Python, zero-config) | Ollama (requires running server) |
| **External dependency** | ❌ None — model loads in-process | ✅ Ollama must be running |
| **Setup** | `pip install sentence-transformers` | `ollama pull nomic-embed-text` |
| **Model download** | Auto from HuggingFace | Manual `ollama pull` |
| **Best for** | **Default. All setups.** | GPU servers with Ollama already running |

### Takeaway

- Both models deliver **3–4 orders of magnitude** context reduction vs grep.
- **ModernBERT is the default** — zero-config, no running service, pure Python.
- Nomic scores slightly higher on some queries but requires a separate Ollama server.
- **ModernBERT wins** on complex structural queries ("agent run loop", "OAuth flow"), Nomic wins on keyword-aligned queries.
- At ~60ms/query, both are fast enough for interactive agent use.
- The **real cost saving** is tokens: 3K tokens vs 44M tokens means your LLM calls cost **~14,700× less**.

---

## Use Cases

### 🤖 Autonomous Coding Agents (Hermes, Codex, OpenHands)
Your agent explores an unfamiliar codebase autonomously. Instead of reading every file top-to-bottom, it calls `mcp_codebase_search("Where is the payment gateway integration?")` and gets the 5 relevant functions — then acts on them.

### ✨ IDE-Integrated Agents (Cursor, Claude Code, Windsurf)
While you code, the inline agent searches the entire repo semantically: "Find all uses of the `UserService` class" → instant, accurate results across files, no grep gymnastics.

### 🏗️ DevOps / Platform Agents
A CI/CD agent searches infra-as-code repos: "Which Helm charts set resource limits?" → ranked results across `values.yaml`, templates, and helpers, not just filename matches.

### 🔒 Sovereign AI Pipelines
No cloud API calls, no data leaving your infrastructure. ModernBERT loads in-process from HuggingFace, storage in your own PostgreSQL. Your code never leaves your network — perfect for defense, finance, healthcare.

---

## Architecture

```mermaid
graph LR
    subgraph "Indexing Pipeline"
        A[Repositoryany language] --> B[Tree-sitterAST chunking]
        B --> C[Embed ModelModernBERT (default)Nomic (optional)]
        C --> D[(PostgreSQL+ pgvector)]
    end

    subgraph "Query Pipeline"
        E[AI AgentMCP / REST / CLI] --> F[Embed query]
        F --> G[Cosine similarityHNSW index]
        G --> H[Ranked chunks+ metadata]
        H --> E
        D -.-> G
    end

    style D fill:#f0883e,color:#fff
    style B fill:#2563eb,color:#fff
    style C fill:#7c3aed,color:#fff
    style E fill:#3fb950,color:#fff
```

---

## Comparison: codebase-skill vs LangChain + Chroma/Pinecone

| Dimension | **codebase-skill** | **LangChain + Chroma/Pinecone** |
|-----------|-------------------|--------------------------------|
| **Chunking** | Tree-sitter AST (functions, classes, methods) | Recursive text splitter (character-based splits) |
| **Chunk quality** | ✅ Syntactically coherent — never splits a function in half | ⚠️ May cut mid-function, break indentation, lose context |
| **Embedding model** | ModernBERT (default, in-process) or Ollama (optional API) | Cloud API (OpenAI) or local, but no unified config |
| **Vector store** | PostgreSQL + pgvector (HNSW) | Chroma (file-based) or Pinecone (SaaS) |
| **Infrastructure** | 1 PostgreSQL you already run | Chroma = ephemeral/local **or** Pinecone = vendor lock-in |
| **Data sovereignty** | ✅ 100% on-prem — zero data egress | ⚠️ Pinecone = code sent to US cloud; Chroma = not prod-ready |
| **Query latency** | ~60ms (local embed + HNSW) | ~200–500ms (cloud API round-trip) |
| **Incremental reindex** | ✅ Built-in — only changed files | ❌ Full reindex on every change |
| **Cost at scale** | PostgreSQL + ModernBERT = $0/mo extra | Pinecone: $70/mo (S1 pod) + OpenAI embed API fees |
| **Agent protocol** | MCP (native stdio) | Custom Python API, no standard agent protocol |
| **Languages** | 25 out of the box (Tree-sitter) | Unlimited (text-based, no AST awareness) |
| **Metadata** | Symbol name, file path, line range, language | Custom metadata (user must implement) |

### TL;DR

| | codebase-skill | LangChain + Chroma | LangChain + Pinecone |
|--|:--:|:--:|:--:|
| **Setup** | 1 command | Python script | Account + API key |
| **Extra infra** | None (use your PG) | Chroma server | Pinecone SaaS |
| **Monthly cost** | **$0** | $0 (local, fragile) | **$70+** |
| **Code leaves network** | **Never** | No (local) | **Yes** |
| **Chunk coherence** | **AST-aware** | Text-split | Text-split |

---

## Tech Stack

| Layer | Tech | Why |
|-------|------|-----|
| **Parsing** | [Tree-sitter](https://tree-sitter.github.io) + tree-sitter-languages | AST-aware chunking by function/class, not line splits. 25 languages. |
| **Embeddings** | [ModernBERT](https://huggingface.co/nomic-ai/modernbert-embed-base) (sentence-transformers) | Default. In-process, zero-config, auto-downloads from HuggingFace. |
| | [Nomic-embed-text](https://ollama.com/library/nomic-embed-text) (Ollama) | Optional. Requires a running Ollama server. |
| **Storage** | [PostgreSQL](https://postgresql.org) + [pgvector](https://github.com/pgvector/pgvector) | HNSW index for sub-ms cosine search. ACID, proven, no new infra. |
| **Agent Interface** | [MCP](https://modelcontextprotocol.io) (stdio) | Standard protocol — works with Hermes, Claude Code, Cursor, Pi, Codex, any MCP client. |
| **API** | [FastAPI](https://fastapi.tiangolo.com) | Optional HTTP endpoints. Same logic, REST access. |
| **CLI** | argparse | `cbsearch`, `cbcontext`, `cbstats` — terminal-first, scriptable. |

---

## 5 MCP Tools Your Agent Gets

| Tool | What it does | When to use |
|------|-------------|-------------|
| `search` | Semantic search with filters (language, file pattern, repo) | "Where is auth implemented?" "Find all database connection code" |
| `file_context` | A file's chunks + semantically related chunks from other files | Understanding a file and its dependencies without reading everything |
| `stats` | Chunk count, file count, language count | "Is this repo indexed?" "How big is the codebase?" |
| `reindex` | Refresh a repo: re-embed modified files + purge deleted file chunks | After pulling code changes, or when search seems stale |
| `list_projects` | List all indexed repos with last_indexed time | "What repos are tracked?" |

---

## Keeping the Index Fresh

Your codebase changes. The index needs to keep up.

### Incremental Reindex

Re-running `index` on an already-indexed repo only processes changes:

| Change | Handling |
|--------|----------|
| Modified file (mtime > last_indexed) | Re-parsed, old chunks deleted, new chunks embedded & stored |
| Deleted file | Chunks purged automatically (`orphan_chunks_purged` in stats) |
| New file | Parsed, embedded & stored normally |
| Unchanged file | Skipped (zero cost) |

```bash
# Incremental (fast — only changes)
.venv/bin/python3 cli.py index /path/to/repo

# Force full reindex
.venv/bin/python3 cli.py index /path/to/repo --force
```

Or via MCP: `mcp_codebase_reindex(repo_path="/path/to/repo")`

### Auto-Reindex Cron

A cron job runs `auto_reindex.py` every 4 hours, keeping all registered repos fresh automatically. It only reports when changes are detected.

```bash
# Manual run
.venv/bin/python3 auto_reindex.py

# Force full reindex of everything
.venv/bin/python3 auto_reindex.py --force
```

---

## 25 Languages

Python, JavaScript, TypeScript, TSX, JSX, Go, Rust, Java, C, C++, C#, Ruby, PHP, Swift, Kotlin, Scala, Lua, R, Bash, SQL, HTML, CSS, JSON, YAML, TOML, Markdown.

Don't see yours? Tree-sitter supports [many more](https://tree-sitter.github.io/tree-sitter/) — just add the grammar.

---

## Quick Start

### Prerequisites

**Option A — Docker mode (recommended, zero sudo):**
- Docker + docker compose
- Python 3.11+

**Option B — Local PostgreSQL mode:**
- PostgreSQL 15+ (with sudo to create extensions)
- Python 3.11+

**That's it.** ModernBERT auto-downloads from HuggingFace — no external service, no Ollama, no API keys.

Optional: Using Nomic-embed-text via Ollama

If you already have Ollama running on a GPU server and prefer that backend:

```bash
ollama serve
ollama pull nomic-embed-text   # 768-dim, ~8k context
```

Then set `CODEINDEX_EMBED_MODEL=nomic-embed-text` and `CODEINDEX_EMBED_BACKEND=ollama`.

### Deploy

```bash
git clone https://github.com/dr34dl10n/Codebase-Agent-Skill.git /data/codebase-skill
cd /data/codebase-skill
```

**Docker mode** (PostgreSQL in a container — no sudo, no local PG needed):

```bash
bash deploy.sh --docker
```

This clones/builds the [pgvectordb](https://github.com/dr34dl10n/pgvectordb) Docker image (PG 17 + pgvector), starts it, creates tables, sets up the Python venv, and writes a `.env` file. The DB runs on port 5433 by default.

**Local PostgreSQL mode** (use an existing PG installation):

```bash
bash deploy.sh 
```

That one command creates: DB user, database, pgvector extension, tables, Python venv + all dependencies, and runs a verification check.

### Database modes at a glance

| | Local PostgreSQL | Docker (`--docker`) |
|--|:--:|:--:|
| **Requires** | PG 15+ installed, sudo | Docker + compose |
| **Port** | 5432 | 5433 |
| **DB name** | codeindex | codebase |
| **DB user** | codeindex | postgres |
| **Default password** | *(you choose)* | postgres |
| **Schema init** | `init_db.sql` via psql | Auto on first container start |
| **Data persistence** | Your PG data dir | Docker volume `pgvectordb_data` |
| **Good for** | Servers already running PG | Dev machines, CI, no-sudo setups |

Switch modes anytime by setting `CODEINDEX_DB_MODE=local` or `CODEINDEX_DB_MODE=docker` in `.env`.

### Configure your agent

**Hermes Agent** — `~/.hermes/config.yaml`:

```yaml
mcp_servers:
  codebase-skill:
    command: /data/codebase-skill/.venv/bin/python3
    args: ["/data/codebase-skill/mcp_server.py"]
    env:
      CODEINDEX_DB_PASSWORD: "${CODEINDEX_DB_PASSWORD}"
```

**Claude Code / Cursor / any MCP client:**

```json
{
  "mcpServers": {
    "codebase-skill": {
      "command": "/path/to/codebase-skill/.venv/bin/python3",
      "args": ["/path/to/codebase-skill/mcp_server.py"],
      "env": { "CODEINDEX_DB_PASSWORD": "your_password" }
    }
  }
}
```

### Index & search

```bash
# Index a repo (first time: parses + embeds everything)
.venv/bin/python3 cli.py index /path/to/repo

# Search from terminal
.venv/bin/python3 cli.py search "authentication middleware"
./bin/cbsearch "database connection pool" --language python --top-k 5

# Or let your agent do it via MCP tools
# → mcp_codebase_search(query="authentication middleware")
```

---

## Auto-Setup for All Agents (`cbsetup`)

After indexing a repository, run `cbsetup` to **automatically generate instruction files and MCP configuration** for every supported coding agent. This ensures any agent opening the project knows it must use semantic search before reading files.

### Supported Agent Files

| File | Agent | What it does |
|------|-------|-------------|
| `AGENTS.md` | Pi, Codex, generic | Project instructions (search-first protocol) |
| `CLAUDE.md` | Claude Code | Project instructions |
| `.cursorrules` | Cursor (legacy) | Project rules |
| `.cursor/rules/codebase-search.mdc` | Cursor (newer) | Always-apply project rule |
| `.windsurfrules` | Windsurf / Codeium | Project rules |
| `.clinerules` | Cline | Project rules |
| `.github/copilot-instructions.md` | GitHub Copilot | Repo instructions |
| `.claude/settings.json` | Claude Code | MCP server config |
| `.cursor/mcp.json` | Cursor | MCP server config |
| `.cline/mcp.json` | Cline | MCP server config |
| `.windsurf/mcp.json` | Windsurf | MCP server config |
| `.pi-indexed` | All agents | Marker with indexing metadata |

### Usage

```bash
# Full setup — all agents, all MCP configs
.venv/bin/python3 cbsetup.py /path/to/repo
# Or via wrapper:
cbsetup /path/to/repo

# Preview without writing
.venv/bin/python3 cbsetup.py /path/to/repo --dry-run

# Only instruction files, no MCP configs
.venv/bin/python3 cbsetup.py /path/to/repo --instructions-only

# Only MCP configs, no instruction files  
.venv/bin/python3 cbsetup.py /path/to/repo --mcp-only

# Specific agents only
.venv/bin/python3 cbsetup.py /path/to/repo --agents claude_md cursorrules --mcp claude cursor
```

### Idempotent & Safe

- Existing files are **appended to**, never overwritten. A `` marker allows automatic updates without los

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [dr34dl10n](https://github.com/dr34dl10n)
- **Source:** [dr34dl10n/Codebase-Agent-Skill](https://github.com/dr34dl10n/Codebase-Agent-Skill)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** yes
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-dr34dl10n-codebase-agent-skill
- Seller: https://agentstack.voostack.com/s/dr34dl10n
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.