Smart Context Mcp

smart-context-mcp

MCP server that reduces AI agent token usage by up to 90% through intelligent context compression (measured on this project).

[](https://www.npmjs.com/package/smart-context-mcp) [](https://opensource.org/licenses/MIT)

What it is

An MCP (Model Context Protocol) server that provides specialized tools for reading, searching, and managing code context efficiently. Instead of loading full files or returning massive search results, it compresses information while preserving what matters for the task.

Real metrics from production use:

• ~7M tokens → ~800K tokens (approximately 89% reduction)
• 1,500+ operations tracked across development
• Compression ratios: 3x to 46x depending on tool
• Context overhead is tracked separately so reports can show gross and net savings

Workflow-level savings:

• Debugging: ~85-90% token reduction
• Code Review: ~85-90% token reduction
• Refactoring: ~85-90% token reduction
• Testing: ~85-90% token reduction
• Architecture: ~85-90% token reduction

Real adoption in non-trivial tasks:

• Approximately 70-75% of complex tasks use devctx tools
• Most used: smart_read (850+ uses), smart_search (280+ uses), smart_shell (220+ uses)
• Primary reasons for non-usage: task too simple, no index built, native tools preferred

See [Workflow Metrics](./docs/workflow-metrics.md) and [Adoption Metrics](./docs/adoption-metrics-design.md) for details.

Latest Release: 1.20.0

Minor release. Same 20 tools, but several gain new parameters and response fields. SQLite schema bumps 7 → 8 (new read_cache table; auto-migrates on first run). Global memory DB schema bumps 1 → 2 (new noise_hints table). Zero new runtime dependencies.

• Shared tokenBudget across tools. smart_read, smart_read_batch, smart_context, smart_turn (start + end) and smart_resume now accept tokenBudget: number | { id?, maxTokens, shared? }. When shared:true (or id set), the budget is reused across calls inside the same task — so a multi-step agent flow can stay under a hard token ceiling without per-call bookkeeping. Responses include taskBudget, remainingBudget, and budgetDetails (scope, actions, degraded mode) when the budget actually changed the output.
• smart_search search modes. New mode: 'needle' | 'balanced' | 'semantic' (default balanced). needle = literal exact only (no regex / no term expansion) — kills noise on debug queries. balanced = exact + regex + term expansion. semantic = exact-first plus the local semantic block only when exact signal is weak. The previous semantic: true flag remains as a legacy alias for mode: 'semantic'. Default maxFiles tightened 15 → 5. New maxTokens caps the whole response and compacts intelligently (matches first, then diagnostics, then semantic block). Per-file ranking is now inspectable via matchedBy, boostSource, scoreBreakdown, whyRanked. Response also returns hasMore / totalFiles / nextSuggestedMaxFiles and actionable suggestions when the query is too broad or empty.
• smart_read persistent cache + budget-aware full degradation. New SQLite read_cache table keyed by (filePath, mode, selector, content_hash). Second read of an unchanged file is virtually free. Mode full is now an explicit last resort: if a tokenBudget/maxTokens is set, it degrades to lighter modes first (outline → signatures → truncated) and reports the real mode used in fullMode + budgetDetails. New clearReadCachePersistent + GC integration in runStorageMaintenance.
• smart_turn simple-task skip heuristic. When the prompt is short (≤ 40 chars after normalization), classified as a simple task, and no session/task is pinned, smart_turn(start) now returns skipSmartTurn: true with recommendedPath.mode='simple_task_skip' instead of paying the full orchestration cost. Saves continuity-resolution overhead on trivial prompts. minimal verbosity additionally compacts summary/refreshedContext to the fields agents actually consume.
• global_memory noise hints. Per-project, scrubbed noise telemetry persisted to ~/.devctx/global.db (noise_hints table). New actions noise_stats and noise_reset (full or via query). Lets smart_search learn which queries the agent already discovered to be noisy in a given repo and adjust ranking, without ever leaking content.
• KPI baseline infrastructure. New scripts evals/kpi-baseline.js + evals/kpi-utils.js aggregate harness.js and realworld-eval.js runs into a single JSON snapshot with top-5 precision, recall, reread task/call rate, and per-task-size buckets (short / long). Persists kpi-baseline-latest.json for regression detection across releases. New test suite tests/eval-kpis.test.js.

Highlights from 1.19.0 (still current)

Five-step quality jump executed as sequential commits with full dogfooding. MCP grows from 18 → 20 tools, +68 tests, zero new dependencies, suite green at 882/883 (1 skipped).

• smart_playbook (new tool). Declarative composite workflows that run multiple smart_* tools in a single MCP call. Five built-in playbooks ship with the package: preflight-merge (review + affected tests + checkpoint), debug-flake (last failure + curated debug context + affected), refactor-safe (curated context + affected + checkpoint), doc-sync (ADR search + docs context), ramp-up (status + doctor + ADR overview). Project-level overrides via .devctx/playbooks/*.{yaml,json} with {{args.X}} interpolation, when / label / stopOnFail / dryRun. Tool allowlist restricted to smart_*. Zero deps: built-in minimal YAML parser.
• Reactive FS watcher for the index. fs.watch (native, recursive, debounced 600ms + batch flush every 2s) keeps the symbol index hot between calls. Filters .git, node_modules, .devctx, dist, build, lockfiles, .min.*, .map, .snap, and non-indexable extensions. Stats surface in smart_status (enabled, flushes, eventsObserved, filesReindexed, filesRemoved, errors, lastFlushAt, pending). Opt-out via DEVCTX_WATCH_INDEX=false. Wired to MCP shutdown for clean close + final flush.
• Richer Python / Go parsers + pluggable parser registry. Python now captures decorators (decorators: ["dataclass", ...]), async def (kinds async-function / async-method), TypeAlias and TypeVar / NewType / ParamSpec / TypeVarTuple as kind="type", and respects class indent for accurate scope. Go now captures methods with receiver type as parent, interfaces as kind="interface", top-level const / var. src/parsers/registry.js exposes registerParser / getParser so future tree-sitter parsers can plug in without touching index.js. INDEX_VERSION bumped 6 → 7 (auto-reindex).
• Local semantic re-rank on smart_search. Opt-in semantic: true (with semanticLimit) returns a semantic: { embedder, symbols[], files[] } block ranked by hashing/TF-IDF embeddings (256-dim, FNV-1a buckets, L2-normalized, cosine similarity, "The MCP shines in long, multi-session tasks or when you don't know the codebase. For contained refactors where you already know what to touch, native tools are just as fast or faster. The real value was smart_read(outline) for the initial analysis and checkpoints to not lose the thread between sessions."

The 90% token savings are real, but they require the right task type to materialize.

---

Why it exists

AI agents waste tokens in three ways:

1. Reading full files when they only need structure or specific functions
2. Massive search results with hundreds of irrelevant matches
3. Repeating context across conversation turns

This MCP solves all three by providing tools that return compressed, ranked, and cached context.

---

🚨 Agent Ignored devctx? → Paste This Next

📋 Official Prompt (Copy & Paste)

Use smart-context-mcp for this task.
Start with smart_turn(start), then use smart_context or smart_search before reading full files.
End with smart_turn(end) if you make progress.

⚡ Ultra-Short Version

Use devctx: smart_turn(start) → smart_context → smart_turn(end)

When to use: Agent read large files with Read, used Grep repeatedly, or you see no devctx tools in a complex task.

Why this happens: Task seemed simple, no index built, native tools appeared more direct, or rules weren't strong enough.

---

Quick Start: Which Client Should I Use?

🎯 Best Default: Cursor

Use if: You work in Cursor IDE and want the best balance of guidance and flexibility.

Workflow:

1. Install MCP → rules auto-load
2. Start task → agent reads .cursorrules
3. Agent decides when to use devctx
4. Use /prompt commands to force usage if needed

Automaticity: Medium by default. Medium-High if you use the assisted launcher ./.devctx/bin/cursor-devctx for task-runner workflows.

---

🔄 Best Continuity: Claude Desktop

Use if: You want highest session continuity with automatic context recovery.

Workflow:

1. Install MCP + hooks
2. Start task → hook auto-triggers smart_turn(start)
3. Work with devctx tools
4. End task → hook auto-triggers smart_turn(end)

Automaticity: High (with hooks) - Can auto-trigger smart_turn on session start/end.

---

💻 Best Terminal: Codex CLI / Qwen Code

Use if: You prefer terminal-based workflows or scripting.

Workflow:

1. Install MCP
2. Rules embedded in prompts
3. Agent reads rules, decides when to use
4. Explicit instructions work best

Automaticity: Low-Medium - Rules are visible but require explicit prompting.

---

📊 Quick Comparison

| Client | Automaticity | Best For | |--------|--------------|----------| | Cursor | Medium | Complex IDE tasks | | Claude Desktop | High (hooks) | Session continuity | | Codex CLI | Low-Medium | Terminal workflows | | Qwen Code | Low-Medium | Alternative to Cursor |

Important: Agent always decides whether to use devctx. Rules increase probability, but don't guarantee it.

If you want a more repeatable path: use the task runner or the assisted launcher instead of relying on rules alone. See [Task Runner Workflows](./docs/task-runner.md).

📖 Full setup: [Client Compatibility](./docs/client-compatibility.md)

---

🚀 How to Invoke the MCP

Key point: The MCP doesn't intercept prompts automatically. You need to tell the agent to use it.

1️⃣ Use MCP Prompts (Easiest - Cursor only)

/prompt use-devctx

[Your task here]

Other prompts:

• /prompt devctx-workflow - Full workflow
• /prompt devctx-preflight - Build index + start session

2️⃣ Explicit Instruction (Any client)

Use smart_turn(start) to recover context, then [your task]

For a more guided CLI path:

smart-context-task task --prompt "your task"
smart-context-task implement --prompt "your task"
smart-context-task continue --session-id 
smart-context-task doctor

3️⃣ Automatic via Rules (Not guaranteed)

Agent should use devctx for complex tasks if rules are active:

• Cursor: .cursorrules
• Claude Desktop: CLAUDE.md
• Others: AGENTS.md

But: Agent decides based on task complexity.

⚡ Quick Reference

| Scenario | Command | |----------|---------| | Start new task | /prompt devctx-workflow | | Guided terminal workflow | smart-context-task task --prompt "..." | | Guided implementation | smart-context-task implement --prompt "..." | | Continue previous task | smart_turn(start) and continue | | Continue via runner | smart-context-task continue --session-id | | Force MCP usage | /prompt use-devctx | | First time in project | /prompt devctx-preflight | | Trust automatic rules | Just describe your task normally |

---

Recommended Workflow

✅ Setup Checklist (First Time in Project)

Before starting complex tasks, ensure:

# 1. MCP is installed
npm list -g smart-context-mcp  # or check your MCP client

# 2. Build the index (IMPORTANT)
npm run build-index
# or tell the agent: "Run build_index tool"

# 3. Rules are active
# - Cursor: .cursorrules exists
# - Claude Desktop: CLAUDE.md exists
# - Other clients: AGENTS.md exists

# 4. Start with smart_turn
# Tell the agent: "Use smart_turn(start) to begin"

Copy-paste to agent (first time):

Run build_index, then use smart_turn(start) to begin this task.

---

⚠️ Why Index Matters

Without index:

• ❌ smart_search returns unranked results
• ❌ smart_context can't build optimal context
• ❌ Agent may prefer native tools → no savings

With index:

• ✅ smart_search ranks by relevance
• ✅ smart_context includes related files
• ✅ 90% token savings enabled

When to rebuild:

• ✅ First time in project
• ✅ After major refactors (file moves, renames)
• ✅ After adding many new files
• ❌ Not needed every session (index persists in .devctx/)

---

The Entry Point: smart_turn(start)

For non-trivial tasks (debugging, review, refactor, testing, architecture), the optimal flow is:

0. build_index (if first time in project)
   ↓ enables search ranking and context quality
   
1. smart_turn(start, userPrompt, ensureSession=true)
   ↓ recovers previous context, classifies task, checks repo safety
   
2. smart_context(...) or smart_search(intent=...)
   ↓ builds context or finds relevant code
   
3. smart_read(mode=outline|signatures|symbol)
   ↓ reads compressed, cascades to full only if needed
   
4. [work: make changes, analyze, review]
   
5. smart_shell('npm test')
   ↓ verifies changes safely
   
6. smart_turn(end, event=milestone|blocker|task_complete)
   ↓ checkpoints progress for recovery

Why start with smart_turn?

• ✅ Recovers previous task checkpoint (goal, status, decisions)
• ✅ Classifies task continuation vs new task
• ✅ Provides repo safety check
• ✅ Enables task recovery if interrupted
• ✅ Tracks metrics for optimization

When to skip smart_turn:

• ❌ Trivial tasks (read single file, simple search)
• ❌ One-off questions (no continuity needed)
• ❌ Quick diagnostics (no session context)

The Product Entry Point: smart-context-task

If you want the same lifecycle packaged into named workflows, use the task runner:

smart-context-task task --prompt "inspect the auth flow and continue the bugfix"
smart-context-task implement --prompt "add a token guard to loginHandler"
smart-context-task review --prompt "review the latest diff"
smart-context-task doctor

This layer runs the same smart_turn(start) / context / checkpoint flow, but adds:

• workflow-specific preflight (smart_context or smart_search)
• continuity-aware prompt guidance
• blocked-state routing to smart_doctor
• measured task_runner quality signals

For the full command set and client-specific usage, see [Task Runner Workflows](./docs/task-runner.md).

---

How it Works in Practice

The Reality

This MCP does not intercept your prompts magically. Here's what actually happens:

1. You write a prompt: "Fix the login bug"
2. Agent reads rules: Sees debugging workflow suggestion
3. Agent decides: "This is a debugging task, I'll start with smart_turn(start)"
4. Agent calls: smart_turn({ phase: 'start', userPrompt: '...', ensureSession: true })
5. MCP returns: Previous task checkpoint (if exists) + repo safety check
6. Agent continues: Calls smart_search(intent=debug) for error location
7. Agent reads: Calls smart_read(mode=symbol) for specific function
8. Agent fixes bug: Makes changes
9. Agent verifies: Calls smart_shell('npm test')
10. Agent checkpoints: Calls smart_turn(end) to persist progress

Key points:

• ✅ Agent chooses to use devctx tools (not forced)
• ✅ Rules guide the agent (not enforce)
• ✅ smart_turn(start) is recommended entry point for non-trivial tasks
• ✅ Agent can skip workflow for trivial tasks
• ✅ You control nothing directly—t

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Arrayo
• Source: Arrayo/smart-context-mcp
• License: MIT

Install and usage instructions live in the source repository linked above.