Spelunk

Spelunk

Spelunk indexes codebases into local SQLite via Tree-sitter. Querying the index replaces slow file scans (cat/grep) and saves up to 90% of token budget.

Database persistence

.spelunk/data.db (or SPELUNK_DB_PATH) persists across agent sessions. Index status and metadata are durable. Re-verify index freshness only on command failure or file modifications. Do not check environment or rescan on session start.

Guidelines

🚀 Environment & Index Lifecycle

• Check Before Scanning: Run scan.mjs only when .spelunk/data.db is missing/empty or on initial workspace setup.
• Incremental Updates: Limit rescans to modified files when stderr indicates the index is out of date.

🔍 Search & Dependency Resolution

• Symbol & Dependency Lookups: Use deps.mjs to trace import chains down or up the tree instead of reading files line-by-line. Use find.mjs to locate symbol definitions.
• Fuzzy Search: Query files_fts table via query.mjs for trigram search when exact symbol lookups fail.
• Refactor Impact: Run diff.mjs after modifying imports/exports to verify downstream dependencies before committing changes.

🛡️ Safety & Security

• Read-Only SQL: query.mjs strictly permits read-only statements (SELECT, WITH, PRAGMA, EXPLAIN). Always parameterize dynamic inputs with ? placeholders.
• Credential Protection: Credential files (.env, .pem, SSH keys) are automatically excluded during scans.
• Gitignore Hygiene: Ensure .spelunk/ is present in your workspace .gitignore.

⚡ Performance & Batching

• Token Efficiency: Default to --format json for intermediate agent reasoning to save tokens. Reserve --format markdown for final user presentation.
• Batching Queries: Combine sequential SQL lookups into a single query.mjs call to avoid Node process startup overhead (~100–300ms per invocation).
• Scoping Flags: Use --limit, --file, and --query to keep returned payloads targeted.

🪜 Resolution Ladder (Stop at first matching rung)

1. Rung 1: Query existing SQLite index (.spelunk/data.db).
2. Rung 2: Run scan.mjs incrementally for modified files if index is stale.
3. Rung 3: Fall back to regex parsing (--force-fallback) if WASM grammars fail offline.
4. Rung 4: Fall back to text search (grep/cat) as last resort for single-file lookups.

⚠️ System Limitations

• File Size & Type: Files >1MB and binary files are skipped from AST parsing.
• Dynamic Constructs: Dynamic import(), reflection, and C/C++ preprocessor macros are not evaluated statically.

Script usage

Run scripts from workspace root using Node.js. Replace ` with the absolute skill path. Global flags: --format (-f), --dir , --no-download, --force-fallback`.

Quick example: node /scripts/find.mjs Router

• node /scripts/status.mjs []
• node /scripts/query.mjs "" [...]
• node /scripts/scan.mjs [] [--concurrency ] [-w|--watch]
• node /scripts/find.mjs |--query|-q [-l ] [-o ]
• node /scripts/outline.mjs |--file
• node /scripts/deps.mjs |--file |--direction [|-d ] [-l ] [-o ]
• node /scripts/explain.mjs |--file [--set-summary ""]
• node /scripts/export.mjs []
• node /scripts/diff.mjs |--file-a |--file-b

> [!IMPORTANT] > WASM grammars fetch to ~/.cache/spelunk/wasm/ on first run. Force offline mode with --no-download (SPELUNK_OFFLINE=1), force fallback with --force-fallback (SPELUNK_FORCE_FALLBACK=1), or set custom grammars via SPELUNK_WASM_DIR. Credential files (.env, .pem, SSH keys) are automatically skipped. Ensure .spelunk/ is listed in .gitignore.

Database schema

Inspect [references/database.md](./references/database.md) for full DDL, indexes, and triggers.

• files: Primary records (path, parsed, reason, hash, exports, imports, summary, summary_hash, mtime, size).
• file_imports: Resolved dependency links (file_path, imported_path).
• file_exports & file_raw_imports: Flat symbol name indexes (file_path, name).
• metadata: Workspace configuration (key, value). Known keys: rootDir, scanStatus, scanPid, lastScanTime, lastGitCommit.
• files_fts: Virtual FTS5 trigram search table auto-synced with files.

Reference payloads & SQL examples

Inspect Level 3 reference documentation for detailed contracts and schemas:

• [references/examples.md](./references/examples.md): Full JSON output payloads, error schemas, and SQL query examples.
• [references/database.md](./references/database.md): Deep-dive SQLite schema specs, index definitions, and trigram FTS configuration.
• [references/contracts.md](./references/contracts.md): Script invocation contracts and output interface definitions.
• [references/codemap.v1.json](./references/codemap.v1.json): JSON Schema (Draft-07) validation spec for codemap payloads.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: roypriyanshu02
• Source: roypriyanshu02/spelunk
• License: MIT

Install and usage instructions live in the source repository linked above.