Matryoshka

Matryoshka

[](https://github.com/yogthos/Matryoshka/actions/workflows/test.yml) [](https://safeskill.dev/scan/yogthos-matryoshka)

Process documents 100x larger than your LLM's context window—without vector databases or chunking heuristics.

The Problem

LLMs have fixed context windows. Traditional solutions (RAG, chunking) lose information or miss connections across chunks. RLM takes a different approach: the model reasons about your query and outputs symbolic commands that a logic engine executes against the document.

Based on the Recursive Language Models paper.

How It Works

Unlike traditional approaches where an LLM writes arbitrary code, RLM uses Nucleus—a constrained symbolic language based on S-expressions. The LLM outputs Nucleus commands, which are parsed, type-checked, and executed by Lattice, our logic engine.

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   User Query    │────▶│   LLM Reasons   │────▶│ Nucleus Command │
│ "total sales?"  │     │  about intent   │     │  (sum RESULTS)  │
└─────────────────┘     └─────────────────┘     └────────┬────────┘
                                                         │
┌─────────────────┐     ┌─────────────────┐     ┌────────▼────────┐
│  Final Answer   │◀────│ Lattice Engine  │◀────│     Parser      │
│   13,000,000    │     │    Executes     │     │    Validates    │
└─────────────────┘     └─────────────────┘     └─────────────────┘

Why this works better than code generation:

1. Reduced entropy - Nucleus has a rigid grammar with fewer valid outputs than JavaScript
2. Fail-fast validation - Parser rejects malformed commands before execution
3. Safe execution - Lattice only executes known operations, no arbitrary code
4. Small model friendly - 7B models handle symbolic grammars better than freeform code

Architecture

The Nucleus DSL

The LLM outputs commands in the Nucleus DSL—an S-expression language designed for document analysis:

; Search for patterns
(grep "ERROR")

; Filter results
(filter RESULTS (lambda x (match x "timeout" 0)))

; Aggregate
(sum RESULTS)    ; Auto-extracts numbers from lines
(count RESULTS)  ; Count matching items

; Final answer
>>13000000>>

Feature availability by execution path

Matryoshka has two execution paths and not every primitive works in both:

| Feature | runRLM (CLI / programmatic) | lattice-mcp (MCP server) | |---|---|---| | (grep …), (filter …), (map …), etc. | ✅ | ✅ | | (llm_query …), (llm_batch …) | ✅ | ✅ via MCP sampling protocol | | (rlm_query …), (rlm_batch …) | ✅ (concurrent rlm_batch) | ✅ — child Nucleus session spawns via the same MCP sampling bridge; M suspensions per rlm_query call. rlm_batch runs sequentially (children one at a time) because the multi-turn suspension protocol only carries one pending request at a time — concurrent children would lose suspensions. Round-trip count is the same; wall-clock is N×slower for non-sampling clients. | | (context N) selector | ✅ (multi-doc via runRLMFromContent(query, string[])) | partial — (context 0) works; multi-doc loading not exposed via lattice_load | | (grep "X" haystack) | ✅ | ✅ | | (show_vars) | ✅ | ✅ (internal _ bindings filtered out) | | FINAL_VAR(name) resolution | ✅ | N/A — MCP returns query results directly | | maxTimeoutMs / maxChars / maxErrors | ✅ | ❌ — MCP has its own session timeout | | compactionThresholdChars | ✅ | ❌ — MCP doesn't have a multi-turn FSM history |

The resource-limit features remain runRLM-only. The recursive primitives (rlm_query/rlm_batch) work in both paths — the MCP path spawns a child runRLMFromContent whose llmClient is the same sampling bridge as the parent, so each child turn flows through the existing MCP suspension/sampling protocol.

Recursive Primitives

rlm_query spawns a child Nucleus session with its own FSM loop. The child runs to FINAL and returns a string — useful when a sub-task needs multi-turn reasoning over a structured handle:

; Child sees the resolved handle as its working document, NOT a
; JSON-stringified prompt blob. Lets the child use grep/lines/
; chunk_by_lines over arrays without JSON-syntax noise.
(rlm_query "extract dates" (context RESULTS))

; No (context …) → child's document is the prompt itself.
(rlm_query "summarize each error type")

rlm_batch runs the same per-item recursion across a collection. Each item produces one entry in the returned array, in input order. Per-item failures surface as "Error: rlm_batch item N failed — …" strings without aborting the rest of the batch:

(rlm_batch (chunk_by_lines 100)
  (lambda c (rlm_query "extract metrics" (context c))))

• runRLM: children fan out concurrently via a worker pool capped at maxConcurrentSubcalls (default 4).
• lattice-mcp: children run sequentially because the multi-turn suspension protocol can carry only one pending request at a time. Round-trip count is identical to the concurrent path (N children × M turns each); only wall-clock differs.

Multi-Context Loading

Pass string[] to runRLMFromContent to load multiple documents. Address them via (context N); index 0 is the default for primitives that don't specify a haystack:

(grep "DEPLOY" (context 0))   ; deploy.log
(grep "OUTAGE" (context 2))   ; comms.log

; (context N) is just a term — pipe it anywhere a string is expected
(rlm_query "scan" (context (context 1)))   ; child sees doc 1

Per-doc line numbers come back, so the LLM can cite "doc 0 line 4, doc 2 line 2" with confidence rather than inventing absolute offsets across a concatenation.

Introspection

(show_vars)   ; Returns a string summary of every binding currently
              ; in scope. Useful before a (filter RESULTS …) or a
              ; FINAL_VAR(name) reference when the LLM lost track of
              ; what's bound. Same surface as the `lattice_bindings`
              ; MCP tool but reachable from inside a query.

Unknown FINAL_VAR markers surface a clear error rather than passing the literal text through:

>>FINAL_VAR(_99)>>
→ "[FINAL_VAR error: unknown binding "_99". Available: _1, RESULTS]"

Resource Limits

All optional. With none set, behavior is unchanged:

runRLM(query, file, {
  maxTimeoutMs: 30_000,    // wall-clock cap, propagates to children
  maxChars: 100_000,       // cumulative chars sent + received
  maxErrors: 5,            // consecutive parse/execution errors
  compactionThresholdChars: 50_000,  // summarize history when prompt grows past this
})

When a limit hits, the run terminates cleanly with a string of the form:

[aborted: timeout 32100ms of 30000ms]

Best partial answer:

The partial answer is always preserved when present — completed work is never silently lost on abort.

The Lattice Engine

The Lattice engine (src/logic/) processes Nucleus commands:

1. Parser (lc-parser.ts) - Parses S-expressions into an AST
2. Type Inference (type-inference.ts) - Validates types before execution
3. Constraint Resolver (constraint-resolver.ts) - Handles symbolic constraints like [Σ⚡μ]
4. Solver (lc-solver.ts) - Executes commands against the document

Lattice uses miniKanren (a relational programming engine) for pattern classification and filtering operations.

In-Memory Handle Storage

For large result sets, RLM uses a handle-based architecture with in-memory SQLite (src/persistence/) that achieves 97%+ token savings:

Traditional:  LLM sees full array    [15,000 tokens for 1000 results]
Handle-based: LLM sees stub          [50 tokens: "$grep_error: Array(1000) [preview...]"]

How it works:

1. Results are stored in SQLite with FTS5 full-text indexing
2. LLM receives descriptive handle references derived from the command (e.g., $grep_error, $bm25_timeout, $filter_status)
3. Operations execute server-side, returning new handles
4. Full data is only materialized when needed

Handle names are auto-generated from the Nucleus command: (grep "ERROR") produces $grep_error, (list_symbols "function") produces $list_symbols_function. Repeated commands get a numeric suffix ($grep_error_2, $grep_error_3).

Memory Pad

The Lattice engine doubles as a context memory for LLM agents. Instead of roundtripping large text blobs in every message, agents stash context server-side and carry only compact handle stubs:

Agent reads file, summarizes → lattice_memo "auth architecture"
                              → $memo_auth_architecture: "auth architecture" (2.1KB, 50 lines)

20 messages later, needs it  → lattice_expand $memo_auth_architecture
                              → Full 50-line summary

Token math (30-message session, 3 source files stashed):

• Traditional roundtripping: 836K tokens
• Memo-based (stubs + 6 expands): 57K tokens — 93% savings

Memos persist across document loads (lattice_load clears query handles but keeps memos), support LRU eviction (100 memo cap, 10MB budget), and can be explicitly deleted when stale. No document needs to be loaded to use memos.

The Role of the LLM

The LLM does reasoning, not code generation:

1. Understands intent - Interprets "total of north sales" as needing grep + filter + sum
2. Chooses operations - Decides which Nucleus commands achieve the goal
3. Verifies results - Checks if the current results answer the query
4. Iterates - Refines search if results are too broad or narrow

The LLM never writes JavaScript. It outputs Nucleus commands that Lattice executes safely.

Installation

Install from npm:

pnpm add -g matryoshka-rlm

Or run without installing:

npx matryoshka-rlm "How many ERROR entries are there?" ./server.log

Included Tools

The package provides several CLI tools:

| Command | Description | |---------|-------------| | rlm | Main CLI for document analysis with LLM reasoning | | rlm-mcp | MCP server with full RLM + LLM orchestration (analyze_document tool) | | lattice-mcp | MCP server exposing direct Nucleus commands (no LLM required) | | lattice-repl | Interactive REPL for Nucleus commands | | lattice-http | HTTP server for Nucleus queries | | lattice-pipe | Pipe adapter for programmatic access | | lattice-setup | Setup script for Claude Code integration |

From Source

git clone https://github.com/yogthos/Matryoshka.git
cd Matryoshka
pnpm install
pnpm run build

Configuration

Copy config.example.json to ~/.config/matryoshka/config.json:

{
  "llm": {
    "provider": "ollama"
  },
  "providers": {
    "ollama": {
      "url": "http://localhost:11434/api/generate",
      "model": "qwen3-coder:30b",
      "options": { "temperature": 0.2, "num_ctx": 8192 }
    },
    "deepseek": {
      "url": "https://api.deepseek.com/chat/completions",
      "apiKey": "${DEEPSEEK_API_KEY}",
      "model": "deepseek-chat",
      "options": { "temperature": 0.2 }
    },
    "glm": {
      "url": "https://open.bigmodel.cn/api/paas/v4/chat/completions",
      "apiKey": "${ZHIPU_API_KEY}",
      "model": "glm-4-plus",
      "options": { "temperature": 0.2 }
    }
  },
  "rlm": {
    "maxTurns": 10
  },
  "grammars": {
    "ocaml": {
      "package": "tree-sitter-ocaml",
      "extensions": [".ml", ".mli"],
      "moduleExport": "ocaml",
      "symbols": {
        "value_definition": "function",
        "type_definition": "type",
        "module_definition": "module"
      }
    }
  }
}

• llm / providers / rlm — LLM provider selection and RLM tuning (shown above with example providers). Each provider takes a full url (the complete API endpoint), an optional apiKey (supports ${ENV_VAR} interpolation), a model name, and options.
• grammars — custom tree-sitter language mappings for symbol extraction (see [Adding Language Support](#adding-language-support) for the full list of built-in languages). Use the tree-sitter playground to explore node types for your language.

Usage

CLI

# Basic usage
rlm "How many ERROR entries are there?" ./server.log

# With options
rlm "Count all ERROR entries" ./server.log --max-turns 15 --verbose

# See all options
rlm --help

MCP Integration

RLM includes lattice-mcp, an MCP (Model Context Protocol) server for direct access to the Nucleus engine. This allows coding agents to analyze documents with 80%+ token savings compared to reading files directly.

The key advantage is handle-based results: query results are stored server-side in SQLite, and the agent receives compact stubs like $grep_error: Array(1000) [preview...] instead of full data. Handle names are derived from the command for easy identification. Operations chain server-side without roundtripping data.

Available Tools

| Tool | Description | |------|-------------| | lattice_load | Load a document for analysis | | lattice_query | Execute Nucleus commands on the loaded document | | lattice_expand | Expand a handle to see full data (with optional limit/offset) | | lattice_memo | Store arbitrary context as a memo handle (no document required) | | lattice_memo_delete | Delete a stale memo to free memory | | lattice_close | Close the session and free memory | | lattice_status | Get session status, document info, and memo usage | | lattice_bindings | Show current variable bindings and memo labels | | lattice_reset | Reset all bindings and memos but keep document loaded | | lattice_llm_respond | Respond to a pending (llm_query ...) suspension | | lattice_llm_batch_respond | Respond to a pending (llm_batch ...) suspension with all N responses | | lattice_help | Get Nucleus command reference |

Example MCP config

{
  "mcp": {
    "lattice": {
      "type": "stdio",
      "command": "lattice-mcp"
    }
  }
}

Efficient Usage Pattern

1. lattice_load("/path/to/large-file.txt")   # Load document (use for >500 lines)
2. lattice_query('(grep "ERROR")')           # Search → $grep_error: Array(500) [preview]
3. lattice_query('(filter RESULTS ...)')     # Narrow → $filter_timeout: Array(50) [preview]
4. lattice_query('(count RESULTS)')          # Count without seeing data → 50
5. lattice_expand("$filter_timeout", limit=10) # Expand only what you need to see
6. lattice_close()                           # Free memory when done

Token efficiency tips:

• Query results return descriptive handle stubs, not full data
• Use lattice_expand with limit to see only what you need
• Chain grep → filter → count/sum to refine progressively
• Use RESULTS in queries (always points to last result)
• Use descriptive handle names (e.g., $grep_error) with lattice_expand to inspect specific results

Chunking and Sub-LLM Recursion

Two primitive families power the paper's Ω(|P|²) semantic-horizon pattern:

Chunking — pre-slice a document that's too big to map over directly:

(chunk_by_size 2000)                ; 2000-character slices
(chunk_by_lines 100)                ; 100-line slices
(chunk_by_regex "\\n\\n")           ; Split on blank lines; capture groups ignored

Sub-LLM calls — (llm_query ...) invokes a sub-LLM with an interpolated prompt. Works at the top level and nested inside map / filter / reduce lambdas:

(llm_query "Summarize this")                                         ; bare
(llm_query "Classify: {items}" (items RESULTS))                      ; with binding
(map (chunk_by_lines 100)
     (lambda c (llm_query "summarize: {chunk}" (chunk c))))           ; OOLONG
(filter RESULTS (lambda x (match (llm_query "keep?: {item}" (item x)) "keep" 0)))

The last two patt

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: yogthos
• Source: yogthos/Matryoshka
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.