Memkeeper Librarian

The LLM proposes. A deterministic boundary decides what gets in.

Memkeeper: Librarian

Part of the Memkeeper family.

A bounded context-curation layer that turns memkeeper search results into role-specific, budgeted context packs for AI agents. The LLM only proposes; a deterministic Rust layer enforces the boundary.

• Bounded. The selector may select, omit, summarize, and reorder *within the

candidate set the retriever returns*, and nothing else. Retrieval and scoring stay in memkeeper.

• Fail-loud. A selected id that was not among the retrieved candidates is a

FabricatedSelection error, not silently-injected context.

• Budgeted. Packs are clamped to a configurable number of candidates, items,

and characters.

• Dependency-light. The candidate score is memkeeper's, carried through

unchanged; the librarian never computes a ranking of its own.

> Status: pre-release (v0.1). The pack format and CLI may change before 1.0.

> ⚠️ Early prototype. Newer and less battle-tested than the rest of the > Memkeeper family. Expect rough edges; not recommended for production yet.

> ℹ️ Generated release mirror. This repo is generated from a private > development repo and published as releases. The main branch may be > regenerated, so pin to tagged releases (or the release artifacts) rather > than to arbitrary main commits. See [CONTRIBUTING.md](CONTRIBUTING.md) for > how to contribute; issues, security reports, and design feedback are the best > paths today.

How it works

The librarian sits between memkeeper and an agent and runs one bounded invocation per call:

1. Retrieve. It runs a retriever command (the memkeeper CLI, or any

program that prints candidates as JSON) to get a candidate set. Each candidate carries an id, text, and memkeeper's own score.

1. Select. It optionally runs an LLM selector that proposes which

candidates to keep, in what order, and with optional summaries. With no selector configured it falls back to the deterministic DefaultSelector (memkeeper rank order, no judgment).

1. Enforce + assemble. The Rust layer validates the proposal against the

candidate set (rejecting any fabricated id), applies the budget, and emits a context pack as JSON with the kept items, a recorded rationale, and the list of dropped candidates.

| Crate | Role | |---|---| | librarian-core | Candidate/pack model, the Retriever and Selector traits, the boundary + budget enforcement, and the shipped retriever/selector implementations (CommandRetriever, MemkeeperRetriever, CommandSelector, ClaudeSelector, DefaultSelector). | | claude-invoke | Bundled dependency: the shared headless claude -p primitive (flags, auth, output-envelope parse) that ClaudeSelector and the Claude synthesizer call. | | librarian-cli | The librarian binary: one librarian context invocation per call. |

Prerequisites

• Rust toolchain (stable, via rustup), edition 2021,

so Rust 1.56 or newer.

• A retriever to point at (the memkeeper CLI, or any program that prints a JSON

array of candidates). An LLM selector is optional.

Build & test

cargo build --release
cargo test --workspace

Usage

librarian context --query  --role  \
  [--max-items N] [--max-chars N] [--max-candidates N] \
  [--selector-cmd PROGRAM_AND_ARGS | --claude-selector [--selector-model M]] \
  --  [args...]

Everything after -- is the retriever command. The librarian appends the query as the final argument and reads a JSON array of candidates ([{"id": "...", "text": "...", "score": 0.0}, ...]) from its stdout.

# Deterministic pack (no LLM): keep memkeeper's order, just apply the budget.
# Here `my-retriever` is any program that prints candidates as JSON.
./target/release/librarian context \
  --query "auth flow" --role reviewer --max-items 5 \
  -- my-retriever --limit 20

# Let a headless `claude` agent propose the selection within the candidates:
./target/release/librarian context \
  --query "auth flow" --role reviewer --claude-selector \
  -- my-retriever --limit 20

# Or wire in any selector command (it receives the candidates and proposes):
./target/release/librarian context \
  --query "auth flow" --role reviewer --selector-cmd "my-selector --json" \
  -- my-retriever --limit 20

The pack is printed as JSON on stdout. Exit codes: 0 success, 1 librarian error (including a fabricated selection), 2 usage error.

Selector cost

Retrieval and scoring are local: the retriever (e.g. the memkeeper CLI) does the embedding and reranking, and DefaultSelector trusts that rank order. That path sends zero tokens to any model API. An LLM selector only earns its cost when you genuinely need judgment within the retrieved set; otherwise the local default is free.

Measured example - selecting from a 20-candidate pool (~4.4k characters), --output-format json usage, list-price equivalents:

| Selector | API tokens / call | Cost / call (Sonnet) | Cost / call (Opus) | |---|---|---|---| | DefaultSelector (local rank) | 0 | $0.00 | $0.00 | | ClaudeSelector (one-shot) | ~25.7k in / ~24 out | ~$0.06 | ~$0.31 |

Of the ClaudeSelector input, ~24k is the fixed agent-harness prefix that loads on every one-shot invocation; only ~1.4k is the marginal cost of reading the candidate pool. Your numbers will vary with pool size, model, and how much of the prefix is cache-warm. The point stands regardless: keep selection local unless an LLM's judgment changes the outcome, and the retrieve+rerank stage never costs API tokens either way.

License

Dual-licensed under either of [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE) at your option.

Contributing

Issues and pull requests are welcome. Contributions require signing the project [Contributor License Agreement](docs/CLA.md), the CLA bot prompts you on your first pull request; you keep the copyright to your contributions. See [CONTRIBUTING.md](CONTRIBUTING.md).

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: teflon07
• Source: teflon07/memkeeper-librarian
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.