LycheeMem

LycheeMemory: Lightweight Long-Term Memory for LLM Agents

中文 | English

Works across agent runtimes that support plugins, MCP, or Python integration.

OpenClaw

Native plugin

Claude Code

MCP + hooks

Hermes

Runtime plugin

PyPI Package

Python API

Any MCP Client

HTTP MCP server

LycheeMemory is a compact memory framework for LLM agents. It starts from efficient conversational memory—through structured organization, lightweight consolidation, and adaptive retrieval—and gradually extends toward action-aware, usage-aware memory for more capable agentic systems.

---

News • Related Projects • Quick Start • Web Demo • OpenClaw Plugin • MCP • Memory Architecture • Pipeline • API Reference

---

🔥 News

• [05/08/2026] Transformer memory reranker v0 improves evidence selection in semantic memory search, with positive hit@10 gains on LoCoMo and zero-shot LongMemEval-S / MSC-MemFuse / HotpotQA fixtures. See [Transformer Reranker v0](docs/transformerrerankerv0.md).
• [04/29/2026] Hermes and Claude Code plugin integrations are now available, bringing LycheeMemory's automatic recall, turn mirroring, and consolidation workflow to more agent runtimes. Setup guides: [Hermes](hermes-plugin/lycheemem/INSTALLHERMES.md) · [Claude Code](claude-plugin/lycheemem/INSTALLCLAUDE.md)
• [04/26/2026] Visual (Multimodal) Memory module added! See [Visual Memory](#visual-memory).
• [04/13/2026] LycheeMem is now LycheeMemory.
• [04/03/2026] The project now supports installation via pip install lycheemem. You can easily start the service from anywhere using lycheemem-cli!
• [03/30/2026] We evaluated LycheeMemory on PinchBench with the OpenClaw plugin: compared to OpenClaw's native memory, it achieved an ~6% score improvement, while reducing token consumption by ~71% and cost by ~55%!
• [03/28/2026] Semantic memory has been upgraded to Compact Semantic Memory (SQLite + LanceDB), no Neo4j required. See [/quick-start](#quick-start) for details.
• [03/27/2026] OpenClaw Plugin is now available at [/openclaw-plugin](#openclaw-plugin) ! [Setup guide →](openclaw-plugin/INSTALL_OPENCLAW.md)
• [03/26/2026] MCP support is available at [/mcp](#mcp) !
• [03/23/2026] LycheeMemory is now open source: GitHub Repository →

---

🔗 Related Projects

LycheeMemory is part of the 3rd-generation Lychee (立知) large model series, which focuses on memory intelligence, continual learning, and long-context reasoning.

We welcome you to explore our related works:

• LycheeMemory (ACL 2026, CCF-A): a unified framework for implicit long-term memory and explicit working memory collaboration in large language models

[](https://arxiv.org/abs/2602.08382) [](https://github.com/owoakuma/LycheeMemory) [](https://huggingface.co/lerverson/LycheeMemory-7B)

• LycheeMem (this project): long-term memory infrastructure for LLM-based agents

[](https://lycheemem.github.io) [](https://github.com/LycheeMem/LycheeMem)

• LycheeDecode (ICLR 2026, CCF-A): selective recall from massive KV-cache context memory

[](https://lg9077.github.io/lycheedecode) [](https://arxiv.org/abs/2602.04541) [](https://github.com/HITsz-TMG/TMGNLP/tree/main/LycheeDecode)

• LycheeCluster (ACL 2026, CCF-A): structured organization and hierarchical indexing for context memory

[](https://arxiv.org/abs/2603.08453)

---

⚡ Quick Start

Prerequisites

• Python 3.9+
• An LLM API key (OpenAI, Gemini, or any litellm-compatible provider)

Installation

Install the core package:

pip install lycheemem

Recommended install with the default transformer memory reranker:

pip install "lycheemem[rerank]"

The rerank extra adds PyTorch / Transformers runtime dependencies. With it installed, LycheeMemory enables the hosted LycheeMem/reranker checkpoint by default. Without the extra, the core memory system still works and reranking falls back safely.

Once installed, you can start the backend server instantly using the CLI:

lycheemem-cli

For development or if you prefer to run from source:

git clone https://github.com/LycheeMem/LycheeMem.git
cd LycheeMem
pip install -e .

Configuration

Create a .env file in your working directory and fill in your values. The full template in .env.example also includes session/user DB paths, JWT settings, and working-memory thresholds; the snippet below shows the most important ones:

# LLM — litellm format: provider/model
LLM_MODEL=openai/gpt-4o-mini
LLM_API_KEY=sk-...
LLM_API_BASE=                     # optional

# Embedder
EMBEDDING_MODEL=openai/text-embedding-3-small
EMBEDDING_DIM=1536
EMBEDDING_API_KEY=                # optional
EMBEDDING_API_BASE=               # optional

> Supported LLM providers (via litellm): > openai/gpt-4o-mini · gemini/gemini-2.0-flash · ollama_chat/qwen2.5 · any OpenAI-compatible endpoint

Transformer Reranker

LycheeMemory includes a transformer reranker for semantic memory search. It can improve evidence selection when the correct memory is already in the wider candidate pool.

For the smoothest experience, install LycheeMemory with the rerank extra:

pip install "lycheemem[rerank]"

After that, no extra model command is required. The reranker is enabled by default and loads the current v0 checkpoint from Hugging Face on first use:

EXPERIMENTAL_TRANSFORMER_RERANK=true
TRANSFORMER_RERANK_MODEL_PATH=LycheeMem/reranker

To disable it explicitly:

EXPERIMENTAL_TRANSFORMER_RERANK=false

If you prefer to pin the model to a local directory, download it once and point the same variable at that path:

mkdir -p ~/.cache/lycheemem/models
huggingface-cli download LycheeMem/reranker \
  --local-dir ~/.cache/lycheemem/models/reranker-v0
export TRANSFORMER_RERANK_MODEL_PATH=~/.cache/lycheemem/models/reranker-v0

The base install still works without PyTorch or Transformers. If rerank dependencies or the checkpoint are unavailable, LycheeMemory logs a warning, disables reranking for that process, and continues with baseline memory search. See [Transformer Reranker v0](docs/transformerrerankerv0.md) for metrics, limitations, and diagnostics.

Start the Server

If you installed via pip, you can start the LycheeMemory background service from anywhere using:

lycheemem-cli

(If running from source, you can also use python main.py to start the server.)

The API is served at http://localhost:8000. Interactive docs at /docs.

> main.py currently starts Uvicorn without enabling live reload. For development reload, run Uvicorn directly, for example: > > ``bash > uvicorn src.api.server:create_app --factory --reload > ``

---

🎨 Web Demo

A frontend demo is included under web-demo/. It provides a chat interface alongside live views of the semantic memory tree, skill library, and working memory state.

cd web-demo
npm install
npm run dev      # served at http://localhost:5173

> Make sure the backend is running on port 8000 (or update proxy settings in web-demo/vite.config.ts) before starting the frontend.

---

🦞 OpenClaw Plugin

LycheeMemory ships a native OpenClaw plugin that gives any OpenClaw session persistent long-term memory with zero manual wiring.

What the plugin provides:

• lychee_memory_smart_search — default long-term memory retrieval entry point
• Automatic turn mirroring via hooks — the model does not need to call append_turn manually
• User messages are appended automatically
• Assistant messages are appended automatically
• /new, /reset, /stop, and session_end automatically trigger boundary consolidation
• Proactive consolidation on strong long-term knowledge signals

Under normal operation:

• The model only calls lychee_memory_smart_search when recalling long-term context
• The model may call lychee_memory_consolidate manually when an immediate persist is warranted
• The model does not need to call lychee_memory_append_turn at all

Quick Install

openclaw plugins install "/path/to/LycheeMem/openclaw-plugin"
openclaw gateway restart

See the full setup guide: [openclaw-plugin/INSTALLOPENCLAW.md](openclaw-plugin/INSTALLOPENCLAW.md)

---

🔧 MCP

LycheeMemory also exposes an HTTP MCP endpoint at http://localhost:8000/mcp.

• Available tools: lychee_memory_smart_search, lychee_memory_search, lychee_memory_append_turn, lychee_memory_synthesize, lychee_memory_consolidate
• lychee_memory_consolidate works for sessions that already contain mirrored turns from /chat, /memory/reason, or lychee_memory_append_turn

MCP Transport

• POST /mcp handles JSON-RPC requests
• GET /mcp exposes the SSE stream used by some MCP clients
• The server returns Mcp-Session-Id during initialize; reuse that header on later requests

Client Configuration

For any MCP client that supports remote HTTP servers, configure the MCP URL as:

http://localhost:8000/mcp

Generic config example:

{
  "mcpServers": {
    "lycheemem": {
      "url": "http://localhost:8000/mcp"
    }
  }
}

Manual JSON-RPC Flow

1. Call initialize
2. Reuse the returned Mcp-Session-Id
3. Send initialized
4. Call tools/list
5. Call tools/call

Initialize example:

curl -i -X POST http://localhost:8000/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "initialize",
    "params": {
      "protocolVersion": "2025-03-26",
      "capabilities": {},
      "clientInfo": {
        "name": "debug-client",
        "version": "0.1.0"
      }
    }
  }'

Tool call example:

curl -X POST http://localhost:8000/mcp \
  -H "Content-Type: application/json" \
  -H "Mcp-Session-Id: " \
  -d '{
    "jsonrpc": "2.0",
    "id": 2,
    "method": "tools/call",
    "params": {
      "name": "lychee_memory_smart_search",
      "arguments": {
        "query": "what tools do I use for database backups",
        "top_k": 5,
        "mode": "compact",
        "include_graph": true,
        "include_skills": true
      }
    }
  }'

Recommended MCP Usage Pattern

1. Use /chat or /memory/reason with a stable session_id to write conversation turns, or mirror external host turns with lychee_memory_append_turn.
2. Use lychee_memory_smart_search in compact mode for the default one-shot recall path.
3. Use lychee_memory_search + lychee_memory_synthesize only when you explicitly want search and synthesis as separate stages.
4. After the conversation ends, call lychee_memory_consolidate with the same session_id.

---

📚 Memory Architecture

LycheeMemory organizes memory into three complementary stores:

Working Memory Semantic Memory Procedural Memory Visual Memory

(Episodic)

Session turns Summaries Token budget management

(Typed Action Store)

7 MemoryRecord types Conflict-aware Record Fusion Hierarchical memory tree Action-aware hierarchical retrieval Usage feedback loop + RL-ready statistics

(Skills)

Skill entries HyDE retrieval

(Multimodal)

VLM-driven image understanding Dual embedding (caption + CLIP visual) Text ↔ image cross-modal retrieval Ebbinghaus forgetting curve Three-layer storage (SQLite + LanceDB + filesystem)

💾 Working Memory

The working memory window holds the active conversation context for a session. It operates under a dual-threshold token budget:

• Warn threshold (70%) — triggers asynchronous background pre-compression; the current request is not blocked.
• Block threshold (90%) — the pipeline pauses and flushes older turns to a compressed summary before proceeding.

Compression produces summary anchors (past context, distilled) + raw recent turns (last N turns, verbatim). Both are passed downstream as the conversation history.

🗺️ Semantic Memory

Semantic memory is organised around typed MemoryRecords plus action-grounded retrieval state. The storage layer is SQLite (FTS5 full-text search) + LanceDB (vector index), while retrieval is conditioned on recent context, tentative action, constraints, and missing slots.

Memory Record Types

Each memory entry is stored as a MemoryRecord. The memory_type field distinguishes seven semantic categories:

| Type | Description | |------|-------------| | fact | Objective facts about the user, environment, or world | | preference | User preferences (style, habits, likes/dislikes) | | event | Specific events that have occurred | | constraint | Conditions that must be respected | | procedure | Reusable step-by-step procedures / methods | | failure_pattern | Previously failed action paths and their causes | | tool_affordance | Capabilities and applicable scenarios of tools/APIs |

Beyond text, every MemoryRecord carries action-facing metadata (tool_tags, constraint_tags, failure_tags, affordance_tags) and usage statistics (retrieval_count, action_success_count, etc.) to seed future reinforcement-learning signals. Retrieval logs also persist retrieval_plan, action_state, response excerpts, and later user feedback so the system can close a lightweight action-outcome loop without training.

Related MemoryRecords can be fused online by the Record Fusion Engine into denser CompositeRecords. Composite entries persist direct child_composite_ids, so long-term semantic memory is organised as a hierarchical memory tree instead of a flat bag of summaries.

Four-Module Pipeline

Module 1: Compact Semantic Encoding

A single-pass pipeline that converts conversation turns into a list of MemoryRecords:

1. Typed extraction — LLM extracts self-contained facts and assigns a semantic category to each record.
2. Decontextualization — Pronouns and context-dependent phrases are expanded into full expressions, so each record is understandable without the original dialogue.
3. Action metadata annotation — LLM annotates each record with memory_type, tool_tags, constraint_tags, failure_tags, affordance_tags, and other structured labels.

record_id = SHA256(normalized_text) — naturally idempotent; duplicate content is deduplicated automatically.

Module 2: Record Fusion, Conflict Update, and Hierarchical Consolidation

Triggered online after each consolidation. No LLM calls — pure embedding cosine similarity math:

1. Deduplication — For each new record, ANN search finds existing records of the same memory_type with cosine similarity > 0.85. Near-duplicates are soft-expired; composites covering affected source records are invalidated.
2. Clustering — ANN search builds a similarity graph (cosine > 0.75) over surviving records. Union-Find finds connected components; each component containing at least one new record becomes a candidate cluster.
3. Composite construction — The representative record (highest confidence / most recent) provides semantic_text;

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: LycheeMem
• Source: LycheeMem/LycheeMem
• License: Apache-2.0
• Homepage: https://lycheemem.github.io

Install and usage instructions live in the source repository linked above.