# Memorystress

> The first longitudinal benchmark for AI memory systems. Test LLM agents, RAG pipelines, MCP servers, and memory backends across 1,000 sessions.

- **Type:** MCP server
- **Install:** `agentstack add mcp-omega-memory-memorystress`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [omega-memory](https://agentstack.voostack.com/s/omega-memory)
- **Installs:** 0
- **Category:** [Databases](https://agentstack.voostack.com/c/databases)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [omega-memory](https://github.com/omega-memory)
- **Source:** https://github.com/omega-memory/memorystress
- **Website:** https://huggingface.co/datasets/singularityjason/memorystress

## Install

```sh
agentstack add mcp-omega-memory-memorystress
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# MemoryStress

**The first longitudinal benchmark for AI memory systems.**

Test any memory system — [OMEGA](https://github.com/omega-memory/core), [Mem0](https://github.com/mem0ai/mem0), [Zep](https://github.com/getzep/zep), [LangMem](https://github.com/langchain-ai/langmem), or your own — against 1,000 sessions of realistic agent conversations. Works with OpenAI, Anthropic Claude, Google Gemini, or any LLM backend.

Every existing memory benchmark tests recall from a handful of sessions. MemoryStress tests what happens at session 1,000 — when facts contradict, memories fade, and noise accumulates over 10 simulated months.

```
583 facts  ×  1,000 sessions  ×  300 questions  ×  6 scoring dimensions
```

## Why MemoryStress?

Current benchmarks don't test what breaks in production:

| Benchmark | Sessions | Contradictions | Degradation Curve | Multi-Agent |
|-----------|:--------:|:--------------:|:-----------------:|:-----------:|
| LongMemEval | ~40 | No | No | No |
| MemoryAgentBench | Short | No | No | No |
| BEAM | Synthetic | No | No | No |
| **MemoryStress** | **1,000** | **Yes (40 chains)** | **Yes (4 checkpoints)** | **Yes** |

Real agents — Claude Code, ChatGPT with memory, custom OpenAI Assistants, LangChain agents — run for months. Their memory systems face:
- **Accumulation pressure**: 1,000 sessions of growing noise
- **Contradiction resolution**: Facts that change, revert, and partially update
- **Degradation over time**: Can you still find a fact from session 12 after ingesting session 997?
- **Cross-agent recall**: Information shared across agent identities
- **Cold start recovery**: Fresh agent instance accessing an existing memory store

MemoryStress measures all of these.

## Use Cases

- **Evaluate production memory systems** before deploying long-running AI agents
- **Compare RAG approaches**: vector databases, graph memory, hybrid retrieval systems
- **Benchmark MCP memory servers** for Claude Code, Claude Desktop, Cursor, Cline, Windsurf
- **Test OpenAI Assistants API** thread persistence and memory retention
- **Validate custom memory architectures** against a standardized dataset
- **Research memory degradation** in conversational AI systems over time

## Leaderboard

| System | Model | Score | Contradiction | Degradation Slope | Cost/Correct | Details |
|--------|-------|:-----:|:-------------:|:-----------------:|:------------:|---------|
| **OMEGA v1.0** | GPT-4o | **32.7%** (98/300) | 21.4% | -0.033 | $0.041 | [results](results/omega_v5/) |
| Raw Context | GPT-4o | TBD | TBD | TBD | TBD | — |
| Mem0 | GPT-4o | TBD | TBD | TBD | TBD | — |
| Zep | — | TBD | TBD | TBD | TBD | — |
| LangMem | — | TBD | TBD | TBD | TBD | — |
| ChatGPT Memory | — | TBD | TBD | TBD | TBD | — |
| Null (baseline) | — | 0% | 0% | 0.000 | — | [results](results/null/) |

> **32.7% is a strong result on an intentionally brutal benchmark.** The null baseline scores 0%. A context-window approach hits its token ceiling around session 200. This benchmark asks about single-mention facts buried in noisy conversations from hundreds of sessions ago.

**Submit your results**: Run the benchmark, open a PR with your `results/` directory.

## Quick Start

```bash
# Install
pip install memorystress

# Download dataset
# Option A: From HuggingFace
pip install huggingface_hub
huggingface-cli download singularityjason/memorystress --local-dir data/

# Option B: Generate your own (~$5 in API costs)
python scripts/generate.py --model gpt-4o --seed 42 --output data/memorystress_v1.json

# Run the null baseline (free, instant)
python scripts/run.py --dataset data/memorystress_v1.json --adapter null --grade

# Run with OMEGA (requires: pip install omega-memory)
python scripts/run.py --dataset data/memorystress_v1.json --adapter omega --grade \
  --extract-facts --query-augment --output-dir results/omega_v5

# Run with Mem0 (requires: pip install mem0ai)
python scripts/run.py --dataset data/memorystress_v1.json --adapter mem0 --grade \
  --output-dir results/mem0
```

## How It Works

### 1. Dataset Generation

A GPT-4o pipeline generates:
- **583 facts** across 6 categories (preferences, decisions, technical facts, personal info, events, relationships)
- **1,000 conversation sessions** across 3 phases of increasing noise
- **40 contradiction chains** where facts update, revert, accumulate, or partially change
- **300 questions** at 4 phase checkpoints, spanning 7 question types

### 2. Three Phases of Pressure

```
Phase 1 (Sessions 1-100)     Low noise, establishing baseline
Phase 2 (Sessions 101-500)   Growing noise, contradictions emerge
Phase 3 (Sessions 501-1000)  Dense, high-entropy, multi-topic stress
Phase 4 (Recovery)           No new sessions — can you still recall?
```

### 3. Six Scoring Dimensions

| Dimension | What It Measures |
|-----------|-----------------|
| **Recall@Age** | Accuracy by fact age (0-100, 100-500, 500-1000 sessions) |
| **Degradation Curve** | Accuracy at each phase checkpoint (slope = degradation rate) |
| **Contradiction Resolution** | Can the system identify the most recent value? |
| **Cost Efficiency** | Dollars per correct answer at each phase |
| **Cross-Agent Recall** | Information recall across agent identities |
| **Cold Start Recovery** | Fresh agent accessing an existing memory store |

### 4. Seven Question Types

- Fact recall, Preference recall, Contradiction resolution, Temporal ordering, Cross-agent recall, Cold start recall, Single-mention recall

## Add Your Memory System

Built an MCP memory server? Using OpenAI's Assistants API with threads? Running Mem0, Zep, LangMem, or a custom RAG pipeline? Test it here.

Implement 4 methods:

```python
from memorystress.adapters.base import MemorySystemAdapter, IngestResult, QueryResult, CostSnapshot

class MyAdapter(MemorySystemAdapter):
    def ingest(self, session: dict) -> IngestResult:
        """Store a conversation session."""
        # session has: session_id, turns, simulated_date, agent_id, phase
        ...

    def query(self, question: dict) -> QueryResult:
        """Answer a question from memory."""
        # question has: question, agent_scope, question_type
        ...

    def reset(self) -> None:
        """Clear all stored data."""
        ...

    def get_cost(self) -> CostSnapshot:
        """Return cumulative token/cost accounting."""
        ...
```

Then add it to `scripts/run.py`'s `create_adapter()` factory and run the benchmark.

## OMEGA's Results: Deep Dive

### Degradation Curve

```
Phase 1 (100 sessions):   31.4%  ██████
Phase 2 (500 sessions):   42.4%  ████████    ← peak: more data helps retrieval
Phase 3 (1000 sessions):  27.3%  █████       ← noise dilution, not data loss
Phase 4 (recovery):       25.5%  █████
```

### Per-Type Breakdown

| Question Type | Score | N |
|---------------|:-----:|:-:|
| Temporal ordering | 41.2% | 34 |
| Fact recall | 37.5% | 80 |
| Cold start recall | 37.5% | 16 |
| Preference recall | 37.1% | 35 |
| Cross-agent recall | 31.2% | 32 |
| Single-mention recall | 27.7% | 47 |
| Contradiction resolution | 21.4% | 56 |

### Recall by Fact Age

| Age | Score |
|-----|:-----:|
| 0-100 sessions | 33.8% |
| 100-500 sessions | 39.7% |
| 500-1000 sessions | 21.3% |

Mid-range facts (100-500 sessions old) are easiest — enough time to be reinforced, not so old that noise drowns them.

## Comparison with Other Benchmarks

| Feature | MemoryStress | LongMemEval | MemoryAgentBench | BEAM |
|---------|:------------:|:-----------:|:----------------:|:----:|
| Session count | 1,000 | ~40 | Short | Synthetic |
| Degradation measurement | Yes | No | No | No |
| Contradiction chains | 40 | No | No | No |
| Multi-agent | Yes | No | No | No |
| Cold start testing | Yes | No | No | No |
| Cost tracking | Yes | No | No | No |
| Temporal questions | Yes | Yes | No | No |
| Question types | 7 | 4 | Task-based | 3 |

## Dataset

The full dataset is available on HuggingFace: [singularityjason/memorystress](https://huggingface.co/datasets/singularityjason/memorystress)

See [data/README.md](data/README.md) for format details and generation instructions.

## Citation

```bibtex
@software{memorystress2026,
  title={MemoryStress: The First Longitudinal Benchmark for AI Memory Systems},
  author={OMEGA Memory},
  url={https://github.com/omega-memory/memorystress},
  year={2026}
}
```

## License

Apache-2.0. See [LICENSE](LICENSE).

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [omega-memory](https://github.com/omega-memory)
- **Source:** [omega-memory/memorystress](https://github.com/omega-memory/memorystress)
- **License:** Apache-2.0
- **Homepage:** https://huggingface.co/datasets/singularityjason/memorystress

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-omega-memory-memorystress
- Seller: https://agentstack.voostack.com/s/omega-memory
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.