Mcp Replay

mcp-replay

-brightgreen)

Record real MCP (Model Context Protocol) traffic once, then replay it against a new build of your server to catch behavioral regressions before they ship.

Every other quality check on an MCP server tests it with inputs you generate: schema linters read the tool surface, fuzzers throw malformed payloads, load harnesses measure latency. None of them tell you whether your server still behaves the way it did last week. mcp-replay captures what your server actually returned in a real session and locks it in as a contract, so a change in behaviour shows up as a failing check even when the tool schema is untouched.

Stdlib only. No runtime dependencies. Python 3.9+.

Why this exists

A static surface diff catches a renamed tool or a widened schema. It cannot catch a server that quietly starts returning the wrong number, drops a field from a result, or flips a success into an error while keeping the same schema. That is exactly the class of regression mcp-replay is built for.

static surface        ->  mcp-surface-diff      (did the schema change?)
valid-input contract  ->  mcp-conformance-kit   (does it follow the spec?)
adversarial input     ->  mcp-fuzz              (does bad input crash it?)
performance           ->  mcp-load-lab          (is it fast enough?)
context cost          ->  mcp-context-budget    (is the surface cheap?)
behaviour over time   ->  mcp-replay            (does it still do the same thing?)

Install

pip install mcp-replay        # once published
# or, from a clone:
pip install -e .

You can also run it without installing: python -m mcp_replay ....

30-second demo

mcp-replay demo

This records a session against a bundled sample server, then replays it against a healthy build and against a regressed twin whose tool surface is identical but whose behaviour has drifted:

== Replay against the HEALTHY server (expect grade A, no regressions) ==
  [PASS] #0 initialize
  [PASS] #2 tools/list
  [PASS] #3 tools/call
  [PASS] #4 tools/call
  [PASS] #5 tools/call
  GRADE A  (100.0/100)

== Replay against the REGRESSED server (same tool surface, changed behaviour) ==
  [PASS] #0 initialize
  [PASS] #2 tools/list
  [FAIL] #3 tools/call  structure-change
         $.result: key 'echoed' added
  [FAIL] #4 tools/call  value-change
         $.result.content.0.text: '5' -> '6'
  [PASS] #5 tools/call
  GRADE B  (80.0/100)

tools/list is byte-for-byte identical between the two servers, so a static surface diff sees nothing. mcp-replay still catches the two behaviour changes.

Record a real session

record is a transparent stdio proxy. Point your MCP client at mcp-replay in place of the server, and it forwards everything both ways while teeing request/response pairs into a cassette. For Claude Desktop:

{
  "mcpServers": {
    "my-server": {
      "command": "mcp-replay",
      "args": [
        "record",
        "--server", "python /path/to/my_server.py",
        "--cassette", "/path/to/session.jsonl",
        "--redact", "session_id"
      ]
    }
  }
}

Use the client normally. Every tool call flows through and lands in the cassette. Secrets in known keys (authorization, api_key, token, and so on) plus bearer tokens and long opaque strings are redacted before they touch disk; add more keys with --redact.

Replay in CI

mcp-replay replay \
  --server "python /path/to/my_server.py" \
  --cassette session.jsonl \
  --min-grade B \
  --fail-on-regression

--fail-on-regression exits non-zero if any regression or missing response is found. --min-grade exits non-zero below the given letter. Wire either into a GitHub Actions step to block a merge that changes behaviour.

Output formats: default console, --json, --markdown, and --badge FILE which writes a shields.io endpoint JSON you can publish as a status badge.

Tune the comparison when a field is legitimately volatile:

• --ignore-path result.content[].text ignores a response path entirely.
• --allow-reorder treats a list with the same members in a different order as

a match rather than a regression.

Timestamps and UUIDs are masked automatically, so a server that stamps every result with the current time still scores a clean match.

Regression kinds

| Kind | Meaning | | --- | --- | | missing-response | server returned nothing or crashed on replay | | error-status-change | a result became an error, or the reverse | | structure-change | a key was added or removed, or a value's type changed | | ordering-change | a list has the same members in a different order | | value-change | a leaf value differs |

Grade

A 0..100 score over four weighted dimensions, mapped to A-F:

| Dimension | Weight | Meaning | | --- | --- | --- | | Responsiveness | 25% | every request got a well-formed response | | Match rate | 45% | normalized response matched the recording | | Error parity | 20% | success/error class matched the recording | | Structural stability | 10% | no keys added/removed, no type flips |

Any missing response caps the grade at D, because a server that stops answering is not "mostly stable".

Cassette format

JSONL. The first line is a meta header; each following line is one interaction.

{"schema": "mcp-replay/cassette@1", "meta": {"server": ["python", "my_server.py"]}}
{"seq": 0, "kind": "request", "ts": 0.0, "message": {"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}, "response": {"jsonrpc":"2.0","id":1,"result":{}}}
{"seq": 1, "kind": "notification", "ts": 0.0, "message": {"jsonrpc":"2.0","method":"notifications/initialized"}, "response": null}

Notifications are recorded so the handshake replays faithfully. They are sent fire-and-forget on replay and are not asserted.

Scope (v1)

Focuses on the client-to-server request/response flow over stdio, which covers tool calls, listing, and the initialize handshake. Server-initiated messages (notifications, sampling requests) are forwarded during recording but not asserted on replay. HTTP/SSE transports are not yet supported.

Development

python -m unittest discover -s tests -v
python -m mcp_replay demo

License

MIT. See [LICENSE](LICENSE).

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: shriramkv
• Source: shriramkv/mcp-replay
• License: MIT

Install and usage instructions live in the source repository linked above.