# Mcp Server Diff

> A reusable GitHub Actions workflow for testing MCP server conformance between versions

- **Type:** MCP server
- **Install:** `agentstack add mcp-sammorrowdrums-mcp-server-diff`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [SamMorrowDrums](https://agentstack.voostack.com/s/sammorrowdrums)
- **Installs:** 0
- **Category:** [Developer Tools](https://agentstack.voostack.com/c/developer-tools)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [SamMorrowDrums](https://github.com/SamMorrowDrums)
- **Source:** https://github.com/SamMorrowDrums/mcp-server-diff

## Install

```sh
agentstack add mcp-sammorrowdrums-mcp-server-diff
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# MCP Server Diff

[](https://github.com/marketplace/actions/mcp-server-diff)
[](https://www.npmjs.com/package/mcp-server-diff)
[](https://github.com/SamMorrowDrums/mcp-server-diff/releases)
[](https://opensource.org/licenses/MIT)

A GitHub Action for diffing [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) server **public interfaces** between versions. Compares the current branch against a baseline to surface any changes to your server's exposed tools, resources, prompts, and capabilities.

> **Also available as a standalone CLI** — see [CLI Documentation](#cli-tool) or install with `npx mcp-server-diff`

## Overview

MCP servers expose a **public interface** to AI assistants: tools (with their input schemas), resources, prompts, and server capabilities. As your server evolves, changes to this interface are worth tracking. This action automates public interface comparison by:

1. Building your MCP server from both the current branch and a baseline (merge-base, tag, or specified ref)
2. Querying both versions for their complete public interface (tools, resources, prompts, capabilities)
3. Generating a diff report showing exactly what changed
4. Surfacing results directly in GitHub's Job Summary

This is **not** about testing internal logic or correctness—it's about visibility into what your server _advertises_ to clients.

## Quick Start

Create `.github/workflows/mcp-diff.yml` in your repository:

```yaml
name: MCP Server Diff

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]
    tags: ['v*']

permissions:
  contents: read

jobs:
  mcp-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: SamMorrowDrums/mcp-server-diff@v2
        with:
          setup_node: true
          install_command: npm ci
          build_command: npm run build
          start_command: node dist/stdio.js
```

## Language Examples

### Node.js / TypeScript

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_node: true
    node_version: '22'
    install_command: npm ci
    build_command: npm run build
    start_command: node dist/stdio.js
```

### Python

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_python: true
    python_version: '3.12'
    install_command: pip install -e .
    start_command: python -m my_mcp_server
```

### Go

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_go: true
    install_command: go mod download
    build_command: go build -o bin/server ./cmd/stdio
    start_command: ./bin/server
```

### Rust

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_rust: true
    install_command: cargo fetch
    build_command: cargo build --release
    start_command: ./target/release/my-mcp-server
```

### C# / .NET

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_dotnet: true
    dotnet_version: '9.0.x'
    install_command: dotnet restore
    build_command: dotnet build -c Release
    start_command: dotnet run --no-build -c Release
```

### Custom Setup

If you need more control over environment setup (caching, specific registries, etc.), do your own setup before calling the action:

```yaml
steps:
  - uses: actions/checkout@v4
    with:
      fetch-depth: 0

  - uses: actions/setup-node@v4
    with:
      node-version: '22'
      cache: 'npm'
      registry-url: 'https://npm.pkg.github.com'

  - uses: SamMorrowDrums/mcp-server-diff@v2
    with:
      install_command: npm ci
      build_command: npm run build
      start_command: node dist/stdio.js
```

## Testing Multiple Transports

Test both stdio and HTTP transports in a single run using the `configurations` input:

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_node: true
    install_command: npm ci
    build_command: npm run build
    configurations: |
      [
        {
          "name": "stdio",
          "transport": "stdio",
          "start_command": "node dist/stdio.js"
        },
        {
          "name": "streamable-http",
          "transport": "streamable-http",
          "start_command": "node dist/http.js",
          "server_url": "http://localhost:3000/mcp"
        }
      ]
```

## Inputs Reference

### Language Setup (Optional)

| Input | Description | Default |
|-------|-------------|---------|
| `setup_node` | Set up Node.js environment | `false` |
| `node_version` | Node.js version | `20` |
| `setup_python` | Set up Python environment | `false` |
| `python_version` | Python version | `3.11` |
| `setup_go` | Set up Go environment | `false` |
| `go_version` | Go version (reads from go.mod if empty) | `""` |
| `setup_rust` | Set up Rust environment | `false` |
| `rust_toolchain` | Rust toolchain | `stable` |
| `setup_dotnet` | Set up .NET environment | `false` |
| `dotnet_version` | .NET version | `8.0.x` |

### Required Inputs

| Input | Description |
|-------|-------------|
| `install_command` | Command to install dependencies (e.g., `npm ci`, `pip install -e .`, `go mod download`) |

### Server Configuration

| Input | Description | Default |
|-------|-------------|---------|
| `build_command` | Command to build the server. Optional for interpreted languages. | `""` |
| `start_command` | Command to start the server for stdio transport | `""` |
| `transport` | Transport type: `stdio` or `streamable-http` | `stdio` |
| `server_url` | Server URL for HTTP transport (e.g., `http://localhost:3000/mcp`) | `""` |
| `configurations` | JSON array of test configurations for testing multiple transports | `""` |
| `server_timeout` | Timeout in seconds to wait for server response | `10` |
| `env_vars` | Environment variables as newline-separated `KEY=VALUE` pairs | `""` |

Either `start_command` (for stdio) or `server_url` (for HTTP) must be provided, unless using `configurations`.

### Comparison Configuration

| Input | Description | Default |
|-------|-------------|---------|
| `compare_ref` | Git ref to compare against. Auto-detects merge-base on PRs or previous tag on tag pushes if not specified. | `""` |
| `fail_on_diff` | Fail the action if API changes are detected. Useful for release validation workflows. | `false` |
| `fail_on_error` | Fail the action if a genuine probe error occurs (e.g. the server fails to start on **both** sides). A configuration that starts on only one side is reported as "missing on one side" and is **not** treated as a probe error — see [One-Sided Startup Failures](#one-sided-startup-failures). | `true` |

### Configuration Object Schema

When using `configurations`, each object supports:

| Field | Description | Required |
|-------|-------------|----------|
| `name` | Identifier for this configuration (appears in report) | Yes |
| `transport` | `stdio` or `streamable-http` | No (default: `stdio`) |
| `start_command` | Server start command (stdio: spawns process, HTTP: starts server in background) | Yes for stdio, optional for HTTP |
| `server_url` | URL for HTTP transport | Required for `streamable-http` |
| `startup_wait_ms` | Milliseconds to wait for HTTP server to start (when using `start_command`) | No (default: 2000) |
| `pre_test_command` | Command to run before probing (alternative to `start_command` for HTTP) | No |
| `pre_test_wait_ms` | Milliseconds to wait after `pre_test_command` | No |
| `post_test_command` | Command to run after probing (cleanup, used with `pre_test_command`) | No |
| `headers` | HTTP headers for this configuration | No |
| `env_vars` | Additional environment variables | No |
| `custom_messages` | Config-specific custom messages | No |
| `base_start_command` | Command for baseline comparison (skips git checkout for this config) | No |
| `base_server_url` | URL for baseline HTTP server (used with `base_start_command`) | No |

### Comparing Against External Servers

When comparing against external servers (e.g., Docker images, remote services), use `base_start_command` to specify a different command for the baseline. This skips git checkout for that configuration and probes the specified server directly:

```yaml
configurations: |
  [
    {
      "name": "compare-versions",
      "transport": "stdio",
      "start_command": "docker run -i ghcr.io/example/mcp-server:v2.0.0",
      "base_start_command": "docker run -i ghcr.io/example/mcp-server:v1.0.0"
    }
  ]
```

This is useful for:
- **Version comparison**: Compare a new release against the previous version
- **Golden reference testing**: Compare your local code against a known-good reference
- **Cross-implementation testing**: Compare different implementations of the same server
- **Self-testing CI**: Verify the action detects diffs by comparing two known-different servers

For HTTP transport, use `base_server_url` alongside `base_start_command`:

```yaml
configurations: |
  [
    {
      "name": "http-comparison",
      "transport": "streamable-http",
      "start_command": "docker run -p 3000:3000 myserver:latest",
      "server_url": "http://localhost:3000/mcp",
      "base_start_command": "docker run -p 3001:3000 myserver:v1.0.0",
      "base_server_url": "http://localhost:3001/mcp"
    }
  ]
```

## How It Works

### Execution Flow

1. **Baseline Detection**: Determines the comparison ref:
   - For pull requests: merge-base with target branch
   - For tag pushes: previous tag (e.g., `v1.1.0` compares against `v1.0.0`)
   - Explicit: uses `compare_ref` if provided
2. **Build Baseline**: Creates a git worktree at the baseline ref and builds the server
3. **Build Current**: Builds the server from the current branch
4. **Conformance Testing**: Sends MCP protocol requests to both servers:
   - `initialize` - Server capabilities and metadata
   - `tools/list` - Available tools and their schemas
   - `resources/list` - Available resources
   - `prompts/list` - Available prompts
5. **Report Generation**: Produces a Markdown report with diffs, uploaded as an artifact and displayed in Job Summary

### What Gets Compared

The action queries the **public interface** of both server versions and compares the responses:

| Method | What It Reveals |
|--------|----------------|
| `initialize` | Server name, version, capabilities |
| `tools/list` | Available tools and their JSON schemas |
| `resources/list` | Exposed resources |
| `prompts/list` | Available prompts |

Differences appear as unified diffs in the report. Common changes include:

- New tools, resources, or prompts added
- Schema changes (new parameters, updated descriptions)
- Capability changes (new features enabled)
- Version string updates

### One-Sided Startup Failures

Sometimes a configuration starts on one side of the comparison but fails to start on
the other. The most common cause is a **brand-new configuration** that depends on a CLI
flag or option that doesn't exist on the compare ref yet — for example a PR that
introduces a new scoped server mode. On the compare ref the unknown flag is rejected and
the server exits with something like `MCP error -32000: Connection closed`.

When **exactly one** side fails to start (and the other probes successfully), the action:

1. **Reports a fail on the side that could not start**, naming the version (current
   branch or compare ref) in a dedicated "🚫 missing on one side" section and a per-config
   callout, instead of an opaque probe error.
2. **Diffs the working side against an empty baseline**, so the entire surface of the
   new (or removed) configuration renders as added/removed — giving you a complete
   picture of what it exposes.
3. **Does not trip `fail_on_error`.** A one-sided startup failure is treated as a
   surface difference (`config-missing`), not a genuine probe error, so an introducing PR
   isn't failed for the expected "this config doesn't exist on the base ref yet" case. It
   still counts as a difference, so `fail_on_diff` will flag it if enabled.

When **both** sides fail to start, that is a genuine probe failure: it is reported as an
error and trips `fail_on_error` (when enabled).

## Transport Support

### stdio Transport

The default transport communicates with your server via stdin/stdout using JSON-RPC. For stdio, each configuration spawns a fresh server process:

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_node: true
    install_command: npm ci
    build_command: npm run build
    start_command: node dist/stdio.js
```

### Streamable HTTP Transport

For HTTP servers, you typically want to **start the server once** and test multiple configurations against it. Use `start_command` at the configuration level—the action spawns the server, waits for startup, probes it, then terminates it after that configuration completes:

```yaml
configurations: |
  [{
    "name": "http-server",
    "transport": "streamable-http",
    "start_command": "node dist/http.js",
    "server_url": "http://localhost:3000/mcp",
    "startup_wait_ms": 2000
  }]
```

**Per-configuration server lifecycle**: If your use case requires a fresh server instance per configuration (e.g., testing different flags or environment variables), include `start_command` in each configuration—each will get its own server process started and stopped.

**Shared server for multiple configurations**: If you want one HTTP server to handle multiple test configurations, use `pre_test_command`/`post_test_command` on the first/last configuration, or start the server in a prior workflow step:

```yaml
configurations: |
  [
    {
      "name": "config-a",
      "transport": "streamable-http",
      "server_url": "http://localhost:3000/mcp",
      "pre_test_command": "node dist/http.js &",
      "pre_test_wait_ms": 2000
    },
    {
      "name": "config-b",
      "transport": "streamable-http",
      "server_url": "http://localhost:3000/mcp"
    },
    {
      "name": "config-c",
      "transport": "streamable-http",
      "server_url": "http://localhost:3000/mcp",
      "post_test_command": "pkill -f 'node dist/http.js' || true"
    }
  ]
```

**Pre-deployed servers**: For already-running servers (staging, production), omit lifecycle commands entirely:

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    install_command: 'true'
    transport: streamable-http
    server_url: https://mcp.example.com/api
```

## Version Comparison Strategies

### Pull Requests

On pull requests, the action automatically compares against the merge-base with the target branch. This shows exactly what changes the PR introduces.

### Tag Releases

When triggered by a tag push matching `v*`, the action finds the previous tag and compares against it:

```yaml
on:
  push:
    tags: ['v*']

# v1.2.0 will automatically compare against v1.1.0
```

### Explicit Baseline

Specify any git ref to compare against:

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_node: true
    install_command: npm ci
    build_command: npm run build
    start_command: node dist/stdio.js
    compare_ref: v1.0.0
```

### Failing on Changes (Release Validation)

For release workflows where you want to ensure no API changes, use `fail_on_diff`:

```yaml
- uses: SamMorrowDrums/mcp-server-diff@v2
  with:
    setup_node: true
    install_command: npm ci
    build_command: npm run build
    start_command: node dist/stdio.js
    compare_ref: v1.0.0
    fail_on_diff: true  # Action fails if any API changes are detected
```

## Artifacts and Reports

The action produces:

1. **Job Summary**: Inline Markdown report in the GitHub Actions UI showing test results and diffs
2. **Artifact**: `mcp-diff-report` artifact containing `MCP_DIFF_REPORT.md` for download or further processing

## Example Output

### No Changes Detected

When the MCP server's public interface hasn't changed between branches:

```
📊 Comparison:
  Current: HEAD
  Compare: abc1234 (v1.0.0)

🧪 Running diff...

📊 Phase 3: Comparing results...
📋 Configuration stdio: ✅ No changes

✅ No API Changes - All configurations match the baseline.
```

### Changes Detected

When changes are

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [SamMorrowDrums](https://github.com/SamMorrowDrums)
- **Source:** [SamMorrowDrums/mcp-server-diff](https://github.com/SamMorrowDrums/mcp-server-diff)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-sammorrowdrums-mcp-server-diff
- Seller: https://agentstack.voostack.com/s/sammorrowdrums
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.