Talkthrough Mcp

talkthrough-mcp

[](https://github.com/korovin-aa97/talkthrough-mcp/actions/workflows/ci.yml) [](LICENSE) [](pyproject.toml) [](https://pypi.org/project/talkthrough-mcp/)

[Quickstart](#quickstart) · [Tools](#tools) · [FAQ](#faq) · [Troubleshooting](docs/TROUBLESHOOTING.md) · [Changelog](CHANGELOG.md) · [Contributing](CONTRIBUTING.md)

Feedback ingestion for AI agents. Record your screen and talk; your agent does the rest — files the bugs, writes the spec, builds the backlog.

talkthrough-mcp is a local-first MCP server that turns a narrated screen recording (or any video/audio file) into agent-ready structured data: timestamped transcript segments, scene-change keyframes, OCR'd on-screen text, and wall-clock anchoring. Everything is served through lazy retrieval tools, so a 30-minute recording never floods the model context — the agent pulls exactly the transcript slice, moment bundle, or frame it needs.

There is no LLM inside the server and no cloud anywhere in the path: ffmpeg, faster-whisper, and RapidOCR run on your machine, and the calling agent brings the intelligence. What makes it different from screen-recorder SaaS and video-analyzer MCPs: it works on arbitrary local files, it ships the agent workflows (server prompts + example agents), and it anchors every timestamp to wall-clock time — so "the moment I said the checkout hung" maps straight to the right window of your server logs.

Quickstart

One command, no system dependencies: ffmpeg falls back to a bundled build, OCR is pip-only, and whisper models download themselves on first use.

[](https://cursor.com/en/install-mcp?name=talkthrough&config=eyJjb21tYW5kIjoidXZ4IiwiYXJncyI6WyJ0YWxrdGhyb3VnaC1tY3AiXX0%3D) [](https://vscode.dev/redirect/mcp/install?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22talkthrough-mcp%22%5D%2C%22type%22%3A%22stdio%22%7D) [](https://insiders.vscode.dev/redirect/mcp/install?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22talkthrough-mcp%22%5D%2C%22type%22%3A%22stdio%22%7D&quality=insiders) [](https://lmstudio.ai/install-mcp?name=talkthrough&config=eyJjb21tYW5kIjoidXZ4IiwiYXJncyI6WyJ0YWxrdGhyb3VnaC1tY3AiXX0%3D) [](https://kiro.dev/launch/mcp/add?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22talkthrough-mcp%22%5D%7D)

Claude Code

Two install paths — pick one, not both (the plugin already includes the server; installing both would register it twice):

Server only — the 7 tools + 5 prompts, and nothing else on your system. Choose this for a minimal setup, or when you manage MCP servers yourself across several clients:

claude mcp add -s user talkthrough -- uvx talkthrough-mcp

Full plugin — the same server, plus native slash commands (/talkthrough:triage-recording, …) that handle the ceremony for you, a ready-made triage subagent, and an agent skill that teaches Claude the workflow. Choose this for the best out-of-the-box experience:

/plugin marketplace add korovin-aa97/talkthrough-mcp
/plugin install talkthrough@talkthrough

Every other MCP client

Claude Desktop

claude_desktop_config.json:

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: [integrations/claude-desktop/](integrations/claude-desktop/)

Cursor

~/.cursor/mcp.json (or project .cursor/mcp.json):

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: [integrations/cursor/](integrations/cursor/)

OpenAI Codex CLI

~/.codex/config.toml (or project-scoped .codex/config.toml in trusted projects):

[mcp_servers.talkthrough]
command = "uvx"
args = ["talkthrough-mcp"]

More: [integrations/codex/](integrations/codex/)

Gemini CLI

~/.gemini/settings.json:

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: [integrations/gemini-cli/](integrations/gemini-cli/)

Cline / Roo Code

cline_mcp_settings.json (via MCP Servers UI):

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: [integrations/cline/](integrations/cline/)

OpenClaw

~/.openclaw/openclaw.json:

{
  "mcp": {
    "servers": {
      "talkthrough": {
        "command": "uvx",
        "args": [
          "talkthrough-mcp"
        ]
      }
    }
  }
}

More: [integrations/openclaw/](integrations/openclaw/)

OpenCode

opencode.json (project) or ~/.config/opencode/opencode.json:

{
  "mcp": {
    "talkthrough": {
      "type": "local",
      "command": [
        "uvx",
        "talkthrough-mcp"
      ],
      "enabled": true
    }
  }
}

More: [integrations/opencode/](integrations/opencode/)

Goose

~/.config/goose/config.yaml:

extensions:
  talkthrough:
    enabled: true
    type: stdio
    cmd: uvx
    args: ["talkthrough-mcp"]

More: [integrations/goose/](integrations/goose/)

GitHub Copilot CLI

~/.copilot/mcp-config.json:

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: [integrations/copilot-cli/](integrations/copilot-cli/)

Windsurf

~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: [integrations/windsurf/](integrations/windsurf/)

Zed

settings.json (Zed):

{
  "context_servers": {
    "talkthrough": {
      "source": "custom",
      "command": {
        "path": "uvx",
        "args": [
          "talkthrough-mcp"
        ]
      }
    }
  }
}

More: [integrations/zed/](integrations/zed/)

Any other MCP stdio client uses the same server command: uvx talkthrough-mcp. Per-engine folders with exactly these snippets plus verification steps live in [integrations/](integrations/); agents can self-install via [llms-install.md](llms-install.md).

Local checkout (development)

git clone https://github.com/korovin-aa97/talkthrough-mcp
claude mcp add talkthrough -- uv run --directory /path/to/talkthrough-mcp talkthrough-mcp

Then, in your agent:

> Process ~/Desktop/recording.mov and triage it — or just invoke the > triage-recording server prompt.

Tools

| Tool | What it does | |---|---| | process_media(path, recorded_at?, vocabulary?, language?, model?, force?) | Ingest a video/audio file: local STT, keyframes, OCR, wall-clock. Returns a compact summary. Idempotent by content hash — re-calls are instant. | | get_transcript(job_id, start_ms?, end_ms?, format?) | Paginated transcript as segments, text, or srt; truncation returns next_start_ms. | | get_frames(job_id, at_ms? \| start_ms?+end_ms?, max_frames?, include_duplicates?) | Keyframe images nearest a timestamp or evenly thinned across a range (unique frames by default, max 6/call). | | get_moment(job_id, start_ms, end_ms) | The "one remark" bundle: transcript slice + up to 3 frames + their OCR text + wall-clock range. | | search(job_id, query) | Substring search over the transcript AND on-screen OCR text; hits carry t_ms/t_wall and frame refs. | | extract_frame(job_id, at_ms, crop?) | Exact-timestamp full-resolution re-extract from the source video (optional crop) when keyframes miss the instant. | | list_jobs() | Recent processed recordings with durations, wall-clock starts, and counts. |

Every tool description ships 10+ usage examples, so agents pick the right tool without extra prompting.

Server prompts (slash commands in MCP clients)

| Prompt | Workflow | |---|---| | triage-recording | Narrated screencast → precise findings JSON (bug/feature/question routing, frame evidence) | | spec-from-workshop | Recorded workshop → structured spec with quoted decisions and open questions | | backlog-from-demo | Product demo → prioritized backlog with timestamped evidence | | meeting-actions | Meeting audio → action items, decisions, open questions | | correlate-with-logs | Recording remarks ↔ system logs via wall-clock windows |

The same prompts live as plain files in [examples/prompts/](examples/prompts/) if your client doesn't surface MCP prompts. The findings contract used by triage-recording is [examples/output-contract.schema.json](examples/output-contract.schema.json).

Works as a skill too (no MCP required)

The same workflow ships as a cross-engine Agent Skill at [.agents/skills/talkthrough/](.agents/skills/talkthrough/) — Claude Code, Codex CLI ($talkthrough), Cursor, Copilot, Gemini CLI, Goose and other SKILL.md-compatible tools read it. Agents without MCP wiring can drive the [CLI](#cli) directly: talkthrough-mcp process recording.mov --json prints the same summary the MCP tool returns, and the job store is shared either way.

Wall-clock anchoring

Every timestamped result carries both t_ms (video-relative) and t_wall (ISO 8601 real time) once the recording start is known. Resolution ladder:

1. recorded_at parameter (agent/user override) → confidence exact
2. QuickTime com.apple.quicktime.creationdate tag, carries the local

timezone (QuickTime Player recordings; ⌘⇧5 wrote it before macOS 26) → high

1. Container creation_time tag (UTC) → medium — macOS 26+ ⌘⇧5/ReplayKit

screen recordings land here (no creationdate tag anymore); pass recorded_at= when local-tz t_wall matters

1. File mtime minus duration (recorders finalize files at recording END) → low
2. Nothing → tools still work with relative t_ms only

Why it matters: "the upload spinner froze here" becomes a ±30 s grep window in your server logs.

Privacy

Everything runs locally: your recordings never leave your machine, speech is transcribed by a local whisper model, OCR is local ONNX inference, and there is no telemetry. The only network access is one-time tool/model downloads (ffmpeg build, whisper model, OCR models).

Languages

Narration in any of Whisper's ~99 languages works: the language is auto-detected per recording, and the summary reports both language and language_probability so agents can tell a confident detection from a shaky one (silence or music at the start can fool the detector — pin it with language="ru" and force=true when that happens).

Pick the model for your languages — per call (model= parameter, agents do this themselves when a transcript comes back garbled) or as the server default (TALKTHROUGH_WHISPER_MODEL):

| Model | Size | Best for | |---|---|---| | small (default) | 464 MB | English and major-language narration on CPU | | large-v3-turbo | ~1.5 GB | recommended for non-English — near-large quality at near-small speed | | medium | ~1.5 GB | conservative alternative to turbo | | tiny / base | 75–145 MB | quick drafts, CI | | *.en variants | — | English-only, slightly faster/better for EN |

Tips that work in every language: pass product names via vocabulary="Term1, Term2" (biases the decoder so jargon survives), and note that the workflow prompts instruct agents to write digests in the narrator's language while keeping quotes verbatim — the server never translates (exact quotes are evidence; translation is the agent's job).

On-screen text (OCR) defaults to RapidOCR's Latin + Chinese models. For other scripts set TALKTHROUGH_OCR_LANG to your language — ru/uk (→ the eslav pack), ja, ko, ar, hi, el, th, or any RapidOCR pack name like cyrillic — and reprocess with force=true; the matching recognition model downloads once. Spoken-language support is unaffected either way.

Configuration

| Env var | Default | Meaning | |---|---|---| | TALKTHROUGH_WHISPER_MODEL | small | default whisper model (tiny/base/small/medium/large-v3/large-v3-turbo); the model tool param overrides per call | | TALKTHROUGH_OCR | on | set off to skip OCR | | TALKTHROUGH_OCR_LANG | Latin+Chinese | recognition script for on-screen text: a language code (ru, ja, ko, ar, hi, …) or a RapidOCR pack name (eslav, cyrillic, latin, …); the model downloads once | | TALKTHROUGH_OCR_PARAMS | — | advanced: JSON object of raw RapidOCR params merged over the derived ones, e.g. {"Rec.lang_type": "cyrillic"} | | TALKTHROUGH_MAX_SECONDS | 7200 | max media duration | | TALKTHROUGH_MAX_FRAMES | 600 | keyframe cap per job | | TALKTHROUGH_HOME | ~/.talkthrough | job store root |

CLI

The pipeline is also a CLI — useful for pre-processing long recordings outside an agent session (the store is content-addressed, so the agent then queries the same job instantly):

talkthrough-mcp process ~/Videos/long-session.mov   # prints the summary
talkthrough-mcp process demo.mov --json             # machine-readable
talkthrough-mcp gc --keep-days 30                   # clean the job store
talkthrough-mcp serve                               # stdio MCP server (default)

First run notes: missing system ffmpeg triggers a one-time static-ffmpeg download; the first transcription downloads the whisper model (~460 MB for small); both are cached. After that, expect roughly 3× faster than real time on an Apple-Silicon CPU with the default model, OCR included (a 2-minute clip processes in ~40 s) — and instant re-runs on the same file. Progress streams as MCP progress notifications, and the CLI prints stage lines. More: [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md).

Windows (best-effort)

CI runs lint, the unit suite, and a full CLI smoke on windows-latest (static-ffmpeg Windows build, whisper tiny transcription, OCR, and the instant idempotent re-run). Notes: the per-job lock is POSIX fcntl and degrades to a no-op on Windows — fine for a single-user machine; quote paths with spaces (uv run talkthrough-mcp process "C:\Videos\Screen Recording.mp4"). Windows is not a release gate — if something breaks, please open an issue.

Supported inputs

Video: .mov .mp4 .webm .mkv — audio-only: .m4a .mp3 .wav .ogg .flac (transcript tools only; frame tools explain why they're unavailable). Local files only.

Limitations

Honest edges, so you can decide fast:

• One speaker stream. No diarization yet — "who said it" isn't tracked

(#4).

• Local files only. No URL/YouTube ingestion

(#5) — download first.

• Keyframes + transcript, not motion analysis. A glitch between scene

changes can be invisible in the frame set; extract_frame re-checks any instant, but frame-by-frame motion reasoning is your multimodal model's job.

• STT quality tracks the model you pick. The default small favors speed;

non-English narration wants model="large-v3-turbo" (see [Languages](#languages)).

• OCR reads crisp UI text well; tiny or low-contrast print is best-effort.
• Wall-clock confidence depends on recorder metadata — worst case pass

recorded_at= (see the ladder above).

• Windows is best-effort (see above).

How it compares

| | talkthrough | cloud recorder SaaS | meeting notetakers | typical video-analyzer MCPs | |---|---|---|---|---| | Runs fully locally | ✅ | ❌ | ❌ | varies | | Any local video/audio file | ✅ | browser/app captures | meetings only | ✅ | | Wall-clock anchoring (log correlation) | ✅ | ❌ | ❌ | ❌ | | Ships agent workflows (prompts, skill, findings contract) | ✅ | ❌ | ❌ | ❌ | | OCR of on-screen text, searchable | ✅ | some | ❌ | rare |

FAQ

Why not just upload the video to a multimodal model (e.g. Gemini)? For a short, non-sensitive clip — do that. The trade-offs appear with length and sensitivity: an hour of sc

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: korovin-aa97
• Source: korovin-aa97/talkthrough-mcp
• License: MIT

Install and usage instructions live in the source repository linked above.