Video Extract Mcp

video-extract-mcp

Turn any video URL into a transcript and the handful of frames that actually matter — locally, from an MCP server your AI agent can call.

Give it a YouTube link, a TikTok, a WeChat Channels share URL, a raw .mp4, or a page from a site nobody has heard of. It fetches only what the request actually needs, produces a transcript (real captions when the platform has them, local speech recognition when it does not), and returns a small set of important keyframes — deduplicated, scene-aware, and scored — instead of a thousand near-identical stills.

Built for AI agents. Two MCP tools, no cloud, no API keys, no Python.

[](https://www.npmjs.com/package/@yanlinglabs/video-extract-mcp) [](LICENSE) [](https://nodejs.org) [](https://www.typescriptlang.org/) [](#testing) [](https://modelcontextprotocol.io)

---

Why this exists

An LLM cannot watch a video. The usual workaround — dump every Nth frame into the context window — burns enormous amounts of context on frames that are 98% identical to the one before, and still misses the slide that changed while nothing else moved.

video-extract-mcp does the selection work first:

• Transcript, honestly sourced. The platform's own captions are used whenever the video has any — human-written first, otherwise the platform's automatic ones. Audio is transcribed locally (Whisper or SenseVoice) only for videos with no captions at all. The result tells you which you got, via transcript.source.
• Keyframes chosen, not sampled. Scene-boundary detection, blur/quality filtering, on-screen-text novelty (subtitle-aware, so burned-in captions don't preserve redundant frames), and image-embedding similarity feed an iterative diversity-aware selector.
• Output goes to disk, not into your context. The tool reply is a compact summary plus file paths. A 35-frame manifest and a full transcript don't belong in a conversation where the agent needs three numbers from them.
• Everything runs on your machine. No third-party API, no upload, no key. Long analyses can run as MCP background tasks — the tool returns a handle immediately and pushes progress; see Background tasks below.

Quick start

Install the system binaries first — these can't come from npm:

# macOS; use your package manager elsewhere
brew install ffmpeg yt-dlp tesseract tesseract-lang

Then point your MCP client at the package. There are two ways, and they differ in ways worth thirty seconds of your time.

Option A — npx, nothing installed. Simplest, and it picks up new releases on its own.

Claude Code:

claude mcp add --scope user video-extract -- npx -y @yanlinglabs/video-extract-mcp@latest

Codex:

codex mcp add video-extract -- npx -y @yanlinglabs/video-extract-mcp@latest

Another agent? Point it at SKILL.md and it can install itself.

Keep the @latest — without it npx pins to the first version it cached and never updates.

Option B — installed globally. Starts faster and gives you the video-extract status CLI as a real command.

npm install -g @yanlinglabs/video-extract-mcp

Then register it — Claude Code:

claude mcp add --scope user video-extract -- video-extract-mcp

Codex:

codex mcp add video-extract -- video-extract-mcp

| | npx (A) | global install (B) | |---|---|---| | Updates | automatic only with @latest in the spec — a bare npx -y @yanlinglabs/video-extract-mcp pins to the first version it cached and never updates | manual: npm update -g @yanlinglabs/video-extract-mcp. You stay on the installed version until you run it | | Startup | ~0.9s (npm resolution on every launch) | ~0.1s | | video-extract status in your shell | not on PATH — needs npx -y -p @yanlinglabs/video-extract-mcp video-extract status | works directly | | Working directory | must not be this package's own checkout (see below) | irrelevant |

Neither affects what agents can do: an agent checks on background work over HTTP using the statusUrl handed to it in the reply, never a shell command. The CLI is for humans.

Or in any MCP client's config — "command": "npx", "args": ["-y", "@yanlinglabs/video-extract-mcp@latest"] for A, or "command": "video-extract-mcp" with no args for B:

{
  "mcpServers": {
    "video-extract": {
      "command": "npx",
      "args": ["-y", "@yanlinglabs/video-extract-mcp@latest"]
    }
  }
}

> One gotcha with npx, and it only bites contributors. Run inside this package's own git checkout, npx @yanlinglabs/video-extract-mcp fails with command not found — npx sees the local package.json claiming that name, looks for the binary in a local node_modules/.bin that was never populated, and gives up. Since MCP clients launch servers with the working directory set to your project, option A cannot work in this repo. Working on the tool itself? Point that one project at your build — claude mcp add --scope local video-extract -- node "$PWD/dist/mcp.js" — which also means a npm run build takes effect immediately, with no publish round-trip. Everywhere else, npx is fine.

That is enough for any video that has captions — which, thanks to the caption-first transcript policy, is most of them. The vision model downloads itself on first use.

Speech models are only needed for videos with no captions at all. They are ~1.5 GB, so they are not bundled. Fetch them when you want that fallback:

npx -y @yanlinglabs/video-extract-mcp --help   # installs the package
curl -fsSL https://raw.githubusercontent.com/yanlingLabs/video-extract-mcp/main/scripts/fetch-models.sh \
  | bash -s -- ~/.cache/video-extract-mcp/models

~/.cache/video-extract-mcp/models is where the tool looks by default. Override with VIDEO_EXTRACT_MODELS_DIR. Without them, an uncaptioned video still returns frames and records a warning explaining the transcript is missing — it degrades rather than fails.

From source (contributors)

git clone https://github.com/yanlingLabs/video-extract-mcp.git
cd video-extract-mcp
npm install && npm run build
./scripts/fetch-models.sh    # into ./models, which takes precedence when present
npm run preflight            # verifies ffmpeg / ffprobe / yt-dlp / tesseract

Environment variables

| Variable | Purpose | |---|---| | VIDEO_EXTRACT_MODELS_DIR | Where speech models live. Defaults to ./models when that exists, else ~/.cache/video-extract-mcp/models. | | VIDEO_EXTRACT_COOKIES_FILE | Path to a Netscape-format cookie jar, used for every yt-dlp source at once — YouTube, Instagram, Facebook, X, TikTok, Twitch and the rest. See [Authenticated sources](#authenticated-sources). | | VIDEO_EXTRACT_COOKIES_FROM_BROWSER | Load cookies from a local browser instead: chrome, firefox, safari, edge, brave, chromium, opera, vivaldi, whale, optionally browser:profile. Ignored when VIDEO_EXTRACT_COOKIES_FILE is set. | | VIDEO_EXTRACT_WECHAT_COOKIE | A yuanbao session cookie, required only for WeChat Channels links. Separate from the above by design — a different protocol with its own credential. | | VIDEO_EXTRACT_MAX_CONCURRENCY | Caps concurrent analyze_video item executions — plain calls and background tasks, batch items and separate calls, all count against the same limit. Default 4. resolve_video is exempt: it loads no models, so there is nothing to throttle. | | VIDEO_EXTRACT_TASK_TTL_MS | How long a completed background-task handle stays queryable before it expires. Default 1800000 (30 minutes). 0 (or any non-positive value) means the handle never expires. Governs the in-memory handle only — files already written to destinationPath are never deleted by the tool, expired handle or not. | | VIDEO_EXTRACT_STATUS_PORT | Pins the port of the localhost /status endpoint (see [Watching progress](#watching-progress)). Unset picks an ephemeral port each start (default: endpoint on). The literal value 0 disables the endpoint entirely — note the contrast with VIDEO_EXTRACT_TASK_TTL_MS above, where 0 means no expiry, not disabled. |

Authenticated sources

Plenty of media is not public, and the answer is the same one the platforms themselves ask for: cookies. One jar covers every yt-dlp source at once — cookies are scoped by domain inside the file, so a single export authenticates YouTube, Instagram, Facebook, X, TikTok and Twitch together. It is not a YouTube-only setting.

export VIDEO_EXTRACT_COOKIES_FROM_BROWSER=auto       # recommended: borrow only when blocked

auto is lazy, and that is the point. Ordinary requests send no cookies at all. Only when a platform actually refuses one does the server detect an installed browser, retry that single request with its cookies, and stop — no loop. You pay the cost of touching a credential store only when something is genuinely blocked, which is also what keeps a borrowed session from being rotated out from under you on every public video.

The other two modes are eager — cookies on every request:

Read a browser's store directly (chrome, safari, edge, brave, …):

export VIDEO_EXTRACT_COOKIES_FROM_BROWSER=firefox

Or point at an exported Netscape-format jar:

export VIDEO_EXTRACT_COOKIES_FILE=~/cookies.txt

Set both and the file wins — they are alternatives, not a pair.

Expect an OS keychain prompt. Every Chrome-family browser encrypts its cookie store against the system keyring, so the first read shows a dialog you must approve — on macOS, "Chrome Safe Storage". Firefox does not; its store is plain SQLite, which is why auto prefers it when both are present. If you set nothing at all and a request is refused, the reply tells you the command to enable this and warns about that prompt, rather than leaving you to discover it.

What it unlocks, beyond simply logging in: age-restricted and members-only YouTube, most of Instagram and Facebook, much of X, subscriber-only Twitch — and rate_limited / "sign in to confirm you're not a bot", which an anonymous fetch hits far sooner than a signed-in one.

Three things worth knowing before you use it:

• It is read from the environment only. A caller can never name a cookie file or a browser per-request. An agent that could do either could read any file on the machine, or lift a live session — so that stays the operator's decision, not the agent's.
• Your jar is never modified. --cookies FILE doesn't only read that file, it rewrites it on exit; the tool copies your jar to a private temp file, hands yt-dlp the copy, and deletes it afterwards. The cost is that refreshed cookies aren't written back.
• Exporting from a browser you are actively logged into can log you out. Platforms rotate session cookies, and a copy taken from a live session goes stale — YouTube is especially prone to it. yt-dlp's own advice: export from a private/incognito window, then close that window without logging out.

A wrong path fails loudly rather than quietly fetching anonymously — an unreadable jar is a broken setup, not something to degrade past, and silently anonymous results would send you hunting the wrong bug.

Check what you actually configured, without printing a single cookie value:

$ video-extract cookies
cookie jar: /Users/you/cookies.txt
  .youtube.com      12 cookies  expires in 23 days
  .instagram.com     4 cookies  expires in 3 days
  .x.com             2 cookies  EXPIRED 5 days ago
  3 domain(s), 18 cookies, 1 with expired cookies

It reports domains, counts and expiry — never names or values, so the output is safe to paste into an issue. It also names the mistake people actually make: an exporter set to JSON rather than the Netscape format produces a file that looks fine and contains nothing yt-dlp can read, which otherwise surfaces much later as a confusing auth_required on an unrelated video. Add --json for scripting.

Three ways to use it

The MCP server is the main surface, but the same engine is available two other ways.

As a CLI, which is the quickest way to see what it does before wiring up an agent:

npm run cli -- "https://youtube.com/watch?v=..." --max-frames 10 --out ./output

# just the transcript, no frames
npm run cli -- "" --frames none --out ./output

# one exact frame at 7s, as cheap as this gets
npm run cli -- "" --start 7 --end 7 --frames even --max-frames 1 --no-transcript --out ./output

It writes manifest.json plus the frame images into --out, and also prints the manifest to stdout.

If you want to pipe that JSON somewhere, call the built entry point directly — npm run prefixes its own banner lines to stdout, so npm run cli output is not valid JSON on its own:

npm run build
node dist/cli.js "" --max-frames 10 | jq '.transcript.source'

As a library, if you want the pipeline without an agent in the loop:

import { analyzeVideo } from '@yanlinglabs/video-extract-mcp/dist/analyze.js';

const manifest = await analyzeVideo('https://youtube.com/watch?v=...', {
  start: 30, end: 90, frames: 'key', maxFrames: 12, outDir: './output',
});
console.log(manifest.transcript?.source);   // 'manual' | 'auto' | 'asr'
console.log(manifest.frames.map((f) => f.image));

analyzeVideo never throws for expected failures — a DRM page or a dead link comes back as a manifest whose source.status is not 'ok', carrying a readable reason. Check processing.warnings too: any optional stage that failed and was skipped past records an entry there.

Note that both the CLI and library paths run the compiled output. The speech and vision models run in separate worker processes resolved next to the compiled module, so running the TypeScript sources directly leaves those workers unresolvable — they degrade to a warning rather than an error, which is quiet enough to miss. npm run cli builds first for this reason.

The two tools

The surface is deliberately small. Earlier versions had four tools and the descriptions had to shout about which ones took URLs versus local paths — a sign the design was wrong, not that the warning needed to be louder.

resolve_video — look it up, optionally fetch it

resolve_video({
  destinationPath: string,          // required — shared by every item below
  videos: [{                        // one entry per video, at least one
    url:             string,        // required
    returnVideo?:    boolean,       // default false: metadata only, no download
    start?:          number,        // seconds; only with returnVideo: true
    end?:            number,
    comments?:       boolean,       // default false — slow on popular videos
  }],
})

One video — the common case, written flat into destinationPath:

resolve_video({
  destinationPath: "./out",
  videos: [{ url: "https://youtube.com/watch?v=..." }],
})
// -> ./out/metadata.json

Several videos in one call — each gets its own subdirectory, video-1/, video-2/, ... in array order:

resolve_video({
  destinationPath: "./out",
  videos: [
    { url: "https://youtube.com/watch?v=..." },
    { url: "https://tiktok.com/@user/video/...", returnVideo: true },
  ],
})
// -> ./out/video-1/metadata.json
// -> ./out/video-2/metadata.json + source.mp4 (returnVideo: true)

By default it downloads nothing heavy. You get title, creator, duration, the chapter list when the platform publishes one, and a short description preview. That is usually enough to decide what to do next — and it composes with ranges into the workflow that makes this whole thing efficient:

> Read the chapters → see the demo starts at 12:04 → analyze only 12:04–20:00 → skip 90% of the download, transcription, and frame work.

analyze_video — the real work

analyze_video({
  destinationPath: string,                      // required — shared by

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [yanlingLabs](https://github.com/yanlingLabs)
- **Source:** [yanlingLabs/video-extract-mcp](https://github.com/yanlingLabs/video-extract-mcp)
- **License:** MIT
- **Homepage:** https://github.com/yanlingLabs/video-extract-mcp

Install and usage instructions live in the source repository linked above.