Ai Video Editor

AI Video Editor

A local, open-source CLI tool that automatically edits talking-head videos using AI. Point it at a raw recording and it removes silences, filler words ("um", "uh", "ну", "типа"), and failed takes (say "cut cut" to mark a restart). It uses Silero VAD for speech detection, Whisper large-v3 for transcription with word-level timestamps, and FFmpeg for frame-accurate assembly. Optionally generates a smart hook opener and YouTube chapter markers via LLM. Runs entirely on your machine — no cloud APIs required for the core pipeline. Designed to be invoked by AI agents (structured JSON output) or used as an MCP server in Claude Desktop.

Features

• Silence removal — Silero VAD strips dead air; short natural pauses are preserved
• Filler word removal — English and Russian filler words ("um", "uh", "ну", "типа", "эээ", ...)
• Restart detection — "cut cut" / "кат кат" removes the entire failed take; short isolated bursts (coughs, false starts) are auto-removed via VAD duration filtering
• Repeated sentence detection — auto-detects and cuts duplicate sentence starts
• Smart hook generation — OpenRouter LLM picks the best 8-second opener (optional)
• YouTube chapter markers — LLM generates timestamped chapters from the transcript (optional)
• Hardware encoding — h264_videotoolbox on Apple Silicon for fast final encode
• Configurable YAML pipeline — every threshold, model, and feature toggle is overridable

Requirements

• Python 3.11+
• FFmpeg 7+ (brew install ffmpeg)
• macOS with Apple Silicon (for VideoToolbox hardware encoding; falls back to libx264 elsewhere)
• ~4 GB RAM for Whisper large-v3 (use --whisper-model small for lighter machines)

Installation

cd Tools/ai-video-editor
uv venv
source .venv/bin/activate
uv pip install -e .

Quick Start

# Core editing only (no API calls)
ai-video-editor process video.mp4 --no-hook --no-chapters

# Full pipeline with smart hook opener + YouTube chapters (requires OPENROUTER_API_KEY)
ai-video-editor process video.mp4

Example output (stdout):

{
  "status": "complete",
  "input": "video.mp4",
  "output_video": "video_edited.mp4",
  "duration_original_sec": 154.6,
  "duration_edited_sec": 145.9,
  "segments_removed": 1,
  "silence_removed_sec": 2.1,
  "restarts_removed": 1,
  "fillers_removed": 4
}

Progress events are emitted to stderr as JSON lines during processing.

CLI Commands

process — Edit a video

ai-video-editor process VIDEO [OPTIONS]

| Option | Description | |--------|-------------| | --config, -c PATH | Custom YAML config file (merged over defaults) | | --whisper-model, -m MODEL | Whisper model size: tiny / base / small / medium / large / large-v3 | | --lut PATH | Path to a .cube LUT file for color grading | | --output, -o PATH | Output file path (default: {input}_edited.mp4) | | --no-hook | Skip smart hook generation (no OpenRouter call) | | --no-chapters | Skip YouTube chapter generation (no OpenRouter call) |

info — Inspect a video file

ai-video-editor info VIDEO

Prints codec, resolution, FPS, duration, and bitrate as JSON.

models — List Whisper model sizes

ai-video-editor models

Configuration

The default config lives in config.default.yml. Override any section with --config my.yml — your file is deep-merged over the defaults, so you only need to specify what changes.

| Section | Key settings | |---------|-------------| | whisper | model, language, device | | silence | min_gap_sec (merge threshold), padding_sec (breathing room at cuts) | | restarts | enabled, trigger_phrases, detect_repeated_starts, max_burst_duration_sec | | fillers | enabled, min_filler_duration_sec, words.en, words.ru | | audio | enabled (off by default), noise reduction and loudness settings | | video | lut_path | | hook | enabled, duration_sec, model | | chapters | enabled, model | | encoding | codec, quality, audio_codec, audio_bitrate |

Example override — use a smaller Whisper model and enable audio enhancement:

# my-config.yml
whisper:
  model: small
audio:
  enabled: true

ai-video-editor process video.mp4 --config my-config.yml

How It Works

The pipeline runs four sequential phases:

1. Analysis — Extract audio → Silero VAD (speech segments) → Whisper transcription → edit decisions (remove short bursts, merge short gaps, remove restarts and fillers, apply padding)
2. Assembly — Frame-accurate FFmpeg segment extraction → concat → optional audio enhancement → optional LUT color grade
3. AI Enhancement — Smart hook selection and YouTube chapters via OpenRouter (skipped if --no-hook --no-chapters or no API key)
4. Encode — Final h264_videotoolbox (or libx264) encode with AAC audio, optimized for web playback

Claude Desktop (MCP Server)

The tool includes a built-in MCP server so Claude Desktop can use it as a native capability.

Setup:

1. Add to your claude_desktop_config.json (see claude_desktop_config.example.json for the template):

"ai-video-editor": {
  "command": "/path/to/ai-video-editor/.venv/bin/python",
  "args": ["-m", "src.mcp_server"],
  "cwd": "/path/to/ai-video-editor"
}

1. Restart Claude Desktop.
2. Optionally install SKILL.md as a capability for model selection guidance and workflow tips.

Claude gets three tools: process_video, video_info, and list_models.

Environment Variables

| Variable | Description | |----------|-------------| | OPENROUTER_API_KEY | Required for smart hook and chapter generation. Set in a .env file next to pyproject.toml or export in your shell. Without it, AI enhancement steps are skipped gracefully. Get a key at openrouter.ai/keys. | | OPENROUTER_REFERER | Optional. Shown in your OpenRouter usage dashboard for attribution tracking. |

License

MIT — see [LICENSE](LICENSE).

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: timkulbaev
• Source: timkulbaev/ai-video-editor
• License: MIT

Install and usage instructions live in the source repository linked above.