Video Production

Video Production Skill

End-to-end pipeline for producing AI-narrated educational videos with slides. Distilled from producing 40+ real videos for an AI-run YouTube channel (蝦說 AI), including every mistake we only made once.

Who this is for: an AI agent (Claude, Codex, Gemini, or any coding agent) asked to produce a narrated slide video. A human can follow it too.

Quick Start

0. Read references/teaching-style.md + references/narration-style.md (content quality rules)
1. Create project directory and cd into it; copy config.json from references/config-example.json and fill it in
2. Write narration.json (the script — one string per slide)
3. Generate slides (Path A: gpt-image-2 hand-drawn / Path B: HTML + Playwright screenshot)
4. Visually inspect every slide PNG (wrong characters / clipping / unreadable fonts)
5. TTS narration → MP3 (with built-in ASR verification)   → node scripts/tts_with_asr.js
6. FFmpeg assemble → video.mp4                            → node scripts/assemble.js
7. Quality check (bitrate + frame extraction + visual)
8. Subtitles → SRT (+ optional burn-in)                   → node scripts/gen_subtitles.js
9. Cover image → thumbnail                                → python scripts/cover_gen.py
10. Upload wherever you publish; verify the thumbnail and visibility after upload

Requirements

• Node.js ≥ 18 (scripts use only built-ins + playwright for HTML screenshots)
• Python ≥ 3.9 (only for gpt-image-2 slide/cover generation and optional rescore)
• FFmpeg + FFprobe on PATH (or set explicit paths in config.json)
• API keys (environment variables, or a .env file in the project directory):
• ELEVENLABS_API_KEY — TTS
• OPENAI_API_KEY — Whisper ASR verification + gpt-image-2 slides/cover
• A TTS voice: set tts.voiceId in config.json (any voice from your ElevenLabs

voice library — a premade voice works out of the box; a cloned voice makes it yours).

> ⚠️ Never hardcode API keys in scripts and never commit them to git. One of our keys > was auto-revoked from a public repo push and 32 videos rendered silent. Environment > variables only.

---

Configuration

Create config.json in your project directory (start from references/config-example.json):

{
  "tts": {
    "provider": "elevenlabs",
    "voiceId": "YOUR_ELEVENLABS_VOICE_ID",
    "model": "eleven_multilingual_v2",
    "maxRetries": 5
  },
  "asr": {
    "provider": "openai",
    "model": "whisper-1",
    "language": "zh",
    "passThreshold": 0.85
  },
  "video": {
    "width": 1920,
    "height": 1080,
    "audioBitrate": "192k",
    "slidePadding": 1.0
  },
  "branding": {
    "watermark": "🎬 Your Channel Name"
  },
  "ffmpeg": "ffmpeg",
  "ffprobe": "ffprobe"
}

---

Checklist (mandatory, do not skip steps)

□ 0. Read references/teaching-style.md — content/teaching rules (pre-flight constitution)
□ 1. Write narration.json
□ 1.5 ⭐ ALIGNMENT CHECK: narration entries count == slide count. MUST be equal!
     If narration splits a topic across 2 entries, there must be 2 separate slides.
     Mismatch = audio/visual desync for every slide after the mismatch point.
□ 2. Generate slides (Path A or B below)
□ 3. ⭐ Visual check: open every slide PNG with an image tool
     (wrong/garbled characters? clipped edges? readable on a phone?)
□ 4. TTS synthesis + ⭐ ASR verification (similarity ≥ 0.85, never skip)
□ 5. FFmpeg assemble → video.mp4
□ 6. ⭐ Quality check: ffprobe audio bitrate + extract a frame + visual verify
□ 7. Subtitles → subtitles_aligned.srt (external track and/or burn-in)
□ 8. ⭐ Cover image → thumbnail (16:9; pad, don't crop)
□ 9. Upload (start unlisted/private, review, then publish)
□ 10. ⭐ After upload: verify the thumbnail actually shows YOUR cover, not an auto frame

The ⭐ steps are non-negotiable. Each one exists because skipping it once ruined an entire batch of videos.

> 🔴 The cover is part of the video, not an optional extra. We once shipped a video > where every step was done except the cover — it went live with a default gray frame. > "No cover = not shipped."

---

Step 1: Script (narration.json)

A JSON array of strings, one per slide (CRITICAL: array length MUST equal slide count):

[
  "開場白，吸引注意力...",
  "第一個重點，配合比喻...",
  "第二個重點...",
  "結尾，call to action..."
]

Writing guidelines

• 80–150 characters per slide (Chinese). Too short = rushed; too long = TTS breaks.
• Conversational tone: metaphors, rhetorical questions, humor.
• One concept per slide. Don't cram.
• Story > Information. Viewers remember stories, not bullet lists.
• Full teaching methodology (audience empathy, glossing jargon, honest attribution,

first-person endings…): references/teaching-style.md. Voice & TTS-safe writing: references/narration-style.md.

TTS-friendly writing (important!)

TTS engines mispronounce things. Prevent it at the script stage:

| Problem | Fix | |---------|-----| | English abbreviations (LLM, NLP) | Spell out in Chinese, or letters with periods (P.U.A.) | | Raw numbers (135,000) | Chinese numerals (十三萬五千) | | Version numbers (4.5) | Display text keeps 4.5, TTS text gets 四點五 | | Long sentences (>50 chars) | Split at natural breath points | | Parenthetical content | Rewrite as natural speech | | Chinese heteronyms 破音字 (還/重/長/得…) | Scan with references/heteronyms.json, rewrite |

Punctuation and pacing: our TTS script strips punctuation before synthesis (each punctuation mark becomes a pause in Chinese TTS — dense commas produce machine-gun narration). Write narration that flows in breath-groups of ~8–22 characters; put commas only at real breath points. Subtitles still use the original punctuated text.

---

Step 2: Slides

Two first-class paths. Pick ONE per video (Path A is our channel default; Path B has no image-API dependency).

Path A — gpt-image-2 hand-drawn teaching style (scripts/slides_gen.py)

Full-bleed AI-generated slides in a "professor's hand-drawn lecture notes" style: white background, bold black CJK title top-left with underline, thin black arrows, lots of whitespace, a small mascot in the corner, stick figures only (no real faces).

1. Write one Chinese prompt per slide into slides_prompts.json (array of strings).

Wrap every string that must appear on the slide in 「」. Always end the shared style block with: 「所有中文字必須完全正確、清楚可讀、不可有亂碼或錯字。數字要正確。」

1. python scripts/slides_gen.py → generates slides_raw/slide_NN.png (1536×1024).

Batch ≤4–5 concurrent lanes (~120s per image). Do not run Whisper at the same time — concurrent heavy API calls starve each other.

1. Visually inspect every image (garbled characters / wrong or invented numbers /

typos → regenerate just that slide; HTTP 502 → just retry that slide). gpt-image-2 WILL invent numbers to fill tables unless your prompt explicitly says 「畫面只能出現 X 這幾個數字，其他留空」.

1. node scripts/pad_and_burn.js pad → 1536×1024 scaled to 1410×940, padded onto a

1920×1080 white canvas with a 140px bottom band reserved for subtitles.

Path B — HTML slides + Playwright screenshot (scripts/screenshot.js)

Create one HTML file per slide: slides/slide_01.html, slide_02.html, … (template: references/slide-template.html), then node scripts/screenshot.js.

Design rules (hard-won; mobile viewers are the majority):

| Element | Minimum font size | |---------|-------------------| | Main title | ≥72px | | Subtitle / section header | ≥36px | | Body text | ≥32px (absolute floor) | | Key numbers | ≥48px | | Watermark | ≥20px (only element allowed smaller) |

• Resolution = video.width × video.height (default 1920×1080).
• Fill 80%+ of the canvas — padding: 60-80px, flexbox centering, no floating cards.
• Every element on the slide must be mentioned in the narration (and vice versa).
• 🔴 Never use more points in tiny font.

Path C — Node Canvas fallback (scripts/generate_slides.js)

No browser, no image API: renders simple title+bullets slides from slides.json. Use only when Playwright and the image API are both unavailable.

---

Step 3: TTS + ASR Verification (scripts/tts_with_asr.js)

node scripts/tts_with_asr.js [project_dir]

Reads narration.json, synthesizes each entry via ElevenLabs, saves audio/slide_XX.mp3, then verifies every clip with Whisper ASR:

1. Transcribe the generated audio
2. Compute character-overlap similarity vs the original text
3. ≥ 0.85 = PASS (target ≥0.90 for important videos)
4. 🔴 Mixing clips from different sources (e.g. your narrated slides + a real screen

> recording)? Do NOT use the concat demuxer with -c copy — strict players (Windows > Media Player) refuse to play the result. Re-encode through the concat FILTER into one > continuous stream, normalize every input (scale, fps=30, format=yuv420p, > 44100 stereo audio), re-encode audio, add faststart, and align loudness > (loudnorm=I=-16:TP=-1.5:LRA=11). Details in references/lessons-learned.md.

> 🔴 Mixing two TTS providers in one video? Their sample rates differ (ElevenLabs > 44100 Hz vs OpenAI TTS 24000 Hz). Resample everything to 44100 BEFORE assembly or > some players choke exactly at the voice-switch point.

---

Step 5: Quality Check (three checks, all required)

5a. Audio bitrate

ffprobe -v error -show_entries stream=codec_type,codec_name,bit_rate -of default video.mp4

✅ ~130–192 kbps AAC ❌ ~2 kbps or N/A = TTS failed silently

5b. Visual verification

ffmpeg -ss 3 -i video.mp4 -frames:v 1 -update 1 verify.png

Compare verify.png against slides/slide_01.png with an image tool. Content must match.

5c. Font size check

Inspect verify.png: is every text element readable at mobile size?

---

Step 6: Subtitles (scripts/gen_subtitles.js)

node scripts/gen_subtitles.js [project_dir]

Produces subtitles_aligned.srt: Whisper word timestamps for timing, original narration text for display (never use ASR output as subtitle text — it mishears). Line breaks are width-aware (CJK=1, Latin=0.5, ≤16 full-width per line) and never cut inside an English word.

> ⚠️ The #1 subtitle bug: drift from assuming clip duration = audio + padding. > FFmpeg's -shortest truncates the padding, so real clip duration ≈ audio duration. > The script therefore reads ACTUAL durations from temp/clip_XX.mp4 with ffprobe. > If you ever hand-roll offsets: never offset += audioDur + padding — that drifts > +1s per slide and by slide 16 subtitles are 15 seconds late.

Ship options:

• External SRT track (recommended when your platform supports it — viewers can

toggle, nothing covers the slides)

• Burn-in: node scripts/pad_and_burn.js burn (Path A white-band layout: dark text

FontSize 30 / MarginV 30 sits inside the reserved 140px band) For full-bleed dark HTML slides use FontSize=14–18, MarginV=6–12, BorderStyle=3 instead.

For rough/cloned voices where Whisper timestamps collapse, there is a geometry-based fallback (ffmpeg silencedetect + character-width proportional alignment) — see references/lessons-learned.md § subtitle alignment.

---

Step 7: Cover / Thumbnail (scripts/cover_gen.py)

python scripts/cover_gen.py "一張 YouTube 影片封面，橫式 16:9，白底手繪教學風…主標題用超大粗黑體繁體中文寫「你的標題」…"

• Generate at 1536×1024, then pad — do not crop — to 1280×720:

crop cuts off the top of your title. scale=1080:720 + pad=1280:720:100:0:color=white (white side bars are invisible on a white cover).

• Target ≤2MB for YouTube.
• Same prompt hygiene as slides: quote exact text in 「」, forbid invented numbers,

visually verify before shipping.

---

Step 8: Upload

Platform-specific; do it however you normally operate (browser automation, manual, CLI). Regardless of method, these rules survived contact with reality:

1. Upload as unlisted/private first, review the actual playback, then publish.
2. Verify the thumbnail after upload — the preview must show YOUR cover, not an

auto-selected frame.

1. Re-check title/description/language settings after any dialog reopens (some upload

wizards silently reset radio buttons).

1. If you used an external SRT: upload it as a caption track ("with timing"), publish

the track, then verify captions actually appear on the watch page.

1. Disclose AI authorship in the description if the narration/production is AI-made.

---

Output Structure

my-video-project/
├── config.json              ← project config (from references/config-example.json)
├── narration.json           ← script text per slide
├── slides_prompts.json      ← (Path A) one gpt-image-2 prompt per slide
├── slides_raw/              ← (Path A) raw 1536×1024 generations
├── slides/                  ← final 1920×1080 PNGs (padded or screenshotted)
│   ├── slide_01.png
│   └── ...
├── audio/
│   ├── slide_01.mp3
│   └── ...
├── temp/                    ← per-slide clips + whisper word caches
├── video.mp4                ← assembled (no subtitles)
├── subtitles_aligned.srt    ← aligned subtitle track
├── video_sub.mp4            ← (optional) burned-in version
└── thumbnail.jpg            ← cover, 1280×720

---

Scripts Reference

All scripts take the project directory as an optional first argument (default: CWD).

| Script | Purpose | |--------|---------| | scripts/slides_gen.py | gpt-image-2 slide generation from slides_prompts.json | | scripts/pad_and_burn.js | pad 3:2 images to 16:9 + subtitle band / burn SRT | | scripts/screenshot.js | Playwright HTML→PNG screenshots | | scripts/generate_slides.js | Node Canvas fallback slide renderer | | scripts/tts_with_asr.js | ElevenLabs TTS + Whisper ASR verification loop | | scripts/assemble.js | FFmpeg per-slide clips + concat | | scripts/gen_subtitles.js | aligned SRT (whisper timing + original text) | | scripts/rescore.py | homophone/digit-tolerant second-chance ASR scoring | | scripts/cover_gen.py | gpt-image-2 cover generation |

---

Sub-agents

If you spawn a sub-agent to produce a video, the task prompt must explicitly say "read the video-production SKILL.md first". Sub-agents that aren't told skip the pipeline and reinvent (worse) wheels. Re-read the skill fresh each session — it evolves.

When things break

references/lessons-learned.md is the accident report archive: path bugs, silent audio, subtitle drift, moderation blocks on image generation, players that refuse concat-copied files, Whisper false alarms, batch API contention, and more. Read it before your first video; grep it when something looks weird — the odds are good we already hit it.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: speechlab0210
• Source: speechlab0210/video-production-skill
• License: MIT

Install and usage instructions live in the source repository linked above.