Video To Shorts

Video → Shorts

Turn a long talking-head video into one or more polished vertical shorts (≤ 60s, 9:16, burned-in subtitles). The skill transcribes the source, auto-suggests the best moments, then cuts, crops, and subtitles a clip you approve.

The deliverable is a finished *-en.mp4 ready to upload by hand. No YouTube account, API key, or upload CLI needed.

When to use

• "Make shorts/reels/clips out of this podcast / interview / talk"
• "Find the best moments in this video and cut vertical clips"
• "Transcribe this and pull out quotable soundbites"
• Any long-form talking footage that needs to become short vertical content

Best for static, single- or two-person talking footage. Fast-cut montages, music videos, or heavy on-screen graphics need a human editor — flag that and stop.

One-time setup (cross-platform)

You need three tools: ffmpeg with libass (cut/crop/burn-in), Whisper (transcription), and optionally yt-dlp (only if the source is a URL).

ffmpeg — must include libass (for subtitle burn-in)

Check first: ffmpeg -version | grep -o libass. If it prints libass, you're done.

| OS | Install | |----|---------| | Linux | sudo apt install ffmpeg (Debian/Ubuntu builds include libass) — or brew install ffmpeg on Linuxbrew. | | Windows (WSL2) | Inside Ubuntu-on-WSL: sudo apt install ffmpeg. Do everything from the WSL shell, not PowerShell. | | macOS | ⚠️ Default Homebrew ffmpeg omits libass — the subtitles filter fails with a cryptic "No option name near …". Install a libass static build to ~/.local/bin: |

# macOS only — libass-enabled static ffmpeg
mkdir -p ~/.local/bin
curl -L https://evermeet.cx/ffmpeg/getrelease/ffmpeg/zip -o /tmp/ff.zip
unzip -o /tmp/ff.zip -d ~/.local/bin/
chmod +x ~/.local/bin/ffmpeg
~/.local/bin/ffmpeg -version | grep -E "libass|libfreetype"   # both must appear

On macOS, use ~/.local/bin/ffmpeg for the burn-in step only; the system ffmpeg is fine for cut/crop.

Whisper — transcription

| OS / hardware | Recommended | Install | |---------------|-------------|---------| | macOS (Apple Silicon) | mlx-whisper (GPU via MLX, fastest) | pip install mlx-whisper | | Linux + NVIDIA | faster-whisper (CUDA) | pip install faster-whisper | | Any (CPU fallback) | openai-whisper | pip install -U openai-whisper |

Models download lazily on first run (English base ≈ 140 MB; multilingual large-v3 ≈ 3 GB). Use a venv to keep it clean.

yt-dlp (only if source is a URL)

pip install -U yt-dlp. For unlisted/age-gated videos add --cookies-from-browser chrome.

Workflow

1. Get the source video

• Local file → use directly.
• URL → download:

``bash yt-dlp -f "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]" \ "" -o "/tmp/source.mp4" ``

• Note the source resolution (ffprobe -v error -select_streams v -show_entries stream=width,height -of csv=p=0 source.mp4) — you need it for the crop math.

2. Transcribe → SRT

Transcribe the whole source to an SRT with real timestamps. Pick the command for the platform:

# macOS (Apple Silicon)
mlx_whisper source.mp4 --model mlx-community/whisper-base-mlx \
  --language en --output-format srt --output-dir /tmp/transcript \
  --condition-on-previous-text False

# Linux + NVIDIA (faster-whisper via the CLI wrapper, or use the python API)
whisper-ctranslate2 source.mp4 --model base --language en \
  --output_format srt --output_dir /tmp/transcript --vad_filter True

# Any CPU (openai-whisper)
whisper source.mp4 --model base --language en \
  --output_format srt --output_dir /tmp/transcript

For non-English source, set --language accordingly (e.g. zh, ja, es). --condition-on-previous-text False (or --condition_on_previous_text False) reduces hallucination loops.

Quick clean-up: Whisper mangles proper nouns and often hallucinates text over intro music. Skim the SRT and fix names/products in any segment you plan to clip, and delete gibberish in the first block. A full proofread isn't needed yet — just enough to read the content.

3. Auto-suggest the best moments ← the core value

Read the full SRT and propose the strongest short-worthy segments. A good short moment is:

• Self-contained — makes sense with zero setup; doesn't reference "what I just said."
• A complete thought — starts at a sentence start, ends at a sentence end.
• Hooky — a surprising claim, a strong opinion, a vivid story, a counterintuitive insight, a crisp how-to, or a quotable one-liner.
• 15–30s ideal, ≤ 60s hard max (Shorts limit).

Present the top 5–8 candidates as a table the user can choose from:

| # | In–Out | Dur | Speaker | The hook (why it works) | |---|--------|-----|---------|--------------------------| | 1 | 12:04–12:31 | 27s | … | "…" — counterintuitive take on X |

Never use chapter markers as cut points — chapters mark a topic's start (with preamble), not the soundbite. Always anchor in/out to the actual SRT blocks where the line begins and ends.

If the user gave a topic/quote up front, find that segment instead (or in addition).

4. Confirm the segment & refine boundaries

Once the user picks a moment, confirm exact in/out from the SRT. For static single-camera footage, transcript timestamps are enough — skip ahead.

For footage with camera cuts / slides / multi-cam, keyframe-validate:

for t in START-2 START START+1 END-1 END END+2; do
  ffmpeg -y -ss  -i source.mp4 -frames:v 1 /tmp/kf_.png
done

Read each frame: mid-blink? mouth frozen mid-word? slide changing? Shift the boundary ±0.5–2s for a clean in/out. Output the refined timestamps.

5. Determine the vertical crop — never guess

Decide the 9:16 crop before cutting. This is a two-stage decision: first which side the speaker is on, then fine-tune the offset within that side.

a. Read the framing & pick the side. Extract a frame from the middle of the segment and look at it: ffmpeg -y -ss -i source.mp4 -frames:v 1 /tmp/mid.png Decide which side of the frame the speaker occupies — left, center, or right — and confirm the camera is static (pull a second frame elsewhere in the segment; if it pans/cuts, a fixed crop may not work). This narrows the offset range you sample in step c — no point testing the empty half of the frame.

b. Compute the crop size from the source height (ffprobe it first). Both crop dimensions and the x offset must be even — libx264 (yuv420p) rejects odd values:

• crop_w = height × 9 ÷ 16, rounded to the nearest even number.
• crop_h = height (full height, so y = 0).
• offset range: x runs from 0 (hard left) to source_width − crop_w (hard right); center is (source_width − crop_w) ÷ 2.

| Source | cropw × croph | left x | center x | right (max) x | |--------|-----------------|--------|----------|---------------| | 1920×1080 | 608×1080 | 0 | 656 | 1312 | | 1280×720 | 404×720 | 0 | 438 | 876 | | 3840×2160 (4K) | 1216×2160 | 0 | 1312 | 2624 |

c. Slice within the chosen side — take 3–5 screenshots at offsets spanning only that side, then present them and let the user pick. Composition depends on gaze direction (looking-room: leave space where they look), and what's in the background — an empty wall can be worse than a tighter, busier crop. The user decides. ``bash # Speaker on the LEFT of a 1920×1080 frame → sample left-half offsets: for x in 0 160 320 480 640; do ffmpeg -y -ss -i source.mp4 -frames:v 1 -vf "crop=608:1080:$x:0" /tmp/crop_$x.png done # Speaker on the RIGHT → sample right-half offsets up to max x (1312): # for x in 704 856 1008 1160 1312; do ... done # Centered speaker → sample around center (e.g. 496 576 656 736 816). ` Keep all offsets even. Substitute your computed crop_w`/max-x if the source isn't 1080p.

6. Cut & crop

Substitute your computed even crop_w:crop_h and x from step 2 (the 608:1080 below is the 1080p case):

ffmpeg -y -ss  -to  -i source.mp4 \
  -vf "crop=608:1080::0" \
  -c:v libx264 -preset medium -crf 20 \
  -c:a aac -b:a 128k \
  /tmp/clip.mp4

Verify: extract a frame, confirm framing, confirm duration ≤ 60s, and confirm the output is exactly your target dimensions (ffprobe -v error -select_streams v -show_entries stream=width,height -of csv=p=0 /tmp/clip.mp4) — if ffmpeg silently shaved a pixel, your crop_w was odd.

Optional — upscale to full 1080×1920 for max resolution: append ,scale=1080:1920:flags=lanczos to the -vf chain (a 608×1080 crop is true 9:16 but lower-res; platforms upscale it anyway, so this is optional).

7. Re-transcribe the CUT clip — don't offset the source SRT

⚠️ Do not zero/offset the source SRT timestamps — they drift and go out of sync by the end of the clip. Instead, run Whisper on the cut clip itself for accurate timings:

mlx_whisper /tmp/clip.mp4 --model mlx-community/whisper-base-mlx \
  --language en --output-format srt --output-dir /tmp/clip_srt \
  --condition-on-previous-text False
# (or the whisper / whisper-ctranslate2 equivalent for your platform)

Then write clip.srt using Whisper's timestamps but the clean text from the source SRT:

1. Read each Whisper block to identify what is said at that timestamp (its text may be garbled — use it as a phonetic guide, e.g. "a genetic engineering" = "agentic engineering").
2. Pull the matching clean wording from your reviewed source SRT.
3. Write clean text onto Whisper's timestamp boundaries; split/merge cues to read naturally.
4. Keep cues short — at the burn-in font size each line fits ~14–16 chars, so 2–4 short lines per cue, not long sentences.

8. Burn in subtitles

Shorts/Reels display caption tracks unreliably on mobile, so bake subs into the pixels. Keep the un-subtitled clip.mp4 so you can burn other languages later.

# macOS: use ~/.local/bin/ffmpeg   |   Linux/WSL: use ffmpeg
ffmpeg -y -i /tmp/clip.mp4 \
  -vf "subtitles=/tmp/clip_srt/clip.srt:force_style='FontName=Arial,FontSize=16,Bold=1,PrimaryColour=&H00FFFFFF,OutlineColour=&H00000000,BorderStyle=1,Outline=2,Shadow=0,Alignment=2,MarginV=60'" \
  -c:v libx264 -preset medium -crf 20 -c:a copy \
  /tmp/clip-en.mp4

Style choices (don't change without reason):

• FontSize=16 — mobile-legible (~44–48px on 1080; libass scales to its internal PlayResY of 288). Each line fits ~14–16 chars.
• Bold=1, Outline=2, Shadow=0 — readable on any backdrop, no shadow noise.
• Alignment=2, MarginV=60 — bottom-center, in the lower third, above the platform UI band. Bump MarginV to 80–100 if the bottom line gets clipped.
• -c:a copy — preserve speech audio, no re-encode.

9. Verify & deliver

• Extract frames at 3–5 timestamps and check subs are on-screen, in sync, not clipped:

ffmpeg -ss -i /tmp/clip-en.mp4 -frames:v 1 /tmp/check_.png

• On macOS, open /tmp/clip-en.mp4 for a real-time preview.
• Final output is -en.mp4. Report the path. Done — ready to upload by hand.

Output conventions

For each short, keep:

• .mp4 — cropped clip, no subs (reuse for other languages)
• -en.mp4 — burned-in final ← the deliverable
• .srt — the clip's subtitle file
• .md — cut metadata (source, in/out, duration, crop offset)

slug = kebab-case summary of the moment, e.g. capability-not-functionality.

Common mistakes

| Mistake | Do instead | |---------|------------| | Offsetting the source SRT to start at 0 | Re-transcribe the cut clip (step 7) — offsets drift | | Using chapter timestamps as cut points | Anchor to the actual SRT block where the line starts | | Guessing the crop position | First pick the speaker's side, then slice 3–5 offsets within that side; let the user pick | | Odd crop width/height (e.g. 405) | Round to even — libx264 errors or silently shifts a pixel otherwise | | Copy-pasting 608:1080 on a non-1080p source | Recompute crop_w from the source height (step 2 table) | | Long sentences on one subtitle cue | Short phrases, ~14–16 chars/line, 2–4 lines | | Burning subs with default macOS Homebrew ffmpeg | Use the libass static build (~/.local/bin/ffmpeg) | | Clip over 60s | Trim to ≤ 60s (Shorts limit); 15–30s is ideal | | Mapping source SRT blocks 1:1 onto Whisper timestamps | Read Whisper text to find what's actually said, then place clean text |

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: augchan42
• Source: augchan42/inkstone
• License: MIT
• Homepage: https://sixlines.online

Install and usage instructions live in the source repository linked above.