Music Video

music-video

Generate a 60-second 9:16 vertical music video from a music file + mood keywords. Designed for YouTube Shorts / TikTok / Reels upload.

What this produces

Given:

• A music file (mp3 / wav / m4a / aac) — typically 60–240 seconds of

operator-supplied music (Suno-generated, YouTube Audio Library, Pixabay, etc.). Music itself is the only audio track in the output.

• A short list of mood keywords (3–6 comma-separated phrases like

"rainy street, jazz cafe, vinyl, wet pavement").

Produces:

• A 1080 × 1920 (9:16 vertical) mp4, exactly 60 seconds.
• 8 B-roll clips fetched from Pexels (per-keyword), trimmed and

ordered to match phrase boundaries detected via aubiotrack.

• Drum-onset-aligned glitch micro-edits (aubioonset) on static-

camera clips only.

• Vintage lo-fi processing (film grain, vignetting, zoom-pulse) per

v6 defaults; tunable via env vars.

• Optional post-shader pass: pond (water-surface ripple),

breathing (5-s scale wave), halation (warm bloom), or combo (phrase-aware pond + halation envelope).

How to invoke

User-facing invocation: /music-video ""

Examples:

/music-video "assets/music/Rainy Bossa.mp3" "rainy street, jazz cafe, vinyl, wet pavement"
/music-video ~/Downloads/track.wav "tokyo neon, vibraphone, late night, shibuya"

If the user invokes without a path or without keywords, ask them for the missing input rather than guessing.

Step-by-step (what the agent does)

When this skill activates:

1. Verify the music file exists at the path the operator provided.

If not, ask for the correct path. Do not attempt to fetch music from the network — the operator supplies it.

1. Verify environment readiness. Run

./scripts/bootstrap.sh --check-only (or inspect the resulting warnings). In particular check that PEXELS_API_KEY is set in .env. If missing, point the operator at the Pexels signup link in the README and stop — do not proceed without B-roll source.

1. Generate a mission id in the form

skill-music-video-.

1. Run the bundled pipeline (script symlinked from the mature

agents/missions/music-video/run.sh so this skill inherits all v5 + v6 tuning):

``bash bash scripts/run.sh "" "" "" ``

The pipeline writes to records/missions//music-video--*/outputs/short.mp4 (see agents/missions/music-video/run.sh for the full stage breakdown — 8 stages including beat detection, per-window keyword resolution, Pexels fetch with caching, per-clip trim+crop+speed-class, concat, music overlay, lo-fi shading).

Typical runtime: 3–6 minutes for a 60-second output (depends on Pexels response latency and post-shader presence).

1. (Optional) Apply post-shader for the validated music-video

look. Default recommendation: combo (phrase-aware pond + halation envelope, tuned for a 95.8 BPM cadence by default — override GATE_POND and OPACITY env vars for other tempos):

``bash bash "$REPO_ROOT/scripts/music-video-shaders.sh" combo \ "" \ "" ``

1. Report the final mp4 path to the operator along with file size

and duration (use ffprobe to confirm 60.0 ± 0.5 s and 1080×1920). Suggest the operator move it to outputs/publish/ if they intend to upload (per operator-contract §8 outputs-publish exception).

What this skill does NOT do

• Generate the music itself (operator provides via Suno / YouTube

Audio Library / Pixabay / etc.).

• Upload to YouTube / TikTok / Reels (operator uploads manually

through the platform's UI — public URLs are intentionally not committed to this repo per the 2026-05-18 threat-model decision; see docs/pilots/first-upload-metrics.md "Public URL policy").

• Generate captions or burned-in text (music-video format is

intentionally text-free; for narrated shorts use the faceless-short mission instead).

Required environment

Pulled from .env (rendered locally; not committed):

• FFMPEG_BIN — set automatically by agents/lib/env.sh libass

discovery; only override if you need a specific build.

• FFPROBE_BIN — same discovery.
• OLLAMA_HOST — default 127.0.0.1:11434.
• OLLAMA_MODEL_HIGHLIGHT — the local model used for keyword

expansion (default llama3.2:3b).

• PEXELS_API_KEY — free signup at .
• RECORDS_DIR — defaults to ./records/.

Optional tuning env vars for the lo-fi processing layer (read by the pipeline at run time):

• MUSIC_VIDEO_FILM_GRAIN_INTENSITY
• MUSIC_VIDEO_VIGNETTE_ANGLE
• MUSIC_VIDEO_ZOOM_PULSE_AMP

See agents/missions/music-video/run.sh for the full env-var list and defaults.

See also

• Source pipeline (this skill wraps it):

agents/missions/music-video/run.sh

• Post-shaders: scripts/music-video-shaders.sh
• Engineering case study (why these shader choices, where the wall

is): docs/engineering-case-studies.md §5

• Decision log (niche pivot to music-video format, 2026-05-17):

docs/pilots/decision-log.md

• Daily upload queue (cron / launchd cadence):

scripts/daily-music-video.sh

Genre-aware mode (added 2026-05-21)

The v6 default treatment (drum-onset zoom-pulse + grain + vignette + 12-beat cuts) is applied regardless of song genre. Operator 2026-05-20 flagged that this reads as "띠용" / out-of-place on songs whose genre forbids glitch (lo-fi) or forbids cuts (ambient). See docs/research/2026-05-21-shader-song-mismatch-diagnosis.md.

The fix is additive — existing pipeline unchanged, new behavior opt-in via one of three entry points:

Recommended: all-in-one wrapper (zero-config)

bash scripts/music-video-auto.sh 

Detects genre from filename + ID3 tag, resolves preset from skills/music-video/data/genre-presets.yaml, fills keyword pool from preset, runs pipeline with genre-appropriate env overrides, applies matching post-shader. Add --with-canvas for an 8s Spotify Canvas variant or --with-typography "phrase1,phrase2,..." for a kinetic typography overlay.

For ambient / classical / dreamcore genres (stillzoom presets), pass --image=PATH.

Manual genre selection

bash scripts/music-video-genre.sh    [keywords_csv]

Bypass auto-detection. --list shows available genres.

Standalone shader pass

bash scripts/music-video-shaders.sh   

Effects available:

• Classic: pond breathing halation combo
• Genre-coded: scanline chromatic_split neon_edge vhs

saturation_pulse kaleidoscope

Adjacent output modes

• scripts/music-video-stillzoom.sh — image
• music → 60s slow Ken-Burns 9:16 mp4. Required for ambient /

classical / dreamcore presets where any cut violates the genre contract.

• scripts/music-video-canvas.sh —

Spotify Canvas 8s seamless loop @720×1280, "phrase1,phrase2,..."` — kinetic phrase overlay. Solves the muted-autoplay problem; 4-8 mood phrases fade in/out on evenly-spaced boundaries.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: MelonS
• Source: MelonS/MelonS-Agents
• License: MIT
• Homepage: https://melons.github.io/MelonS-Agents/

Install and usage instructions live in the source repository linked above.