Storyboard Tts

Storyboard TTS

Take a [storyboard](../storyboard/SKILL.md) deliverable and batch-synthesize Chinese + English voice-over with IndexTTS2 (shared setup with [ai-text-to-speech](../ai-text-to-speech/SKILL.md)).

| Output | Path | |--------|------| | Chinese VO | /Chinese/.wav | | English VO | /English/.wav | | Duration doc | /speech-timeline.md | | Chinese subs | /Chinese.srt | | English subs | /English.srt |

Shot id from headers (### Shot 01 — … → 01.wav).

Subtitles: one SRT per language. Shots are laid end-to-end on the VO timeline (shot N starts when N−1 ends). Inside a shot, text is split on sentence punctuation (。！？；… / .!?) into multiple cues; cue lengths share that shot’s WAV duration by non-whitespace character weight. Skip (no VO) / missing audio.

Rules

1. Follow [skill-dependency-manager](../../rules/skill-dependency-manager.md).
2. Batch synthesis only via scripts/synthesize.py with the index-tts interpreter. Do not hand-write IndexTTS loops, temporary batch drivers, or N× single tts.py calls for a full storyboard.
3. Trial / single-line checks may use [ai-text-to-speech](../ai-text-to-speech/SKILL.md) tts.py, or synthesize.py --limit 1.
4. Parse-only / report-only / subtitle-only steps use stdlib python (.dependency/python/python).
5. Never overwrite the storyboard source. Write only under /.
6. Skip (no VO) / empty lines — no empty WAVs or empty subtitle cues.
7. Confirm voice reference (and output dir if unclear) before a full batch.

Inputs

| Required | Notes | |----------|--------| | Storyboard .md | ### Shot NN — title with - **Chinese:** / - **English:** | | Reference voice | WAV/MP3 for IndexTTS (--voice, or --voice-zh / --voice-en) |

| Optional | Default | |----------|---------| | Output dir | /-speech/ | | Language | both (--lang chinese / english) | | --fp16 / emotion / --device | same meaning as ai-text-to-speech | | --force | off (skip existing WAVs) | | --limit N | 0 = all jobs (use 1 for trial) | | --report | write speech-timeline.md + Chinese.srt / English.srt after synth | | --no-subtitles | with --report, skip SRT files |

Layout

/
  Chinese/
    01.wav
    …
  English/
    01.wav
    …
  shots.json
  speech-timeline.md
  Chinese.srt            # all Chinese cues, continuous timeline
  English.srt            # all English cues, continuous timeline
  _text/                 # only with --write-text

Workflow

Task Progress:
- [ ] Confirm storyboard path + voice (+ audio-dir if needed)
- [ ] Ensure index-tts populated (ai-text-to-speech Setup if missing)
- [ ] Optional trial: synthesize.py --limit 1 --fp16
- [ ] Full batch: synthesize.py --fp16 --report
- [ ] Chat: audio-dir, shot count, totals from speech-timeline.md, SRT paths

One command (preferred)

From project root (Windows; use .venv/bin/python on Unix):

.dependency/index-tts/.venv/Scripts/python.exe \
  .cursor/skills/storyboard-tts/scripts/synthesize.py \
  --storyboard path/to/storyboard.md \
  --voice path/to/ref.wav \
  --audio-dir path/to/-speech \
  --fp16 --report

This will:

1. Parse the storyboard → /shots.json
2. Load IndexTTS2 once
3. Write Chinese/.wav and English/.wav (skip existing unless --force)
4. Write speech-timeline.md and Chinese.srt / English.srt when --report

Trial one line

.dependency/index-tts/.venv/Scripts/python.exe \
  .cursor/skills/storyboard-tts/scripts/synthesize.py \
  --storyboard path/to/storyboard.md \
  --voice path/to/ref.wav \
  --audio-dir path/to/ \
  --fp16 --limit 1

Separate voices / language

# Chinese and English different refs
--voice-zh path/to/zh_ref.wav --voice-en path/to/en_ref.wav

# Only Chinese track
--lang chinese

Parse, report, or subtitles alone (stdlib python)

.dependency/python/python .cursor/skills/storyboard-tts/scripts/parse_storyboard.py \
  path/to/storyboard.md -o path/to//shots.json

.dependency/python/python .cursor/skills/storyboard-tts/scripts/duration_report.py \
  --storyboard path/to/storyboard.md \
  --audio-dir path/to/ \
  --shots path/to//shots.json \
  -o path/to//speech-timeline.md

# Subtitles only (after WAVs exist) — one Chinese.srt + one English.srt
.dependency/python/python .cursor/skills/storyboard-tts/scripts/write_subtitles.py \
  --audio-dir path/to/ \
  --shots path/to//shots.json

Resume from an existing shots.json:

.dependency/index-tts/.venv/Scripts/python.exe \
  .cursor/skills/storyboard-tts/scripts/synthesize.py \
  --shots path/to//shots.json \
  --voice path/to/ref.wav \
  --audio-dir path/to/ \
  --fp16 --report

Flags (synthesize.py)

| Flag | Notes | |------|--------| | --storyboard / --shots | Source (one required) | | --audio-dir | Output root (required) | | --voice | Shared speaker ref | | --voice-zh / --voice-en | Per-language refs | | --lang | both (default), chinese, english | | --limit N | First N pending jobs only | | --force | Overwrite existing WAVs | | --report | Write timeline + SRT subtitles | | --report-out | Custom timeline path | | --no-subtitles | Skip SRT when using --report | | --write-text | Dump lines under _text/ | | --fp16 / --device | Runtime | | --emotion-* / --random / --verbose | Same role as tts.py |

On partial failure: script continues remaining jobs, prints Failed jobs: …, exit code 1. Fix install/voice per ai-text-to-speech troubleshooting, re-run (existing OK files are skipped).

Agent notes

1. Do not invent narration — use storyboard Chinese/English fields as-is (audio and subtitles).
2. Prefer one synthesize.py invocation for a full board; model reload cost is the reason.
3. Chat summary: audio-dir, counts, path to speech-timeline.md, Chinese.srt / English.srt, Chinese/English total seconds — no full transcripts unless asked.
4. IndexTTS install lives in [ai-text-to-speech](../ai-text-to-speech/SKILL.md); do not duplicate Setup here beyond “populate index-tts if missing”.
5. Loudnorm / OGG / trim are separate skills after this one.
6. If audio already exists and only subtitles are needed, run write_subtitles.py alone (stdlib python).

Related

• [storyboard](../storyboard/SKILL.md) — source markdown
• [ai-text-to-speech](../ai-text-to-speech/SKILL.md) — single-line TTS + IndexTTS setup
• Optional after: [audio-loudness-normalization](../audio-loudness-normalization/SKILL.md), [audio-to-ogg](../audio-to-ogg/SKILL.md)

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: godot-fun
• Source: godot-fun/godot-framework
• License: MIT

Install and usage instructions live in the source repository linked above.