Replicate Video

What this skill does

Wraps the Replicate API for bytedance/seedance-2.0 so the agent can generate short videos from a prompt (and optional reference images, audios, or videos). The CLI prints exactly one machine-parseable line: RESULT: { ... }. Parse that JSON; ignore the rest of stdout/stderr (logs only).

The user already has REPLICATE_API_TOKEN set in the project .env. Do NOT ask them to paste it. The CLI loads .env automatically.

Files

• scripts/replicate_video.py — single-command CLI
• requirements.txt — replicate, python-dotenv

Install once: pip install -r .agents/skills/replicate-video/requirements.txt

Operating rules

1. One canonical command. Always call python .agents/skills/replicate-video/scripts/replicate_video.py generate .... Do not call the Replicate SDK inline from the agent.
2. Confirm before spending. Seedance runs cost real money and take 1–4 minutes per clip. Before the first generation in a turn, show the user the resolved input JSON (prompt, duration, resolution, aspect_ratio, reference image count) and ask for confirmation. After approval, you can run subsequent clips in the same turn without re-asking unless parameters change materially.
3. Defaults that match the user's reference run: aspect_ratio=16:9, duration=7 (seconds), resolution=1080p, generate_audio=false. Override only when the user asks.
4. Reference images. Accept either local file paths or http(s) URLs. The CLI uploads local files to Replicate automatically. Up to 9 references is fine (matches what Seedance accepts in practice).
5. Output location. Files land in ./output_videos/ at the repo root with name seedance_.mp4. Always print the local path AND the output.url() from Replicate.
6. Never invent the model version. Use the slug bytedance/seedance-2.0. If a future call needs a pinned version (owner/name:hash), pass it via --model.
7. Vertical / Shorts. For TikTok/Reels/Shorts, use --aspect-ratio 9:16. For YouTube landscape, 16:9 (default). For square social, 1:1.
8. Duration limits. Seedance accepts short clips (typically 3–10s). If the user asks for "a 30-second video", explain the limit and offer to generate multiple clips that can be stitched (the user has video_captioner.py, demo_video_composer.py, viral_video_maker.py, etc. in the repo for stitching).

CLI usage

Text-to-video

python .agents/skills/replicate-video/scripts/replicate_video.py generate \
  --prompt "An aerial drone shot pulling back over a misty pine forest at sunrise, cinematic, 35mm" \
  --aspect-ratio 16:9 \
  --duration 7 \
  --resolution 1080p

With reference images (image-to-video / style + subject conditioning)

python .agents/skills/replicate-video/scripts/replicate_video.py generate \
  --prompt "Aerial drone descends and enters through the front door of this inspection center" \
  --reference-image ./apex_inspection_center.jpg \
  --reference-image ./google_earth_overhead.png \
  --reference-image https://example.com/storefront.jpg \
  --duration 7 \
  --resolution 1080p \
  --seed 24353

With audio reference / generated audio

python .agents/skills/replicate-video/scripts/replicate_video.py generate \
  --prompt "..." \
  --generate-audio \
  --reference-audio ./voiceover.mp3

Output

The CLI's last stdout line is always:

RESULT: {"status":"succeeded","prediction_id":"...","output_url":"https://...mp4","local_path":"output_videos/seedance_1714857600.mp4","duration_s":7,"resolution":"1080p","aspect_ratio":"16:9","model":"bytedance/seedance-2.0"}

On failure:

RESULT: {"status":"failed","error":"","model":"bytedance/seedance-2.0"}

Prompting tips for Seedance

• Lead with camera move ("aerial drone shot pulling back", "slow dolly in", "static wide", "handheld follow").
• Then subject + environment + lighting/lens ("35mm cinematic, golden hour, shallow depth of field").
• For interior reveals from exterior shots, name the transition explicitly: "the camera approaches and enters through the front door; interior matches reference image 1".
• When passing reference images, tell the model what each reference is for: "reference 1 is the storefront, references 2–4 are interior, references 5+ are Google Earth overheads to guide the drone descent".
• Keep prompts under ~80 words. Seedance ignores very long prose.

When NOT to use this skill

• User wants talking-head / lip-synced video → use heygen_client.py instead.
• User wants screen recording / typing tutorial → use typing_video_maker.py / tutorial_composer.py.
• User wants stitched multi-clip ads with captions → generate clips here, then hand off to viral_video_maker.py / demo_video_composer.py.
• User wants OpenAI Sora specifically → use sora_standalone.py / sora2_character_consistency.py.

Troubleshooting

• RESULT.error contains 401 or Unauthenticated → REPLICATE_API_TOKEN is missing or invalid in .env.
• RESULT.error mentions NSFW or safety → Seedance refused the prompt; rewrite without flagged terms.
• Generation hangs >6 minutes → Replicate is queued; the CLI polls until done. Tell the user to wait or cancel with Ctrl+C and retry with a lower --resolution (720p).

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Bomx
• Source: Bomx/replicate-video-skill
• License: MIT

Install and usage instructions live in the source repository linked above.