Youtube Synthesizer

YouTube Synthesizer

Orchestrates youtube-transcript and youtube-screenshotter to produce an Obsidian literature note for a video. Drives image selection by value-filter — every candidate frame is classified by KIND, and only frames that communicate something the transcript words don't survive into the entry. All vision and reasoning runs on your own native capability — no separate API calls, no extra billing.

You — the agent running this skill — do the visual judgment inline as you follow the procedure below. The skill produces faithful capture of the source; downstream synthesis into the user's mental model is their job, not yours.

Invocation

Required inputs from the user:

• `` — YouTube video URL or 11-character video ID

Optional:

• **`** — absolute path to the target Obsidian vault, **or** a vault name relative to ~/ObsidianVaults/. The skill writes to /sources/videos/-/`. If omitted, see Phase D.1 for the discovery flow.
• --rerun — overwrite an existing entry at the target path. Without this flag, the skill refuses to overwrite (Phase D.2).
• --why — populate the why_ingested frontmatter field. If omitted, leave it as an empty string; the user can fill it in after.
• --no-sync — skip the post-write Dropbox sync prompt (Phase D.6). Otherwise the skill asks after a successful write whether to push the new entry to dropbox:ObsidianVaults//sources/videos//.

If ` is missing, ask the user before starting work — do not guess. ` resolution is described in Phase D.1; do not guess that either.

Output

A single Obsidian literature note at /sources/videos//-/ containing:

• .md — frontmatter + body (TL;DR, key takeaways, chapter-level prose with inline images, references, commands & code). Same `` as the parent directory's title segment, without the date prefix (the directory carries the date already).
• image-NN.png siblings — one per kept informational frame, in source order

Grouping by channel keeps a creator's videos together — opening /sources/videos/the-grainbound/ shows every Grainbound entry side-by-side.

---

The value-filter principle

> Images appear in the wiki entry only when they communicate something the transcript words don't, or when they help explain the narrative — even if that means several in a row.

The skill does not treat visual change as the trigger for capturing a "scene." A typical explainer video is ~70% talking-head plus filler (title cards, pull-quote text cards, between-content transitions) and ~30% load-bearing visuals (diagrams, maps, charts, code, UI, b-roll). Only the load-bearing 30% should appear as images in the entry. Number of images per video is whatever the source has of substance: 0 in a pure-talking-head section, 4-in-a-row during an animated diagram explanation.

Phase A produces an ordered list of (timestamp, frame_path, kind) triples for the kept frames; Phase D renders them as inline image embeds inside chapter-level prose.

Frame kinds

For each candidate frame, classify as exactly one of:

Drop (never embed):

• talking-head — presenter's face/torso with no overlaid content
• blank — empty background, between-content transition frame, near-uniform pixels
• title-card — text-only on illustrated background, where the text duplicates the transcript ("THE OXEN PARADOX," "CHAPTER 3," "THREE DAY RULE")
• pull-quote-card — emphasized restatement of what the presenter just said, with no additional information

Conditional (keep only when informational):

• map-overview — establishing shot of geography with no annotations or overlays. Drop unless the surrounding transcript prose specifically discusses "this region" or "this part of the map," or it is the very first map shown (intro establisher). Most repeat-views of the same world map are noise.

Keep (embed at the frame's timestamp):

• diagram — abstract illustration explaining a concept (stick-figures, flowcharts, conceptual visuals)
• map-with-overlay — map plus drawn-in markers (red dots, dashed lines, labels, route arrows, radius circles, etc.). The overlays carry the information.
• chart — data visualization (bar/line/pie/scatter, tables, math plots)
• code — source code, terminal output, command lines, config files
• ui — software interface, dashboard, application screenshot
• b-roll — real-world footage that is itself the subject (e.g. demonstrating a tool's physical use)
• animated-build-step — one frame of a multi-step animated diagram where each step adds information; keep all of them, do not collapse to the terminal frame
• hybrid-with-info-overlay — talking-head with a substantive overlay (chart, code, callout) that adds information beyond what the words convey

When uncertain between drop-kinds and keep-kinds, prefer keep. Wasting one frame downstream is cheap; missing a real informational moment is expensive. When uncertain between two keep-kinds, pick the closer match — the kind tag is informational metadata only and does not change rendering in Phase D. When uncertain between map-overview and map-with-overlay, default to map-with-overlay (keep) — visible overlays the agent missed are still informational.

---

Phase A — Value-filter sampling

Produces an ordered list of (timestamp, frame_path, kind) triples for the kept frames.

A.1. Fetch transcript and metadata

• /get_transcript.py "" → transcript with timestamps
• /video.py "" --no-download → metadata (channelid, chaptermarkers, description, etc.)

If the video has no transcript, error with a clear message and stop. Do not fabricate one.

Save the metadata block; Phase B reads description, chapter_markers, source_published_date, etc. from it.

A.2. Discover candidate timestamps

/discover.py "" -o 

Returns a manifest with two source signals (scene_detect and phash_runs) and a unioned candidates list. Typical output for an 18-minute video is 60–140 candidates, depending on visual density.

The candidate list is high-recall: every sustained content stretch ≥ 3 seconds plus every sharp cut shows up. Most candidates will be drop-kinds (talking-head, title-cards) — the agent's job is to filter them down to the informational subset in A.4.

A.3. Extract candidate frames at full resolution

/extract.py "" -t  -t  ... -o 

Pass the timestamp list from discover.py's candidates array. The video file is already cached (discover downloaded it); only the per-timestamp PNG extraction happens here.

A.4. Classify every frame by kind

Delegate this to sub-agents — do not load the frames into your own context. A 100+ candidate video produces ~50–150 MB of PNG data; loading it inline blows the main agent's context window for no benefit (you only need the classifications back, not the pixels). Use the Agent tool (subagent_type: general-purpose) with batches of 10 frames per agent.

For each batch, the sub-agent's job is:

1. Read each frame image in the batch (Read tool on frame_path)
2. Read the transcript text spoken in a small window around each frame's timestamp (e.g. ±5 seconds — the main agent should pass the relevant transcript snippet to the sub-agent in the prompt so the sub-agent doesn't have to re-derive it)
3. Pick exactly one kind per frame from the value-filter principle above
4. Return a JSON array [{timestamp, frame_path, kind, note}, ...] covering every frame in the batch — drop-kinds tagged with kind=drop plus a brief drop_reason, keep-kinds with their kind tag and a one-line note describing what the frame shows

The main agent then collates the per-batch JSON into the running keep-list without ever reading the frames itself.

The sub-agent prompt should inline the kind taxonomy and the value-filter principle (the sub-agent has no memory of the SKILL.md). Keep batches at ≤ 10 frames so each agent stays inside the 32 MB request-size limit on its own image reads (frames at 1280×720 PNG run ~0.5–1.5 MB each, so 20 reads can blow the limit on dense content).

Use the discover.py source and run_duration hints. They're informational signals, not classification rules:

• source: phash_run with run_duration ≥ 5s → likely sustained content; if the transcript at that timestamp doesn't explain a visual, often a sustained talking-head pose (drop).
• source: phash_run with run_duration 3–5s + transcript hint of visual content ("as you can see," "here's") → likely a real diagram.
• source: scene_detect with no run_duration → sharp cut; could be diagram, transition, or pose change.

Use transcript context. The intended visual content is often signaled by the words around it:

• "As you can see," "let me show you," "here's the diagram" → expect a diagram/chart/map-with-overlay/ui/code frame
• "And now we add," "next we get," "step three" → expect animated-build-step
• Continuing to discuss the same conceptual point with no visual cues → likely talking-head (drop)

When a frame is genuinely ambiguous between a keep-kind and talking-head, prefer keep.

A.5. Build-up sequence detection

After the per-frame classification, walk the keep-list in time order. Detect animated build-ups by adjacent same-kind frames whose pHashes are similar-but-distinct:

For each consecutive pair (K_i, K_{i+1}) in the keep-list:

• If both are the same keep-kind (e.g. both diagram or both map-with-overlay)
• AND their pHashes have Hamming distance in the range 10–25 (visually related but not identical)
• AND K_{i+1}.timestamp - K_i.timestamp ≤ 10 seconds

…then this is an animated build-up sequence. The presenter is layering information onto the same canvas (e.g. wheat → wheat+sheep → wheat+sheep+wool, or a map gaining radii circles).

Re-extract intermediate seconds inside the sequence. Sample every 2 seconds between K_i.timestamp and K_{i+1}.timestamp via extract.py, classify each, and keep all build-up steps in the keep-list. Re-tag the original frames as animated-build-step if appropriate.

A.6. Cross-time pHash dedup

Walk the entire keep-list (not just adjacent pairs) and drop any frame whose pHash is within Hamming ≤ 12 of an earlier-kept frame.

This catches the case where the same world-map appears at chapter 1's opening and at chapter 3's opening — visually identical but separated by minutes.

Exclude animated-build-step frames from this pass. A.5 deliberately keeps multiple frames of a single build-up sequence; their pHashes are similar by design, and re-pruning them here would undo A.5's work.

When a near-duplicate is detected, keep the earlier frame (it's the establisher) and drop the later one.

A.7. Whole-list audit pass

Re-read every frame in the keep-list as a curated set and ask, for each one, "is this image actually useful in the entry?" This is a separate pass from per-frame classification (A.4): the per-batch sub-agents in A.4 each see only their own 10–20 frames, so cross-batch redundancy and accumulated drift are invisible there.

Delegate this to a single audit sub-agent (Agent tool, subagent_type: general-purpose). The main agent passes the audit-agent the full keep-list (timestamp + frame_path + kind + note for every entry) and asks it to re-read the frames and return a JSON [{timestamp, decision: "keep"|"drop", reason}, ...]. The audit-agent reads the frames in its own context — the main agent never loads them. If the keep-list is large enough that even one sub-agent can't fit all the frames in 32 MB, partition by chapter or by index range and run several audit-agents in parallel.

For each frame, the audit-agent decides:

• Keep — the image communicates something specific (a concept, a build-up state, a map overlay, a piece of code). The entry is better with it than without it.
• Drop — the image is talking-head, a transition cartoon with no figures or labels, an empty title-card overlay, or visually redundant with another kept frame.

The audit-agent is told it is allowed to drop frames previously classified as keep-kinds — the audit's purpose is precisely to catch that drift. When dropping, prefer dropping later frames in a build-up sequence over earlier ones (the earlier frame usually carries the concept). For redundant maps, keep the one with the richest overlay.

The main agent then applies the audit-agent's decisions to produce the final keep-list.

A.8. Emit the final keep-list

Output a sorted list of (timestamp, frame_path, kind) triples. This is the input to Phases C and D.

The expected size depends entirely on the video's information density. A 1hr lecture-with-slides might produce 30–50 kept frames; a 20-minute video that's 80% talking-head might produce 4–6. There is no target count — the value-filter rule plus the audit pass set the bar.

---

Phase B — Frontmatter

This phase assembles the YAML frontmatter from the screenshotter's metadata block.

B.1. Generate the frontmatter dictionary

The frontmatter has two parts: a base schema shared with future article/pdf/substack source-skills (so the future sqlite3 indexer joins on a single contract), and YouTube-specific extras alongside.

Base schema:

source_type: youtube
source_url: 
source_title: 
source_author:            # channel display name for YouTube
source_published_date:    # YYYY-MM-DD
source_description: |
  
ingested_date: 
tags: []           # populated by Phase C
topics: []         # populated by Phase C
why_ingested: ""   # optional; from --why flag at invocation time

source_description is the verbatim author-written description. It is not rendered in the body — it would duplicate TL;DR / takeaways. It lives in frontmatter only, where it serves two purposes:

1. Hallucination check — Phase C compares the agent-generated TL;DR against this prose. If the TL;DR introduces framing not grounded in either the transcript or the description, the agent has drifted.
2. Searchable reference — the user (or a future re-run with an updated SKILL.md) can compare against the original.

YouTube-specific extras (alongside the base, not nested):

video_id: 
channel:             # mirror of source_author for in-context clarity
channel_id:       # stable handle, survives channel renames
playlist_id:     # null unless invoked via playlist URL
playlist_title: 
duration_seconds: 
chapter_markers:                       # if present in metadata; otherwise omit
  - title: "..."
    start_time: 0.0
    end_time: 153.0
transcript_corrections: []  # populated by Phase C / Phase D as soft-corrects are applied

transcript_corrections is a list of {original, corrected, timestamp} entries logging every soft-correct the synthesizer applied to the auto-caption text. Auditable; lets a future linter spot systematic captioner failures across the corpus. Empty list if no corrections were applied.

channel_id is the key for future "process all videos from this channel and cross-link them" runs — channels can rename themselves, IDs are stable.

Frontmatter discipline: strict key-value, no narrative prose. Structured data (lists of objects) is fine — chapter_markers and transcript_corrections are both lists of dicts. Narrative content goes in the body, never in frontmatter.

---

Phase C — Structured extraction

Produces the body sections of the wiki entry plus the tags / topics frontmatter and the transcript_corrections log. Source-faithful: extract what the video communicated; do not interpret or synthesize. Use the transcript text, the video description, and (for Commands & Code) what's visible in kept frames.

C.1. TL;DR

1–2 sentences capturing the video's central claim or thesis. The highest-leverage section for re-discovery six months later — the user reads this to remember what the video was about.

Source: transcript primarily. Look for explicit thesis statements (often in the first 60s and the final 60

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: devonjones
• Source: devonjones/devon-claude-skills
• License: MIT

Install and usage instructions live in the source repository linked above.