Gemini Video Understanding

Gemini Video Understanding Skill

Purpose

This skill enables video understanding workflows using the Google Gemini API, including video summarization, question answering, transcription with optional visual descriptions, timestamp-based queries (MM:SS), scene/timeline detection, video clipping, custom FPS sampling, multi-video comparison, and YouTube URL analysis.

When to Use

• Summarizing a video into key points or chapters
• Answering questions about what happens at specific timestamps (MM:SS)
• Producing a transcript (optionally with visual context) and speaker labels
• Detecting scene changes or building a timeline of events
• Analyzing long videos by clipping to relevant segments or reducing FPS
• Comparing multiple videos (up to 10 videos on Gemini 2.5+)
• Analyzing public YouTube videos directly via URL

Required Libraries

The following Python libraries are required:

from google import genai
from google.genai import types
import os
import time

Input Requirements

• File formats: MP4, MPEG, MOV, AVI, FLV, MPG, WebM, WMV, 3GPP
• Size constraints:
• Use inline bytes for small files (rule of thumb: 20MB)

myfile = client.files.upload(file="video.mp4")

Wait for processing

while myfile.state.name == "PROCESSING": time.sleep(1) myfile = client.files.get(name=myfile.name)

if myfile.state.name == "FAILED": raise ValueError("Video processing failed")

response = client.models.generate_content( model="gemini-2.5-flash", contents=["Summarize this video in 3 key points", myfile], )

print(response.text)

### YouTube Video Analysis (Public Videos Only)

```python
from google import genai
from google.genai import types
import os

client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=[
        "Summarize the main topics discussed",
        types.Part.from_uri(
            uri="https://www.youtube.com/watch?v=VIDEO_ID",
            mime_type="video/mp4",
        ),
    ],
)

print(response.text)

Inline Video (20MB) and wait for processing to complete before analysis.

• Reduce token usage by clipping to the relevant segment and/or lowering FPS for static content.
• Improve accuracy by being explicit about the desired output (timestamps format, number of events, whether you want scene changes vs actions vs chapters).
• Use gemini-2.5-pro when you need highest-quality reasoning over complex, long, or visually dense videos.

Error Handling

import time

def upload_and_wait(client, file_path: str, max_wait_s: int = 300):
    myfile = client.files.upload(file=file_path)
    waited = 0

    while myfile.state.name == "PROCESSING" and waited < max_wait_s:
        time.sleep(5)
        waited += 5
        myfile = client.files.get(name=myfile.name)

    if myfile.state.name == "FAILED":
        raise ValueError(f"Video processing failed: {myfile.state.name}")
    if myfile.state.name == "PROCESSING":
        raise TimeoutError(f"Processing timeout after {max_wait_s}s")

    return myfile

Common issues:

• Upload processing stuck: wait and poll; fail after a max timeout.
• YouTube errors: verify the video is public and not age-restricted.
• Rate limits: retry with exponential backoff.
• Incorrect timestamps: re-prompt with strict “MM:SS” formatting and request fewer events.

Limitations

• Long-video support is limited by model context and token budget (default vs low-res modes).
• YouTube analysis requires public videos; live streaming analysis is not supported.
• Very long videos may require chunking (clip by time range and process in segments).
• Multi-video comparison is limited (up to 10 videos per request on Gemini 2.5+).

Version History

• 1.0.0 (2026-01-15): Initial release focused on Gemini video understanding (summaries, Q&A, timestamps, clipping, FPS control, YouTube, and structured outputs).

Resources

• Audio API Docs
• Image API Docs
• Video API Docs
• Document API Docs
• Image Gen Docs
• Get API Key
• Pricing

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: benchflow-ai
• Source: benchflow-ai/skillsbench
• License: Apache-2.0
• Homepage: https://www.skillsbench.ai

Install and usage instructions live in the source repository linked above.