# Gemini Count In Video

> Analyze and count objects in videos using Google Gemini API (object counting, pedestrian detection, vehicle tracking, and surveillance video analysis).

- **Type:** Skill
- **Install:** `agentstack add skill-benchflow-ai-skillsbench-gemini-count-in-video`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [benchflow-ai](https://agentstack.voostack.com/s/benchflow-ai)
- **Installs:** 0
- **Category:** [Content & Media](https://agentstack.voostack.com/c/content-and-media)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [benchflow-ai](https://github.com/benchflow-ai)
- **Source:** https://github.com/benchflow-ai/skillsbench/tree/main/tasks-extra/pedestrian-traffic-counting/environment/skills/gemini-count-in-video
- **Website:** https://www.skillsbench.ai

## Install

```sh
agentstack add skill-benchflow-ai-skillsbench-gemini-count-in-video
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Gemini Video Understanding Skill

## Purpose
This skill enables video analysis and object counting using the Google Gemini API, with a focus on **counting pedestrians**, **detecting objects**, **tracking movement**, and **analyzing surveillance footage**. It supports precise prompting for differentiated counting (e.g., pedestrians vs cyclists vs vehicles).

## When to Use
- Counting pedestrians, vehicles, or other objects in surveillance videos
- Distinguishing between different types of objects (walkers vs cyclists, cars vs trucks)
- Analyzing traffic patterns and movement through a scene
- Processing multiple videos for batch object counting
- Extracting structured count data from video footage

## Required Libraries

The following Python libraries are required:

```python
from google import genai
from google.genai import types
import os
import time
```

## Input Requirements
- **File formats**: MP4, MPEG, MOV, AVI, FLV, MPG, WebM, WMV, 3GPP
- **Size constraints**:
  - Use inline bytes for small files (rule of thumb: 20MB)
myfile = client.files.upload(file="surveillance.mp4")

# Wait for processing
while myfile.state.name == "PROCESSING":
    time.sleep(5)
    myfile = client.files.get(name=myfile.name)

if myfile.state.name == "FAILED":
    raise ValueError("Video processing failed")

# Prompt for counting pedestrians with clear exclusion criteria
prompt = """Count the total number of pedestrians who are WALKING through the scene in this surveillance video.

IMPORTANT RULES:
- ONLY count people who are walking on foot
- DO NOT count people riding bicycles
- DO NOT count people driving cars or other vehicles
- Count each unique pedestrian only once, even if they appear in multiple frames

Provide your answer as a single integer number representing the total count of pedestrians.
Answer with just the number, nothing else.
Your answer should be enclosed in  and  tags, such as 5.
"""

response = client.models.generate_content(
    model="gemini-2.0-flash-exp",
    contents=[prompt, myfile],
)

# Parse the response
response_text = response.text.strip()
match = re.search(r"(\d+)", response_text)
if match:
    count = int(match.group(1))
    print(f"Pedestrian count: {count}")
else:
    print("Could not parse count from response")
```

### Batch Processing Multiple Videos

```python
from google import genai
import os
import time
import re

def upload_and_wait(client, file_path: str, max_wait_s: int = 300):
    """Upload video and wait for processing."""
    myfile = client.files.upload(file=file_path)
    waited = 0
    
    while myfile.state.name == "PROCESSING" and waited 
Cyclists: 
Vehicles: 
"""

response = client.models.generate_content(
    model="gemini-2.0-flash-exp",
    contents=[prompt, myfile],
)

# Parse multiple counts
text = response.text
pedestrians = int(re.search(r'Pedestrians:\s*(\d+)', text).group(1))
cyclists = int(re.search(r'Cyclists:\s*(\d+)', text).group(1))
vehicles = int(re.search(r'Vehicles:\s*(\d+)', text).group(1))
```

### Using Answer Tags for Reliable Parsing

```python
# Request structured output with XML-like tags
prompt = """Count the total number of pedestrians walking through the scene.

You should reason and think step by step. Provide your answer as a single integer.
Your answer should be enclosed in  and  tags, such as 5.
"""

response = client.models.generate_content(
    model="gemini-2.0-flash-exp",
    contents=[prompt, myfile],
)

# Robust extraction
match = re.search(r"(\d+)", response.text)
if match:
    count = int(match.group(1))
else:
    # Fallback: try to find any number in response
    numbers = re.findall(r'\d+', response.text)
    count = int(numbers[0]) if numbers else 0
```

## Best Practices
- **Use the File API** for all surveillance videos (typically >20MB) and always wait for processing to complete.
- **Be specific in prompts**: Clearly define what to count and what to exclude (e.g., "walking pedestrians only, not cyclists").
- **Use structured output formats**: Request answers in specific formats (like `N`) for reliable parsing.
- **Ask for reasoning**: Include "think step by step" to improve counting accuracy.
- **Handle edge cases**: Specify rules for partial appearances, people entering/exiting frame, and mode changes.
- **Use gemini-2.0-flash-exp or gemini-2.5-flash**: These models provide good balance of speed and accuracy for object counting.
- **Test with sample videos**: Verify prompt effectiveness on representative samples before batch processing.

## Error Handling

```python
import time

def upload_and_wait(client, file_path: str, max_wait_s: int = 300):
    """Upload video and wait for processing with timeout."""
    myfile = client.files.upload(file=file_path)
    waited = 0

    while myfile.state.name == "PROCESSING" and waited  tags."""
        
        response = client.models.generate_content(
            model="gemini-2.0-flash-exp",
            contents=[prompt, myfile],
        )
        
        # Try structured parsing first
        match = re.search(r"(\d+)", response.text)
        if match:
            return int(match.group(1))
        
        # Fallback to any number found
        numbers = re.findall(r'\d+', response.text)
        if numbers:
            return int(numbers[0])
        
        print(f"Warning: Could not parse count, defaulting to 0")
        return 0
        
    except Exception as e:
        print(f"Error processing video: {e}")
        return 0
```

Common issues:
- **Upload processing stuck**: Use timeout logic and fail gracefully after max wait time
- **Ambiguous responses**: Use structured output tags like `` for reliable parsing
- **Rate limits**: Add retry logic with exponential backoff for batch processing
- **Inconsistent counts**: Be very explicit in prompts about counting rules and exclusions

## Limitations
- Counting accuracy depends on video quality, camera angle, and object size/distance
- Very crowded scenes may have higher counting variance
- Occlusion (objects blocking each other) can affect accuracy
- Long videos require longer processing times (typically 5-30 seconds per video)
- The model may occasionally misclassify similar objects (e.g., motorcyclist as cyclist)
- For highest accuracy, use clear prompts with explicit inclusion/exclusion criteria

## Version History
- **1.0.0** (2026-01-21): Tailored for pedestrian traffic counting with focus on object counting, differentiation, and batch processing

## Resources

- [Video API Docs](https://ai.google.dev/gemini-api/docs/video-understanding)
- [Get API Key](https://aistudio.google.com/apikey)
- [Pricing](https://ai.google.dev/pricing)
- [Python SDK Documentation](https://ai.google.dev/gemini-api/docs/quickstart?lang=python)

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [benchflow-ai](https://github.com/benchflow-ai)
- **Source:** [benchflow-ai/skillsbench](https://github.com/benchflow-ai/skillsbench)
- **License:** Apache-2.0
- **Homepage:** https://www.skillsbench.ai

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** yes
- **Shell / process execution:** no
- **Environment & secrets:** yes
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-benchflow-ai-skillsbench-gemini-count-in-video
- Seller: https://agentstack.voostack.com/s/benchflow-ai
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.