# Ai Vision

> Multimodal UI understanding and single-step planning via OpenAI-compatible Responses APIs. Use when you need AIQuery/AIAssert and plan-next to extract UI element coordinates, validate UI assertions, summarize screenshots, or decide the next UI action from an image. External agents handle execution via adb/hdc and multi-step loops. Defaults to Doubao models but can be pointed at other multimodal p…

- **Type:** Skill
- **Install:** `agentstack add skill-httprunner-skills-ai-vision`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [httprunner](https://agentstack.voostack.com/s/httprunner)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [httprunner](https://github.com/httprunner)
- **Source:** https://github.com/httprunner/skills/tree/main/ai-vision

## Install

```sh
agentstack add skill-httprunner-skills-ai-vision
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# AI Vision

## Overview
This skill provides a standalone CLI to call multimodal models for UI querying, assertion, and **single-step** planning. It does not depend on device type; you supply a screenshot and receive structured output (coordinates, decisions, or next actions). Execution and multi-step loops are handled externally by agents using adb/hdc or other drivers. Prefer storing screenshots in `~/.eval/screenshots/` and add timestamps to avoid overwriting.

## Path Convention

Canonical install and execution directory: `~/.agents/skills/ai-vision/`. Run commands from this directory:

```bash
cd ~/.agents/skills/ai-vision
```

One-off (safe in scripts/loops from any working directory):

```bash
(cd ~/.agents/skills/ai-vision && npx tsx scripts/ai_vision.ts --help)
```

## Model Configuration
Default Doubao configuration via environment variables:
- `ARK_BASE_URL` (e.g. `https://ark.cn-beijing.volces.com/api/v3`)
- `ARK_API_KEY`
- `ARK_MODEL_NAME`

For non-Doubao providers, pass explicit flags:
- `--base-url`, `--api-key`, `--model`

Default model if none provided: `doubao-seed-1-6-vision-250815`.

## Script
Path: `scripts/ai_vision.ts`

Run with:
```bash
npx tsx scripts/ai_vision.ts --help
```

Log level (for troubleshooting raw model response):
```bash
npx tsx scripts/ai_vision.ts --log-level debug  [flags]
```

Output formatting:
- When `--log-json` is set, logs are emitted as JSON.
- Otherwise, the final result is pretty-printed JSON, and logs are colorized when TTY is available.

### AIQuery
```bash
npx tsx scripts/ai_vision.ts query \
  --screenshot ~/.eval/screenshots/ui_YYYYMMDD_HHMMSS.png \
  --prompt "请识别屏幕上的‘搜索’按钮，并返回其坐标"
```

### AIAssert
```bash
npx tsx scripts/ai_vision.ts assert \
  --screenshot ~/.eval/screenshots/ui_YYYYMMDD_HHMMSS.png \
  --prompt "当前页面包含搜索框"
```

### plan-next (single-step planning)
```bash
npx tsx scripts/ai_vision.ts plan-next \
  --screenshot ~/.eval/screenshots/ui_YYYYMMDD_HHMMSS.png \
  --prompt "点击放大镜图标进入搜索页"
```

## Output Notes
- `plan-next` returns a normalized next action with absolute pixel coordinates.
- If the model outputs relative coordinates (1000x1000), the script scales to screen pixels.
- Combine with adb/hdc actions (e.g., `adb shell input tap X Y`) for device control.
- Use `--log-level debug` to print the raw model response for troubleshooting.

## Default Models (Doubao)
- `doubao-seed-1-8-251228`
- `doubao-seed-1-6-vision-250815`

## References
- `references/doubao-api.md`

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [httprunner](https://github.com/httprunner)
- **Source:** [httprunner/skills](https://github.com/httprunner/skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-httprunner-skills-ai-vision
- Seller: https://agentstack.voostack.com/s/httprunner
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.