# Waza > **WORKFLOW SKILL** - Evaluate AI agent skills using structured benchmarks with YAML specs, fixture isolation, and pluggable validators. USE FOR: run waza, waza help, run eval, run benchmark, evaluate skill, test agent, generate eval suite, init eval, compare results, score agent, agent evaluation, skill testing, cross-model comparison. DO NOT USE FOR: improving skill frontmatter (use waza dev), c… - **Type:** Skill - **Install:** `agentstack add skill-microsoft-waza-waza` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [microsoft](https://agentstack.voostack.com/s/microsoft) - **Installs:** 0 - **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [microsoft](https://github.com/microsoft) - **Source:** https://github.com/microsoft/waza/tree/main/skills/waza - **Website:** https://microsoft.github.io/waza/ ## Install ```sh agentstack add skill-microsoft-waza-waza ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # Waza > "The way of technique — measure, refine, master." A Go CLI tool for evaluating AI agent skills through structured benchmarks. Define test cases in YAML, run them against agent engines, and validate results with pluggable scoring validators. ## Help When user says "waza help" or asks how to use waza: ``` ╔══════════════════════════════════════════════════════════════════╗ ║ WAZA - CLI Tool for Evaluating Agent Skills ║ ╠══════════════════════════════════════════════════════════════════╣ ║ ║ ║ COMMANDS: ║ ║ waza run # Run an evaluation benchmark ║ ║ waza init [directory] # Initialize a new eval suite ║ ║ waza generate # Generate eval from SKILL.md ║ ║ waza compare ... # Compare result files ║ ║ waza dev [skill-path] # Improve SKILL.md compliance ║ ║ ║ ║ RUN FLAGS: ║ ║ --context-dir, -c Fixtures directory (default: ./fixtures) ║ ║ --output, -o Save results JSON to file ║ ║ --verbose, -v Verbose output ║ ║ --task, -t Filter tasks by name (repeatable) ║ ║ --parallel, -p Run tasks in parallel ║ ║ --workers, -w Number of parallel workers ║ ║ --transcript-dir Save per-task transcripts ║ ║ ║ ║ COMPARE FLAGS: ║ ║ --format, -f Output format: table or json ║ ║ ║ ║ GENERATE FLAGS: ║ ║ --output-dir, -d Output directory for generated files ║ ║ ║ ║ DEV FLAGS: ║ ║ --target Adherence level: low|medium|high ║ ║ --max-iterations Max improvement iterations (default: 5) ║ ║ --auto Auto-apply without prompting ║ ║ ║ ║ WORKFLOW: ║ ║ 1. waza init my-eval # Scaffold eval suite ║ ║ 2. Edit eval.yaml + tasks # Define test cases ║ ║ 3. waza run eval.yaml -v # Execute benchmark ║ ║ 4. waza compare a.json b.json # Cross-model comparison ║ ║ ║ ║ FIXTURE ISOLATION: ║ ║ Each task gets a fresh temp workspace with fixtures copied ║ ║ in. Original fixtures are never modified. ║ ║ ║ ╚══════════════════════════════════════════════════════════════════╝ ``` ## Commands ### `waza run` Run an evaluation benchmark from a YAML spec file. ```bash # Run with default mock engine waza run path/to/eval.yaml --context-dir path/to/fixtures # Verbose output with results saved waza run eval.yaml -c ./fixtures -v -o results.json # Filter to specific tasks waza run eval.yaml -t "task-name-1" -t "task-name-2" # Parallel execution waza run eval.yaml --parallel --workers 4 # Save per-task transcripts waza run eval.yaml --transcript-dir ./transcripts ``` ### `waza init` Initialize a new evaluation suite with a compliant directory structure. ```bash # Initialize in current directory waza init # Initialize in a named directory waza init my-eval-suite ``` Creates: `eval.yaml`, `tasks/` with example task, `fixtures/` with example fixture. ### `waza generate` Generate an eval suite from an existing SKILL.md file. ```bash # Generate eval from SKILL.md waza generate path/to/SKILL.md # Specify output directory waza generate SKILL.md --output-dir ./my-eval ``` Parses YAML frontmatter (name, description) and creates eval.yaml, starter tasks, and fixtures. ### `waza compare` Compare results from multiple evaluation runs side by side. ```bash # Compare two result files waza compare run1.json run2.json # Compare three or more waza compare gpt4.json claude.json gemini.json # JSON output waza compare run1.json run2.json --format json ``` Shows per-task score deltas, pass rate differences, and aggregate statistics. ### `waza dev` Iteratively improve SKILL.md frontmatter compliance with automated scoring. ```bash # Score current skill and suggest improvements waza dev skills/my-skill # Target high compliance level waza dev skills/my-skill --target high # Auto-apply improvements without prompts waza dev skills/my-skill --target medium --auto --max-iterations 3 ``` **Compliance Levels:** - **Low** (< 150 chars or no triggers) — Minimal description - **Medium** (150+ chars, has triggers) — Basic trigger coverage - **Medium-High** (+ anti-triggers) — Routing clarity improved - **High** (+ routing markers like INVOKES/FOR SINGLE OPERATIONS) — Full compliance **Scoring Checks:** - Description length (150+ chars required, 1024 max) - Trigger phrases (USE FOR: patterns) - Anti-trigger phrases (DO NOT USE FOR: patterns) - Routing clarity markers (**WORKFLOW SKILL**, INVOKES:, etc.) - Token budget (500 soft limit, 5000 hard limit) **Coming Soon:** Trigger accuracy tests (#36), `--skip-integration` (#37), `--fast` (#38), improvement suggestions engine (#34). ## Evaluation Spec Format ```yaml name: my-eval skill: my-skill version: "1.0" executor: mock # or copilot-sdk tasks: - id: task-1 name: "Describe the task" prompt: "Your prompt to the agent" expected: "Expected behavior" validators: - type: code config: language: go - type: text config: pattern: "expected pattern" ``` ## Engines | Engine | Use | Description | |--------|-----|-------------| | `mock` | Testing | Returns canned responses for validator development | | `copilot-sdk` | Production | Executes via Copilot CLI SDK | ## Validators | Validator | What it checks | |-----------|---------------| | `code` | Code compiles / passes syntax check | | `regex` | Output matches regex pattern | ## Configuration | Setting | Flag | Default | |---------|------|---------| | Fixtures dir | `--context-dir` | `./fixtures` | | Output file | `--output` | (none) | | Verbose | `--verbose` | `false` | | Parallel | `--parallel` | `false` | | Workers | `--workers` | CPU count | | Transcript dir | `--transcript-dir` | (none) | ## Scoring Quick Reference Each task produces an `EvaluationOutcome` with: | Field | Description | |-------|-------------| | `score` | 0.0–1.0 normalized score | | `pass` | Boolean pass/fail | | `validator_results` | Per-validator details | | `duration` | Execution time | ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [microsoft](https://github.com/microsoft) - **Source:** [microsoft/waza](https://github.com/microsoft/waza) - **License:** MIT - **Homepage:** https://microsoft.github.io/waza/ Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-microsoft-waza-waza - Seller: https://agentstack.voostack.com/s/microsoft - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.