# Evaluating Skills With Models

> Evaluate skills by executing them across sonnet, opus, and haiku models using sub-agents. Use when testing if a skill works correctly, comparing model performance, or finding the cheapest compatible model. Returns numeric scores (0-100) to differentiate model capabilities.

- **Type:** Skill
- **Install:** `agentstack add skill-taisukeoe-agentic-ai-skills-creator-evaluating-skills-with-models`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [taisukeoe](https://agentstack.voostack.com/s/taisukeoe)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [taisukeoe](https://github.com/taisukeoe)
- **Source:** https://github.com/taisukeoe/agentic-ai-skills-creator/tree/main/plugins/skills-helper/skills/evaluating-skills-with-models

## Install

```sh
agentstack add skill-taisukeoe-agentic-ai-skills-creator-evaluating-skills-with-models
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Evaluating Skills with Models

Evaluate skills across multiple Claude models using sub-agents with quality-based scoring.

> **Requirement:** Claude Code CLI only. Not available in Claude.ai.

## Why Quality-Based Scoring

Binary pass/fail ("did it do X?") fails to differentiate models - all models can "do the steps." The difference is **how well** they do them. This skill uses weighted scoring to reveal capability differences.

## Workflow

### Step 1: Load Test Scenarios

Check for `tests/scenarios.md` in the target skill directory.

**Default to difficult scenarios:** When multiple scenarios exist, prioritize **Hard** or **Medium** difficulty scenarios for evaluation. Easy scenarios often don't show meaningful differences between models and aren't realistic for production use.

**Required scenario format:**

```markdown
## Scenario: [Name]

**Difficulty:** Easy | Medium | Hard | Edge-case

**Query:** User request that triggers this skill

**Expected behaviors:**

1. [Action description]
   - **Minimum:** What counts as "did it"
   - **Quality criteria:** What "did it well" looks like
   - **Haiku pitfall:** Common failure mode
   - **Weight:** 1-5

**Output validation:** (optional)
- Pattern: `regex`
- Line count: ` **Note:** Token usage includes both skill execution AND reporting overhead. Sonnet tends to produce detailed reports, which inflates token count. Compare tool uses for execution efficiency.

## Quick Reference

```
Load scenarios (prioritize Hard) → Execute (parallel) → Score behaviors → Calculate totals → Add to README → Output summary
```

For detailed scoring guidelines, see [references/evaluation-structure.md](references/evaluation-structure.md).

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [taisukeoe](https://github.com/taisukeoe)
- **Source:** [taisukeoe/agentic-ai-skills-creator](https://github.com/taisukeoe/agentic-ai-skills-creator)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-taisukeoe-agentic-ai-skills-creator-evaluating-skills-with-models
- Seller: https://agentstack.voostack.com/s/taisukeoe
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.