Evaluating Skills With Models
Evaluate skills by executing them across sonnet, opus, and haiku models using sub-agents. Use when testing if a skill works correctly, comparing model performance, or finding the cheapest compatible model. Returns numeric scores (0-100) to differentiate model capabilities.
Install
$ agentstack add skill-taisukeoe-agentic-ai-skills-creator-evaluating-skills-with-models ✓ scanned · ✓ verified, works with Claude Code, Cursor, and more.
Security review
✓ PassedNo issues found. Passed automated security review. · v0.1.0 How review works →
- ✓ Prompt-injection patterns
- ✓ Secret / credential exfiltration
- ✓ Dangerous shell & filesystem operations
- ✓ Untrusted network calls
- ✓ Known-malicious package signatures
What it can access
- ✓ Network access No
- ✓ Filesystem access No
- ✓ Shell / process execution No
- ✓ Environment & secrets No
- ✓ Dynamic code execution No
From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.
Verified badge
Passed review? Show it. Paste this badge into your README, it links to the public security report.
Links to your public security report.[](https://agentstack.voostack.com/security/report/skill-taisukeoe-agentic-ai-skills-creator-evaluating-skills-with-models) Reliability & compatibility
Declared compatibility
Compatibility is declared by the source manifest. End-to-end runtime verification is coming, see below.
We're building live execution health for every listing: tool-call success rate, median latency, uptime, and last-checked timestamps, measured, not self-reported. It isn't live yet, so we don't show numbers we can't stand behind.
How agent discovery & health will work →About
Evaluating Skills with Models
Evaluate skills across multiple Claude models using sub-agents with quality-based scoring.
> Requirement: Claude Code CLI only. Not available in Claude.ai.
Why Quality-Based Scoring
Binary pass/fail ("did it do X?") fails to differentiate models - all models can "do the steps." The difference is how well they do them. This skill uses weighted scoring to reveal capability differences.
Workflow
Step 1: Load Test Scenarios
Check for tests/scenarios.md in the target skill directory.
Default to difficult scenarios: When multiple scenarios exist, prioritize Hard or Medium difficulty scenarios for evaluation. Easy scenarios often don't show meaningful differences between models and aren't realistic for production use.
Required scenario format:
## Scenario: [Name]
**Difficulty:** Easy | Medium | Hard | Edge-case
**Query:** User request that triggers this skill
**Expected behaviors:**
1. [Action description]
- **Minimum:** What counts as "did it"
- **Quality criteria:** What "did it well" looks like
- **Haiku pitfall:** Common failure mode
- **Weight:** 1-5
**Output validation:** (optional)
- Pattern: `regex`
- Line count: ` **Note:** Token usage includes both skill execution AND reporting overhead. Sonnet tends to produce detailed reports, which inflates token count. Compare tool uses for execution efficiency.
## Quick Reference
Load scenarios (prioritize Hard) → Execute (parallel) → Score behaviors → Calculate totals → Add to README → Output summary
For detailed scoring guidelines, see [references/evaluation-structure.md](references/evaluation-structure.md).
## Source & license
This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.
- **Author:** [taisukeoe](https://github.com/taisukeoe)
- **Source:** [taisukeoe/agentic-ai-skills-creator](https://github.com/taisukeoe/agentic-ai-skills-creator)
- **License:** Apache-2.0
Install and usage instructions live in the source repository linked above.
Reviews
No reviews yet, be the first.
Write a review
Versions
- v0.1.0 Imported from the upstream source.