Eval Driven Development
Building rigorous LLM evaluation pipelines. Use when developing AI features to ensure quality and prevent regressions across different model versions.
Install
$ agentstack add skill-ishandutta2007-awesome-agent-skills-eval-driven-development ✓ scanned · ✓ verified, works with Claude Code, Cursor, and more.
Security review
✓ PassedNo issues found. Passed automated security review. · v0.1.0 How review works →
- ✓ Prompt-injection patterns
- ✓ Secret / credential exfiltration
- ✓ Dangerous shell & filesystem operations
- ✓ Untrusted network calls
- ✓ Known-malicious package signatures
What it can access
- ✓ Network access No
- ✓ Filesystem access No
- ✓ Shell / process execution No
- ✓ Environment & secrets No
- ✓ Dynamic code execution No
From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.
Verified badge
Passed review? Show it. Paste this badge into your README, it links to the public security report.
Links to your public security report.[](https://agentstack.voostack.com/security/report/skill-ishandutta2007-awesome-agent-skills-eval-driven-development) Reliability & compatibility
Declared compatibility
Compatibility is declared by the source manifest. End-to-end runtime verification is coming, see below.
We're building live execution health for every listing: tool-call success rate, median latency, uptime, and last-checked timestamps, measured, not self-reported. It isn't live yet, so we don't show numbers we can't stand behind.
How agent discovery & health will work →About
Eval-Driven Development
Overview
Eval-Driven Development ensures that AI features behave deterministically and predictably by testing them against a golden dataset using automated evaluators.
When to Use
- Building an AI-powered feature
- Tuning prompts or changing underlying models
- Implementing RAG pipelines
Process
- Curate Golden Dataset: Create diverse test cases including edge cases.
- Define Metrics: Choose appropriate evaluators (e.g., exact match, semantic similarity, LLM-as-a-judge).
- Run Pipeline: Execute the AI feature over the dataset and collect results.
- Analyze Failures: Inspect low-scoring examples and update prompts or logic.
- Establish Baseline: Set a minimum threshold for CI/CD checks.
Common Rationalizations
| Rationalization | Why It Is Wrong | |---|---| | "Manual spot checks are enough." | Spot checks miss regressions across prompts, model versions, and edge cases. | | "We can add evals after launch." | Without a baseline, you cannot tell whether a later prompt or model change improved behavior. | | "The judge model says it is good." | LLM judges need criteria, calibration examples, and failure review before they are trustworthy. |
Red Flags
- No golden dataset exists
- Metrics are vague or not tied to user-visible quality
- Low-scoring examples are ignored instead of inspected
- The baseline threshold is chosen after seeing the desired result
Verification
Before finishing, confirm:
- The golden dataset includes normal, edge, and known-failure cases
- Metrics and evaluator prompts are committed or otherwise reproducible
- The current model/prompt has a recorded baseline
- Failure examples have been reviewed and categorized
Source & license
This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.
- Author: ishandutta2007
- Source: ishandutta2007/Awesome-Agent-Skills
- License: MIT
Install and usage instructions live in the source repository linked above.
Reviews
No reviews yet, be the first.
Write a review
Versions
- v0.1.0 Imported from the upstream source.