# Prompt Eval Harness

> Build a constrained prompt evaluation harness with rubric, test cases, adversarial cases, prompt variants, scoring, failure analysis, evidence confidence, and final recommendation. Adapted from TerminalSkills prompt-tester; never replaces prompts or governance automatically.

- **Type:** Skill
- **Install:** `agentstack add skill-zaixincheng174-ai-codex-agent-governance-skills-prompt-eval-harness`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [zaixincheng174-ai](https://agentstack.voostack.com/s/zaixincheng174-ai)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [zaixincheng174-ai](https://github.com/zaixincheng174-ai)
- **Source:** https://github.com/zaixincheng174-ai/codex-agent-governance-skills/tree/main/core/skills/prompt-eval-harness
- **Website:** https://github.com/zaixincheng174-ai/codex-agent-governance-skills/blob/main/docs/demo-false-pass.md

## Install

```sh
agentstack add skill-zaixincheng174-ai-codex-agent-governance-skills-prompt-eval-harness
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# prompt-eval-harness

Source repo: https://github.com/TerminalSkills/skills
Source path: skills/prompt-tester/SKILL.md
Source commit: b12ada7710427e475de9ab5d864562f1ecfb6f62
Local adaptation note: Adapted from a general prompt testing workflow into a governance-safe evaluation harness. It supports structured scoring and recommendations, but does not automatically replace prompts, AGENTS instructions, or production behavior.

## Purpose

Evaluate prompt or instruction variants with explicit evidence rather than preference.

Use for AI feature prompts, agent instructions, review prompts, classification prompts, extraction prompts, or policy wording where output quality must be compared across cases.

## Activation

Use when the user asks to test, compare, score, harden, or evaluate prompts, system prompts, rubrics, agent instructions, or prompt variants.

Do not activate when the user only asks for a simple rewrite and does not need evaluation.

## Required Harness

Create or report:

```text
Evaluation goal:
Rubric:
Test cases:
Adversarial cases:
Prompt variants:
Scoring method:
Results:
Failure analysis:
Final recommendation:
Evidence confidence:
```

Rubric requirements:
- Criteria with weights or priority order.
- Expected output format when relevant.
- Explicit failure conditions.

Test case requirements:
- Normal cases.
- Edge cases.
- Adversarial cases.
- At least one case that tests instruction conflict or ambiguity when relevant.

Scoring requirements:
- Score each variant against the same rubric.
- Track false positives, false negatives, format failures, hallucinations, missing evidence, and brittle behavior when relevant.
- State evidence confidence as High, Medium, or Low with a reason.

## Workflow

1. Define what good output means before comparing variants.
2. Build test cases and adversarial cases.
3. Compare prompt variants against the same cases.
4. Record failures and likely causes.
5. Recommend a winner, a merge, or no-change.
6. Preserve the distinction between evaluation result and implementation approval.

## Forbidden actions

- Do not automatically replace prompts, system prompts, AGENTS instructions, project instructions, or production configs.
- Do not edit AGENTS, update CONTEXT.md, create ADR, publish, deploy, git push, or run setup.
- Do not treat a small eval as proof of production readiness.
- Do not use post-hoc rubric changes to make a preferred prompt win.
- Do not hide failed cases from the final recommendation.

## Stop conditions

Stop or downgrade confidence when:
- The prompt goal is undefined.
- No expected outputs or scoring rubric can be stated.
- The available cases are too few or too homogeneous for a defensible recommendation.
- Running the evaluation would require unavailable models, credentials, private data, or external systems.
- The user asks for prompt replacement but has not explicitly approved implementation.

## Output Pattern

```text
Prompt Eval Harness
Goal:
Rubric:
Cases:
Variants:
Scores:
Failure analysis:
Recommendation:
Implementation status: recommendation only; no prompt replacement performed
Evidence confidence:
```

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [zaixincheng174-ai](https://github.com/zaixincheng174-ai)
- **Source:** [zaixincheng174-ai/codex-agent-governance-skills](https://github.com/zaixincheng174-ai/codex-agent-governance-skills)
- **License:** MIT
- **Homepage:** https://github.com/zaixincheng174-ai/codex-agent-governance-skills/blob/main/docs/demo-false-pass.md

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-zaixincheng174-ai-codex-agent-governance-skills-prompt-eval-harness
- Seller: https://agentstack.voostack.com/s/zaixincheng174-ai
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.