# Honest Verdicts

> Anti-vacuous-green playbook for test/CI evidence — named-suite + executed-count verdicts, bundle-membership checks, fresh-environment runs, platform-gating, Release builds, self-passing gates. Use when claiming tests pass, authoring CI gates, writing acceptance predicates, or diagnosing "flaky" CI.

- **Type:** Skill
- **Install:** `agentstack add skill-dndungu-agent-skills-honest-verdicts`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [dndungu](https://agentstack.voostack.com/s/dndungu)
- **Installs:** 0
- **Category:** [Developer Tools](https://agentstack.voostack.com/c/developer-tools)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [dndungu](https://github.com/dndungu)
- **Source:** https://github.com/dndungu/agent-skills/tree/main/honest-verdicts

## Install

```sh
agentstack add skill-dndungu-agent-skills-honest-verdicts
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Honest verdicts

A green checkmark is not evidence. Every one of these failure modes has
produced a passing build that verified nothing. The bar: prove the RIGHT
tests RAN and PASSED, in an environment that resembles what ships.

## 1. Named suite + executed count, never exit codes

- A test claim must cite the NAMED suite that passed AND an executed count
  >= 1 (e.g. `Executed [1-9]` in the log). Exit code 0 alone is worthless:
  filters that match nothing, skipped suites, and skip-without-env guards
  all exit 0 having run zero tests.
- Skip-guards make checks vacuous silently: a suite that skips when an env
  var (e.g. TEST_DATABASE_URL) is absent "passes" without running. When you
  rely on such a suite, verify the env was present and the count nonzero.

## 2. The test must be IN the built bundle

- A test file on disk that isn't a member of the built test bundle yields
  TEST SUCCEEDED with the class never loaded. Verify bundle membership
  (target membership / build manifest), or better, verify by executed
  count per suite name.

## 3. Fresh environment, or the run lies

- Cached app installs mask failures: a stale build on a warm simulator can
  pass UI/accessibility queries the current code would fail. UI test
  verdicts and screenshots come from a FRESH simulator / clean environment.
- One observed instance: merged UI tests were silently failing on fresh
  sims because accessibility identifiers on bare containers leak to
  descendants — only a clean run exposed it.

## 4. Build what CI builds, and what ships

- Platform-gate platform-only APIs (`#if os(iOS)`) when CI also builds
  another slice (e.g. a macOS SwiftPM job). An availability check with a
  trailing `*` can resolve TRUE on the other platform — name every
  platform explicitly.
- Build Release (`-c release`) before claiming done: DEBUG-only symbols
  (fixtures, previews outside `#if DEBUG`) compile fine in Debug and break
  the archive. Fixture/preview code lives behind `#if DEBUG`, verified by
  a Release build, and ideally a static CI gate.

## 5. Text-matching gates catch text, not code

- A conformance gate that greps source TEXT also matches comments and doc
  strings (a forbidden type name in a doc comment fails CI). Know this
  when authoring the gate AND when a red looks absurd — the fix may be a
  comment edit, and that is working as designed for a cheap gate.
- Every NEW gate lands GREEN against the code it polices (self-passing
  principle): run it on the current tree, fix or explicitly baseline all
  existing violations in the same PR. A gate that arrives red teaches the
  team to ignore it.

## 6. "Flaky" is a diagnosis of last resort

- After a base-branch refresh, REBASE stale PR branches before diagnosing
  failures: a stale head mixes new base tests with old code and fails for
  reasons that are neither flake nor bug. Observed: a "flaky" webhook test
  was a superseded pre-rebase head; 250+ clean race-detector runs proved
  the test sound.
- To call something flaky, show N clean runs of the same commit under load
  (race detector, CPU saturation) — otherwise keep digging.

## 7. Predicates and acceptance criteria

- When authoring machine-checkable predicates (example-app or CI), apply all of
  the above: no vacuous/always-true predicates, spot-check at least one
  red-to-green transition, and prefer executed-count assertions over
  exit-code assertions.
- Visual claims need visual evidence — greps and unit tests can pass while
  the UI was never really built. Screenshot from a live run.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [dndungu](https://github.com/dndungu)
- **Source:** [dndungu/agent-skills](https://github.com/dndungu/agent-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-dndungu-agent-skills-honest-verdicts
- Seller: https://agentstack.voostack.com/s/dndungu
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.