Ai Launch Check

AI Feature Launch Check

> Iron Law: Do not output GO if any checklist item is ❌. A written exception in the Decision Record with an owner and deadline is required for every ⚠️ item. Verbal assertions do not count as evidence for any item.

Run the unified pre-production launch checklist from the AI Playbook (Layer 4 §4.1). This consolidates the pre-flight questions, pre-production gate, kill-switch tests, and adversarial test catalog into one launch review.

Procedure

1. Ask the user for the feature slug. Look for the Decision Record at docs/ai-decisions/.md.

• If it does not exist: stop. Tell the user to run /ai-decision-record first. Do not proceed without one.

1. Read the Decision Record. Cross-check every item below against what the record says.
2. For each checklist item below, ask the user to show evidence, not assert compliance. Examples of evidence:

• "Output Contract implemented" → point to the validation code; read it.
• "Eval set" → point to the file; show the size and a sample.
• "Kill switch tested" → show the test run log.

1. Mark each item: ✅ verified / ⚠️ partial / ❌ missing.
2. Do not pass the launch check if any item is ❌. Partial is allowed only when the user provides a written exception in the Decision Record with an owner and a deadline.
3. Produce a launch decision: GO / NO-GO / GO-WITH-EXCEPTIONS with a written rationale.

The pre-flight (10 questions)

Walk through these first. Any "no" or "I don't know" is a NO-GO.

• [ ] 1. Can deterministic code solve this reliably? (If yes → why are we shipping AI?)
• [ ] 2. Is the input genuinely unstructured or ambiguous?
• [ ] 3. Does AI measurably beat code on accuracy, speed, cost, or UX?
• [ ] 4. Is the decision reversible, or is there a downstream verifier?
• [ ] 5. Off the critical latency path, or fast fallback exists?
• [ ] 6. What's the cost when AI is wrong, and who absorbs it?
• [ ] 7. Is human approval required for high-impact outcomes?
• [ ] 8. Can every AI output be audited and explained later?
• [ ] 9. Enough labeled examples for risk-tiered evaluation?
• [ ] 10. Deterministic fallback when AI fails or times out?

The artifacts checklist

Each item must point to a file, a log, or a running piece of code. Verbal assertions do not count.

• [ ] Decision Record signed off (docs/ai-decisions/.md)
• [ ] Risk-tiered eval set with target metrics (size matches Layer 2 §2.14 for the declared risk tier)
• [ ] Output Contract implemented: schema → types → business rules → permissions → confidence → audit → route (Layer 2 §2.4)
• [ ] Adversarial tests in CI (Layer 3 §3.2): prompt injection, tool misuse, retrieval poisoning, bad schema outputs
• [ ] RAG requirements met if applicable (Layer 2 §2.7): hybrid search, metadata filtering, ACLs at query time, citations, no-answer test, separate retrieval/answer grading, chunk provenance
• [ ] Multi-modal handling reviewed if applicable (Layer 2 §2.8): each modality validated, cost recalculated, sandboxes for code execution
• [ ] Model and routing strategy decided (Layer 2 §2.9)
• [ ] Cost & latency budget filled (Layer 3 §3.4): max calls, max tokens, p50/p95/p99, cost ceiling, fallback after timeout, cache strategy, rate limit
• [ ] 12-month AI tax estimated and approved (Layer 2 §2.11)
• [ ] Vendor abstraction in place; model and prompt versions pinned (Layer 2 §2.12)
• [ ] Observability schema implemented (Layer 3 §3.5): modelversion, promptversion, inputhash, validation result, confidence, routedto, downstream outcome
• [ ] Human review queue defined: what goes, who reviews, SLA per Layer 2 §2.1
• [ ] Data governance review completed (Layer 2 §2.13): context contents, logging, retention, training-use, tenant isolation, deletion propagation, redaction
• [ ] Security review completed (Layer 2 §2.5): the two questions answered with code, not prompts
• [ ] Rollout plan with promotion criteria (Layer 4 §4.2): offline → dogfood → shadow → beta → percentage → full
• [ ] Incident response plan (Layer 4 §4.3): named on-call, rollback triggers with thresholds, affected-user identification, quarantine, reversal, post-mortem template, leading indicators
• [ ] Kill-switch tests passing (Layer 4 §4.4): all eight tests in staging or shadow, with logs

Kill-switch tests — must all pass

Demand the test logs. If they don't exist, fail the launch check.

• [ ] Disable feature via flag → graceful degradation
• [ ] Force model downgrade → output contract still passes or routes to review
• [ ] Simulate vendor outage → timeout fires, fallback serves, no infinite retry
• [ ] Simulate retrieval poisoning → contract/security rejects; provenance logs source
• [ ] Simulate cost spike (10× tokens) → rate limit + cost ceiling fire; alerts route correctly
• [ ] Simulate prompt injection from user input AND from a retrieved doc → least-privilege scope prevents harm
• [ ] Roll back one prompt or model version → traffic shifts cleanly; metrics return to baseline
• [ ] Verify deletion propagation → user-deleted data gone from embeddings, caches, logs, evals within SLA

Decision output

After running through every item, output:

# Launch Check: 

**Decision**: GO | NO-GO | GO-WITH-EXCEPTIONS
**Reviewer**:  — 
**Decision Record**: docs/ai-decisions/.md

## Verified (✅)
- ...

## Partial — exceptions filed (⚠️)
- : exception in Decision Record, owner , deadline 

## Missing — blockers (❌)
- ...

## Recommendation

## If GO-WITH-EXCEPTIONS: follow-up SLAs
- ...

Default stance

• Bias toward NO-GO. A launch check is a forcing function; if the team passes too easily, the check isn't doing its job.
• Do not let "we'll fix it after launch" pass for kill-switch tests, output contract, or security review. Those are pre-launch by definition.
• Tier-3+ features with no human approval gate are an automatic NO-GO unless the Decision Record has a written domain-risk exception.

If this skill cannot proceed

Abort condition: Decision Record missing
→ Hard stop. Output: "Run /ai-decision-record first. The launch check
  cross-references the Decision Record for every checklist item. Cannot proceed without it."

Abort condition: user provides verbal assertions instead of file:line evidence
→ Mark item ⚠️ at best, ❌ if critical (kill switch, output contract, security review).
→ Output: "Verbal assertions are not evidence. Point to the file, log, or test output."

Abort condition: all 8 kill-switch tests have no logs
→ Automatic NO-GO regardless of other items.
→ Output: "Kill-switch tests must be run in staging or shadow with logs. 'We tested it
  manually once' does not satisfy this requirement."

Verify (after producing the launch decision)

✅ Every checklist item has a status: ✅ / ⚠️ / ❌ — no blanks
✅ Every ⚠️ item has a named owner and a written deadline in the Decision Record
✅ Decision (GO / NO-GO / GO-WITH-EXCEPTIONS) matches the item statuses — no ❌ items in a GO
✅ Launch check report written to docs/ai-audits//04-launch-check.md

References

• AI Playbook Layer 4 §4.1 Unified launch checklist: .claude/skills/ai-playbook/playbook.md
• AI Playbook Layer 4 §4.2 Rollout strategy: .claude/skills/ai-playbook/playbook.md
• AI Playbook Layer 4 §4.4 Kill-switch tests: .claude/skills/ai-playbook/playbook.md
• AI Playbook Layer 2 §2.14 Eval size floors: .claude/skills/ai-playbook/playbook.md
• Prerequisite: /ai-decision-record (must exist before launch check runs)

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: kumaran-is
• Source: kumaran-is/claude-code-onboarding
• License: MIT

Install and usage instructions live in the source repository linked above.