# Auto Skill Build Agent Reliability Loop

> >

- **Type:** Skill
- **Install:** `agentstack add skill-arnie016-codex-prompt-templates-auto-skill-build-agent-reliability-loop`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [Arnie016](https://agentstack.voostack.com/s/arnie016)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [Arnie016](https://github.com/Arnie016)
- **Source:** https://github.com/Arnie016/codex-prompt-templates/tree/main/skills/auto-skill-build-agent-reliability-loop
- **Website:** https://github.com/Arnie016/codex-prompt-templates

## Install

```sh
agentstack add skill-arnie016-codex-prompt-templates-auto-skill-build-agent-reliability-loop
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Agent Reliability Loop

Generated by: Codex Supercharge maintenance automation.

Goal: turn an agent workflow into a closed loop where production traces become
eval cases, eval failures become guardrails or fixes, and gateway policy keeps
cost, routing, and tool risk bounded.

## Skip When

- The user only wants local Codex run/cost tracking.
- The work is a single benchmark or experiment with no production surface.
- The task is only MCP conformance; use `$auto-skill-build-mcp-conformance-harness`.

## Workflow

1. **Map the surface**: list user journeys, models, tools, data classes,
   side effects, risk levels, and quality/cost/latency targets.
2. **Instrument first**: require request IDs, trace/span IDs, session IDs,
   model/provider, token/cost, cache, fallback, guardrail, and tool metadata.
3. **Build eval bundles**: create golden cases for task success, prompt
   conformance, tool correctness, unsafe requests, PII/secrets, latency, and
   cost. Keep judge prompts and heuristic checks versioned.
4. **Set gateway policy**: define provider mapping, model fallbacks, cache
   rules, virtual keys, budgets, rate limits, privacy redaction, and audit logs.
5. **Stage guardrails**: run new pre/post checks in log or monitor mode first,
   then enforce only after false positives and fail-open/fail-closed behavior
   are explicit.
6. **Shadow safely**: mirror sampled traffic to candidate models or prompts
   only when shadow calls cannot trigger external side effects.
7. **Close the loop**: cluster failed traces, match nearest successful traces,
   add representative failures to evals, patch prompts/tools/policies, then
   rerun the bundle before release.

## Commands

```bash
rg -n "trace|span|cost|tokens|guardrail|fallback|budget|rate limit|request_id" .
rg -n "eval|rubric|judge|golden|dataset|experiment|shadow|canary" .
rg -n "tool|mcp|side effect|webhook|shell|filesystem|credential" .
```

## Output

```md
# Agent Reliability Plan
## Surface
## Instrumentation
## Eval Bundle
## Gateway Policy
## Guardrail Rollout
## Shadow Or Canary Plan
## Trace-To-Regression Loop
## Risks And Trust Notes
## Validation
```

## Validation

- Every production route has at least one trace and one regression case.
- Every blocking guardrail has a false-positive review path.
- Every budget/rate limit has an owner and an alert threshold.
- Shadow/canary traffic cannot write to tools, accounts, payments, or user data.

Read `references/future-agi-agent-reliability-loop.md` for the source-backed
pattern and risk notes.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [Arnie016](https://github.com/Arnie016)
- **Source:** [Arnie016/codex-prompt-templates](https://github.com/Arnie016/codex-prompt-templates)
- **License:** MIT
- **Homepage:** https://github.com/Arnie016/codex-prompt-templates

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-arnie016-codex-prompt-templates-auto-skill-build-agent-reliability-loop
- Seller: https://agentstack.voostack.com/s/arnie016
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.