# Ag Benchmark Qualidade

> QAT-Benchmark PDCA Orchestrator — executa benchmark de qualidade AI com dual-run (app vs baseline), triple-scorer, 8 dimensoes, Parity Index e ciclo PDCA de melhoria continua.

- **Type:** Skill
- **Install:** `agentstack add skill-andregusman-raiz-a-gusman-claude-ag-benchmark-qualidade`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [andregusman-raiz](https://agentstack.voostack.com/s/andregusman-raiz)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [andregusman-raiz](https://github.com/andregusman-raiz)
- **Source:** https://github.com/andregusman-raiz/a-gusman-claude/tree/main/skills/ag-benchmark-qualidade

## Install

```sh
agentstack add skill-andregusman-raiz-a-gusman-claude-ag-benchmark-qualidade
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# ag-benchmark-qualidade — QAT-Benchmark (Qualidade Comparativa)

## Papel

O Benchmark Orchestrator: executa ciclo completo Plan-Do-Check-Act de QAT-Benchmark. Compara qualidade dos outputs da app contra baseline de mercado (Claude API direto), usando dual-run, triple-scorer com Judge Jury, e 8 dimensoes.

Diferenca de ag-testar-qualidade-qat: ag-testar-qualidade-qat avalia qualidade ABSOLUTA. ag-benchmark-qualidade avalia qualidade RELATIVA (parity).
Diferenca de ag-testar-ux-qualidade: ag-testar-ux-qualidade avalia qualidade VISUAL/UI. ag-benchmark-qualidade avalia conteudo AI comparativamente.
Diferenca de ag-criar-cenario-benchmark: ag-criar-cenario-benchmark CRIA cenarios. ag-benchmark-qualidade EXECUTA e orquestra PDCA.

## Invocacao

```
/ag-benchmark-qualidade https://app.vercel.app                    # Standard mode, todos cenarios
/ag-benchmark-qualidade https://app.vercel.app BM-01,BM-03        # Cenarios especificos
/ag-benchmark-qualidade https://app.vercel.app all full            # Full jury (3 judges x 2 pos)
/ag-benchmark-qualidade https://app.vercel.app all smoke           # 5 cenarios core, rapido
```

## Pre-requisitos

1. Estrutura `tests/qat-benchmark/` no projeto (copiar de `~/.claude/shared/templates/qat-benchmark/`)
2. `QAT_BENCHMARK_BASE_URL` ou URL no argumento
3. `QAT_BENCHMARK_ANTHROPIC_KEY` configurado (baseline + judge)
4. `playwright-cli` disponivel (app adapter)
5. URL da app acessivel

## Ciclo PDCA

```
PLAN: Preflight + carregar KB (baselines, failure-patterns, learnings)
      Selecionar cenarios com anti-contaminacao (30% fixed + 70% rotatable)
DO:   Dual-run (app via Playwright + baseline via Claude API)
      Triple-score: L1-L2 rule-based → L3 Judge Jury → L4 functional
      Short-circuit: se L1 falha, skip jury (~30% economia)
CHECK: Classificar falhas (INFRA/FEATURE/QUALITY/BUSINESS/RUBRIC/FLAKY/BASELINE)
       Calcular Parity Index overall e por dimensao
       Comparar com baselines historicos, detectar regressoes
ACT:   Atualizar baselines, registrar failure patterns, adicionar learnings
       Gerar report PDCA com parity e classificacao
```

## 8 Dimensoes

| ID | Dimensao | Peso |
|----|----------|------|
| D1 | Content Accuracy | 15% |
| D2 | Teaching Quality | 15% |
| D3 | Agentic Capability | 15% |
| D4 | Calibration | 10% |
| D5 | Safety | 10% |
| D6 | Efficiency | 10% |
| D7 | Robustness | 10% |
| D8 | Response UX | 15% |

## Modos de Execucao

| Modo | Cenarios | Jury | Custo/run |
|------|----------|------|-----------|
| full | 40 | 3x2 | ~$6-12 |
| standard | 40 | 1x2 | ~$2-4 |
| rapid | 12 (fixed) | 1x2 | ~$0.60-1.20 |
| smoke | 5 | 1x1 | ~$0.12-0.25 |

## Output

- `tests/qat-benchmark/results/YYYY-MM-DD-HHmmss/` com subdiretorios por cenario
- Cada cenario: `output-app.json`, `output-baseline.json`, `scores.json`
- Sumario: `summary.json` + `report.md` + `parity-report.json`

## Interacao com outros agentes

- ag-testar-qualidade-qat: Complementar (QAT qualidade absoluta, QAT-B qualidade relativa)
- ag-testar-ux-qualidade: Complementar (UX-QAT visual, QAT-B conteudo AI)
- ag-criar-cenario-benchmark: Complementar (ag-criar-cenario-benchmark cria cenarios, ag-benchmark-qualidade executa PDCA)
- ag-pipeline-deploy: Pos-deploy (QAT-B apos deploy para validar competitividade)
- ag-smoke-vercel: Sequencial (smoke primeiro, QAT-B depois se smoke passa)

## Referencia

- Agent completo: `~/.claude/agents/ag-benchmark-qualidade.md`
- Patterns: `~/.claude/shared/patterns/qat-benchmark.md`, `qat-benchmark-scoring.md`, `qat-benchmark-parity.md`
- Templates: `~/.claude/shared/templates/qat-benchmark/`

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [andregusman-raiz](https://github.com/andregusman-raiz)
- **Source:** [andregusman-raiz/a-gusman-claude](https://github.com/andregusman-raiz/a-gusman-claude)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-andregusman-raiz-a-gusman-claude-ag-benchmark-qualidade
- Seller: https://agentstack.voostack.com/s/andregusman-raiz
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.