# Check Tests

> Run test suite, track coverage trends, and detect regressions. Detect phase skill — uses config.test_command to execute tests and records results.

- **Type:** Skill
- **Install:** `agentstack add skill-mataeil-ooda-loop-check-tests`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [mataeil](https://agentstack.voostack.com/s/mataeil)
- **Installs:** 0
- **Category:** [Cloud & Infrastructure](https://agentstack.voostack.com/c/cloud-infrastructure)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [mataeil](https://github.com/mataeil)
- **Source:** https://github.com/mataeil/OODA-loop/tree/main/skills/check-tests

## Install

```sh
agentstack add skill-mataeil-ooda-loop-check-tests
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# check-tests: Test Suite Runner & Coverage Tracker

Runs the configured test command, tracks pass/fail counts and coverage percentage
over time, and alerts when regressions appear or coverage drops. READ-ONLY in
terms of PRs — writes only to `agent/state/test_coverage.json`.

---

## Safety Rules

1. **HALT File** — Check `config.safety.halt_file` first. If it exists, print reason and stop.
2. **Read-only** — Writes only to `agent/state/test_coverage.json`. Never touches test files or source.
3. **No test_command** — If `config.test_command` is empty or unset, skip gracefully.

---

## Step 0: Safety

**0-A: HALT Check**
```
if file exists at config.safety.halt_file:
  Print "[HALT] check-tests stopped. Reason: {file_content}"
  EXIT immediately.
```

**0-B: Config Validation**
```
if config.test_command is missing or empty:
  Print "No test command configured. Skipping check-tests."
  Print "Set config.test_command (e.g. \"npm test\", \"pytest\", \"go test ./...\") to enable."
  EXIT cleanly (not an error).
```

---

## Step 1: Load Previous State

Read `agent/state/test_coverage.json`. If missing, initialize with:
`{ "schema_version": "1.0.0", "last_run": null, "run_count": 0, "status": "unknown",`
`  "results": { "total": 0, "passed": 0, "failed": 0, "skipped": 0, "coverage_pct": null },`
`  "previous_results": null, "alerts": [], "history": [] }`

Note previous `passed`, `failed`, and `coverage_pct` values for Step 3.

---

## Step 2: Run Tests

Execute with a configurable timeout (read `config.test_timeout_seconds`; default 300):
```bash
{config.test_command} 2>&1
```

Capture exit code, stdout, stderr. Parse:
- **Counts**: try framework-specific patterns in order:
    - **Jest**: Parse each token independently from the `Tests:` summary line — `(\d+) failed`, `(\d+) passed`, `(\d+) skipped`, `(\d+) total` as separate optional matches (default 0 when absent). IMPORTANT: the `Tests:` line omits "failed" when all tests pass (e.g., `Tests: 26 passed, 26 total`), so a combined pattern requiring all tokens will fail.
    - **pytest**: `(\d+) passed`, `(\d+) failed`, `(\d+) skipped`, `(\d+) error`
    - **Go**: count lines matching `^ok\s+\t` as passed packages, lines matching `^FAIL\t` as failed packages (IMPORTANT: Go emits multiple lines containing `FAIL` per failed package — `--- FAIL: TestName`, standalone `FAIL`, `FAIL\tpkg/path`, and a trailing `FAIL` summary. ONLY `^FAIL\t` followed by a package path represents a failed package. Similarly, `--- PASS:` lines are per-test, not per-package.) In verbose mode (`-v`), also count `--- PASS:` lines for individual test counts and `--- FAIL:` for individual test failures, reporting both: `Tests: {test_passed}/{test_total} passed (packages: {pkg_passed}/{pkg_total})`
    - **Mocha**: `(\d+) passing`, `(\d+) failing`, `(\d+) pending`
    - **RSpec**: `(\d+) examples?,\s*(\d+) failures?(?:,\s*(\d+) pending)?`
    - **Rust/Cargo**: `test result: (?:ok|FAILED)\.\s*(\d+) passed;\s*(\d+) failed;\s*(\d+) ignored` — Cargo emits a single summary line. `ignored` maps to `skipped`. Coverage is not emitted by default; requires `cargo-tarpaulin` or `cargo llvm-cov`.
    - **Python unittest** (stdlib `python -m unittest`): `Ran\s+(\d+)\s+tests?` gives `total`. If a line matches `^OK\b` → `failed=0, passed=total` (`OK \(skipped=(\d+)\)` sets `skipped`). If a line matches `^FAILED\s*\((.+)\)` → inside the parens read `failures=(\d+)` and `errors=(\d+)` (sum → `failed`) and `skipped=(\d+)` if present; then `passed = total - failed - skipped`. unittest emits NO coverage — use `python3 -m coverage run -m unittest && coverage report` for the pytest-cov `TOTAL ... %` line. (Listed — and therefore tried — BEFORE Bun, since Bun also matches `Ran (\d+) tests`.)
    - **Bun**: `(\d+)\s+pass(?:\b)` for passed, `(\d+)\s+fail(?:\b)` for failed — Bun uses present tense (`pass`/`fail`) NOT past tense (`passed`/`failed`). Also `Ran\s+(\d+)\s+tests` for total. `(\d+)\s+skip` for skipped. Coverage requires `--coverage` flag.
    - **Vitest**: Uses the same Istanbul/v8 table format as Jest for coverage. Test counts use `Tests\s+(\d+)\s+passed\s+\((\d+)\)` format — note: no colon after `Tests`, no comma separators. Parse each token independently as with Jest.
    - **Fallback**: generic `(\d+)\s+(?:tests?\s+)?passed`, `(\d+)\s+(?:tests?\s+)?failed`
    - **Go skipped**: count `--- SKIP:` lines for skipped tests (only visible in verbose `-v` mode; if not verbose, skipped count defaults to 0)
    - Compute `total = passed + failed + skipped` when the framework does not emit a total
- **Coverage**: try patterns in order:
    1. `All files\s*\|\s*([\d.]+)` (Istanbul/nyc table format — NOTE: data rows use bare numbers, no `%` sign. The first column after `All files |` is statement coverage.)
    2. `TOTAL\s+.*?([\d.]+)%` (pytest-cov)
    3. `coverage:\s*([\d.]+)%` (Go)
    4. `Statements\s*:\s*([\d.]+)%` (Jest text-summary reporter, NOT the default table)
    5. `([\d.]+)%\s*coverage` (generic fallback)
    Use first match; if none match record `null`.
    **Go multi-package note**: Go emits one `coverage:` line per package. When multiple matches exist, compute the average across all matched values (this approximates aggregate coverage since Go does not produce a single aggregate figure). Ignore `coverage: 0.0%` from packages with `[no test files]`.
- **Status**: exit 0 → `"passing"`, exit 127 (command not found) or 126 (permission denied) → `"error"` with detail `"test command not found or not executable"`, timeout → `"error"` with detail `"timeout after Ns"`, other non-zero → `"failing"`

---

## Step 3: Detect Regressions

Skip regression detection when: (a) first run (no previous state), (b) current run status is `"error"`, or (c) previous run status was `"error"`. In these cases record results only, no alerts. Otherwise compare against the last *successful* (`"passing"` or `"failing"`) run:

| Condition | Type | Severity |
|-----------|------|----------|
| failed increased by > 5 | `regression` | `critical` |
| failed increased by 1–5 | `regression` | `warning` |
| coverage dropped by > 5% | `coverage_drop` | `warning` |
| failed → 0 (was > 0) | `recovery` | `info` |

Alert format: `{"severity": "warning", "type": "regression", "detail": "3 new failures (was 2, now 5)"}`

---

## Step 4: State Update

Write to `agent/state/test_coverage.json`:
```json
{
  "schema_version": "1.0.0",
  "last_run": "ISO 8601",
  "run_count": N,
  "status": "passing|failing|error",
  "results": { "total": N, "passed": N, "failed": N, "skipped": N, "coverage_pct": N.N },
  "previous_results": { "...previous results object..." },
  "previous_status": "passing|failing|error",
  "new_failures": N,
  "coverage_drop": N.N,
  "alerts": [{"severity": "warning", "type": "regression", "detail": "..."}],
  "history": [{"timestamp": "...", "passed": N, "failed": N, "coverage_pct": N.N}]
}
```

- `previous_status` — the prior run's `status`, persisted so Step 3's rule (c)
  ("previous run status was error") is actually decidable next run.
- `new_failures` — `max(0, results.failed - previous_results.failed)` this run
  (0 on first run). `coverage_drop` — `max(0, previous coverage_pct - current)`
  in percentage points. **These two are the variables evolve's 4-B chain
  trigger evaluates** (`new_failures >= 1 OR coverage_drop > 5`) — they must be
  written EVERY run, not only when an alert fires.

History: append the current run, then truncate to the most recent 50 entries (drop oldest first). If the array already exceeds 50 (e.g., manual edits), truncate to 50 in this write.

---

## Step 5: Report

```
Tests: {passed}/{total} passed, {skipped} skipped {coverage_section}
Status: {passing|failing|error}
vs Previous: {delta_section or "first run / no comparison available"}
Alerts: {alert list or "none"}
```

Where:
- `{coverage_section}` = `(X.X% coverage)` when available, or `(coverage: n/a)` when `coverage_pct` is `null`.
- If `total` is 0 and status is not `"error"`, display `Tests: 0 found (check test_command output)`.
- `{delta_section}` omits coverage delta when either the current or previous `coverage_pct` is `null`.

Example: `Tests: 142/145 passed, 0 skipped (87.3% coverage) | Status: failing | vs Previous: +3 failed, coverage -1.2% | Alerts: [warning] regression — 3 new failures`

---

## Graceful Degradation

- No `test_command` → skip with message
- Test command fails to start → record `"error"` status, write state
- Coverage parsing fails → record `coverage_pct: null`, continue
- Timeout (exceeds `config.test_timeout_seconds`, default 300) → kill process, record `"error"` with timeout note
- State file corrupt → treat as first run, re-initialize

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [mataeil](https://github.com/mataeil)
- **Source:** [mataeil/OODA-loop](https://github.com/mataeil/OODA-loop)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-mataeil-ooda-loop-check-tests
- Seller: https://agentstack.voostack.com/s/mataeil
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.