# Baseline

> Establish the starting point. Use after implement-and-check and before any algorithm. Creates the run directory, freezes the seeded train/val/test split (written once), scores the unmodified seed capability on val, and records it as the candidate every algorithm must beat. Confirms there is headroom to optimize.

- **Type:** Skill
- **Install:** `agentstack add skill-skillberry-ai-cap-evolve-baseline`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [skillberry-ai](https://agentstack.voostack.com/s/skillberry-ai)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [skillberry-ai](https://github.com/skillberry-ai)
- **Source:** https://github.com/skillberry-ai/cap-evolve/tree/main/skills/phases/baseline

## Install

```sh
agentstack add skill-skillberry-ai-cap-evolve-baseline
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# baseline — freeze splits, score the seed

baseline is the first phase that touches data, so it owns the most
consequential one-time decision in the whole run: **the split**. It writes
`splits.json` once (seeded, reproducible), scores the *unmodified* seed
capability on val, and records that score as the candidate every algorithm must
beat. Get the split right here and every downstream number is honest; get it
wrong and nothing later can fix it.

## Inputs / outputs (manifest tokens)
- **needs:** `project` (the implemented adapter) and `tasks` (the dataset).
- **provides:** `splits` (the frozen train/val/test partition), `baseline` (the
  seed's val score), and `candidate` (the seed, registered as the first best).

## Why it matters
- **Fair comparison point.** Every algorithm hill-climbs *against* the baseline
  val score; a candidate that does not beat it is not progress.
- **Headroom check.** If the seed already scores ~1.0 on val, there is little to
  optimize — stop and save budget rather than chase noise. A near-floor baseline,
  conversely, may signal a broken adapter rather than a hard task.
- **Split sealing.** baseline writes the split *once*; from here on no skill
  re-splits, and **test is untouched until finalize**. Freezing it once (seeded)
  guarantees train/val/test stay disjoint and reproducible across reruns — the
  precondition for the honest test number at the end.

## Splitting choices
- **Seeded ratio split** (default `0.5 / 0.25 / 0.25`): deterministic given
  `--seed`. Reproducible runs partition identically.
- **Pinned split** (`--split-ids`): a JSON `{train,val,test}` of ids — use a
  benchmark's official split, or set all three equal to fit the whole set with
  **no holdout** (the report will then flag the test number as a *fit* metric, not
  a held-out result).
- Tasks must be plentiful enough to split three ways and still leave val/test big
  enough that their standard errors are not dominated by sample size.

## Reusing a prior baseline (`--reuse-baseline PRIOR_RUN_DIR`)
Re-scoring the seed on val every run is wasteful when the split + seed are
unchanged. Pass `--reuse-baseline ` (spec key `reuse_baseline`,
or `cap-evolve run --reuse-baseline ...`) and baseline copies the prior run's
`splits.json`, `baseline.json`, the seed candidate snapshot, and the seed val
rollouts into the fresh run dir, then **skips** the baseline eval — the algorithm
starts at iteration 1 on the reused baseline. The **test seal stays intact**: the
copied split's `test_used` flag is reset so this run can still finalize on test
exactly once. Backward compatible — absent, baseline behaves exactly as before.

## Dual-mode
This phase runs two ways from the **same** SKILL.md: standalone as the slash command `/cap-evolve:baseline` (the `argument-hint` shows its run.py args), and orchestrator-callable — `cap-evolve run` / the `orchestrate` skill invokes the same `scripts/run.py` headlessly and threads the run dir between phases.

## How to run
```
python scripts/run.py --base .capevolve --project .capevolve/project \
    --capability seed_capability --seed 0 --ratios 0.5,0.25,0.25 \
    --max-iterations 10 --stall 2
```
Prints the run-dir path (used by the algorithm + finalize) and the baseline val.
Use `--n-trials ≥ 3` for stochastic agents so the baseline carries an honest
`stderr` for the gate to compare against.

## What good vs bad looks like
- **Good:** a seeded or official split written once; baseline scored with enough
  trials to have real variance; visible headroom between baseline and 1.0.
- **Bad:** re-splitting later in the run (leaks test); a baseline at ~1.0 chased
  anyway (no headroom — wasted budget); a single-trial baseline `stderr=0` that
  makes every later significance comparison meaningless.

## References
- `references/concepts.md` — why splits are frozen once and seeded, the headroom
  check, no-holdout fit metrics, and the train/val/test contract, with sources.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [skillberry-ai](https://github.com/skillberry-ai)
- **Source:** [skillberry-ai/cap-evolve](https://github.com/skillberry-ai/cap-evolve)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-skillberry-ai-cap-evolve-baseline
- Seller: https://agentstack.voostack.com/s/skillberry-ai
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.