# Intake > Phase 1 of the pipeline — collect inputs and scaffold the run. Use at the very start of any optimization. Interviews the user to decide what capability to optimize, which runner/optimizer/algorithm to use, and where the data is; scaffolds .capevolve/project/ (adapter stub, capevolve.yaml, PROJECT.md); and for every NEEDED input that is missing, asks the user (quoting path, how to retrieve it, alt… - **Type:** Skill - **Install:** `agentstack add skill-skillberry-ai-cap-evolve-intake` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [skillberry-ai](https://agentstack.voostack.com/s/skillberry-ai) - **Installs:** 0 - **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills) - **Latest version:** 0.1.0 - **License:** Apache-2.0 - **Upstream author:** [skillberry-ai](https://github.com/skillberry-ai) - **Source:** https://github.com/skillberry-ai/cap-evolve/tree/main/skills/phases/intake ## Install ```sh agentstack add skill-skillberry-ai-cap-evolve-intake ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # intake — collect inputs, scaffold the project The first phase. Its job is to turn a vague wish ("make this agent better at X") into a concrete, runnable project: a filled `capevolve.yaml`, an adapter ready to implement, and **every NEEDED input resolved before any budget is spent**. Intake is cheap; a botched intake is not — an unresolved input discovered three phases later means a wasted optimization run and a meaningless number. ## Inputs / outputs (manifest tokens) - **needs:** *(nothing)* — intake is the pipeline entry point. - **provides:** `project` (the scaffolded `.capevolve/project/`) and `tasks` (the evaluation dataset, resolved either to a path or to the adapter's `tasks()`). Downstream, `implement-and-check` consumes `project`; `baseline` consumes `project` + `tasks`. If intake under-delivers either token, the hard gate in `implement-and-check` fails loudly rather than silently optimizing against a stub. ## What it does 1. **Interview** (driven by this SKILL.md): pick the capability skill (*what* is optimized), the optimizer (*which* coding agent proposes edits), the algorithm (*the search loop*), the dataset, the splits, and the budget. 2. **Scaffold** `.capevolve/project/` from the template (`scripts/run.py`): adapter stub, `inputs/`, `capevolve.yaml`, `PROJECT.md`, and the optimizer-prompt template `optimizer/INSTRUCTIONS.md` (the whole `templates/project/` tree is copytree'd verbatim, so this file is already in place — confirm it exists). 3. **Resolve inputs** per `inputs/INPUTS.md` — the contract below. 4. **Wire trajectories + scoring into the adapter.** From the *trajectories path* and *metric extraction / scoring source* inputs: - implement `adapter.trajectories(split)` to RETURN the runner's native trajectory directory for the last eval of `split` (any structure/format; it is copied verbatim into the optimizer's `./trajectories/`). Return `None` only if there is genuinely no separate native store (cap-evolve then falls back to its per-rollout JSON) — note that choice in `PROJECT.md`. - implement `score()` to extract the OBJECTIVE metric from a rollout, matching the benchmark's own scoring source; verify it reproduces the benchmark's number. - **Make `score()`'s feedback ARGUMENT-LEVEL — it IS the learning signal.** A tool-name-only signal ("action X was wrong") is too coarse for the optimizer to localize a fix; it pattern-matches to prose rules and the run plateaus. For EACH failing check, the feedback must point at the specific argument/value/step that was wrong: name the wrong ARGUMENT key and the **agent's OWN wrong value** (not the gold value), name the wrong target id, and for communication/omission misses name the value or field the agent failed to state **when it is derivable from the agent's own state** (e.g. an un-stated computed total). This is **gold-SAFE**: derive everything from the agent's own messages/tool-calls/observed state (and the user's own profile/db state the agent saw) — use the gold record ONLY to learn WHICH check/argument failed (key names are safe; gold VALUES must never be read or printed). When a piece is not safely derivable, fall back to the coarser tool-name message. Keep `score()` deterministic (the check gate requires it). 5. **Author the optimizer instructions for THIS benchmark — SCOPED TO THE SELECTED CAPABILITIES.** Customize the scaffolded `.capevolve/project/optimizer/INSTRUCTIONS.md`. Keep the `{{...}}` placeholders intact (`{{FOCUS_SUMMARY}}`, `{{FAILURES}}`, `{{CAP_BRIEF}}`, `{{ALGO_BRIEF}}`, `{{BENCH_REPO}}` — the harness fills them per iteration). Keep the authored static guidance **short on meta-narration, explicit and DEMANDING on iteration depth**, and make it **capability-scoped**: include guidance, skill references, and edit-space ONLY for the capabilities actually listed in `capevolve.yaml: capabilities`. - **DEPTH MANDATE — address ALL failure clusters each iteration.** The authored instructions must demand a substantial multi-root-cause pass. Produce this target snippet: > "Each iteration is a substantial, multi-root-cause pass. Diagnose ALL clusters > and fix as many as possible in ONE candidate — improve multiple tools' code, > validation, and return values/errors; add new tools; sharpen many tool docs; > and fix the prompt — together. Scope each fix to protect passing tasks; do NOT > trade breadth for caution. A single small edit is an under-used iteration." - **State the GOAL up front:** maximize the eval score — make the largest improvement you can this iteration, grounded in the trajectories. - **The authored INSTRUCTIONS MUST encode all three of these (generic, capability-scoped):** 1. **STEP-0 reading mandate.** Before diagnosing, the optimizer must READ `./guidance//SKILL.md` (for EACH selected capability) and the optimizer features reference under `./guidance/optimizer/`. State this as an explicit first step. 2. **The EXISTING-tool-code mandate** (when `tools` is selected). Demand: convert violated textual rules into in-code checks across MANY EXISTING tool bodies — most violated rules govern a tool that already exists, so the fix is an in-body guard there, not a new tool. State plainly: *a docstring-only iteration (or one that only adds a single new tool + rewords docstrings, leaving rules as prose) is under-used.* 3. **The explicit TWO-PHASE subagent pattern.** Require: Phase 1 — diagnose fan-out (one read-only subagent per trajectory-group → tight issue list; main dedups into clusters); Phase 2 — implement fan-out (one edit-subagent per ISSUE, each in its own worktree, each PREFERRING to edit the EXISTING tool's code body to enforce its rule); then the main agent MERGES all edits into ONE candidate. Point at `./guidance/optimizer/.md` for the agent's concrete trigger phrasing. 4. **The NON-OVERFITTING guardrail.** Demand that every prompt/tool edit encode a GENERAL rule/policy/validation that generalizes across the whole class of inputs — NEVER hardcode a specific task's id/value/date/name/answer. A guard must fire on the general condition (e.g. "id not in the user's profile"), not match a literal value (NOT `if id == ""`). A literal special-case that only helps one task is forbidden — it overfits, fails the held-out gate, and hurts other tasks. Per-task specifics are for understanding the failure CLASS only; the fix must be general. 5. **EXPLOIT ground-truth/eval present in the trajectories (diagnosis only).** Tell the optimizer that when `./trajectories/` include ground-truth / expected actions / a reward breakdown, it should USE them during diagnosis to localize the exact defect (expected vs actual action/argument/value) — and if not present, infer from the traces + feedback. State plainly that ground truth informs the failure class only; the resulting edit must still be GENERAL (guardrail 4) and never copy a gold value. - **Capability-scoping (the key rule):** reference `./guidance//SKILL.md` and present the editable artifacts **for the selected caps only**. If only `tools` is selected, do NOT include any prompt-editing guidance, do NOT reference the `system-prompt` skill, and do NOT present the prompt/policy file as editable. If only `system-prompt` is selected, do not surface the tools file as editable. Each capability's "What you can change here" lives in its `./guidance//SKILL.md` — point the optimizer there rather than restating it. - **Always include (capability-agnostic):** READ the four cross-iteration files in the working dir FIRST — `./LEDGER.md` (framework facts: each iteration's outcome + tasks broken/fixed), the whole `./JOURNAL.md` (the optimizer's append-only handover across the run), and `./RUNMAP.md` + `./prior_iterations//` (every prior iteration's PROCESS.md + capability diff) — and never re-propose an approach the journal/ledger shows was rejected *as implemented* (a better-designed version may still work — not a permanent ban); READ and USE the diagnose skill `./guidance/diagnose/SKILL.md` (incl. its KNOWLEDGE / BEHAVIORAL / CAPABILITY-GAP tags), the optimizer features reference `./guidance/optimizer/.md`, this step's `./trajectories/`, and any `./guidance/sources/` data-model files. Each iteration the optimizer MUST: fill `./PROCESS.md` (the required explainability template — ranked issues + tags, every edit + class, verify-the-fix, subagents/features used, what to preserve, what was skipped) and APPEND its entry to `./JOURNAL.md` below the marker (what was tried / worked / regressed / refuted / plateau-signal / focus-next). Ship MULTIPLE edit classes and ADD a new code-bearing tool whenever a CAPABILITY-GAP/stall cluster is present. 6. **Set the spec keys** in `capevolve.yaml`: - `runner_repo_path` — the benchmark/runner source, surfaced read-only to the optimizer. - `optimizer_instructions_file` — point at the customized template (default `optimizer/INSTRUCTIONS.md`). - `capability_sources` — the benchmark's data-model / types source files that the tools import (resolved relative to the project dir; copied verbatim into the optimizer's `./guidance/sources/`), so the optimizer can write correct code against the real types. Set this whenever a selected capability's code imports a shared types/data-model module; leave the default `[]` when there is none. ## Ask-the-user-if-missing (mandatory — the core discipline) Read `inputs/INPUTS.md`. It classifies every input as **NEEDED** or **RECOMMENDED**. For each **NEEDED** input that is not already present: > **ASK THE USER.** Quote (a) the exact path where it is expected, (b) the > command or option that produces it, and (c) any alternatives. Then wait. **Never invent a NEEDED input.** Fabricating a dataset, a scorer, or a gold answer does not unblock the run — it produces a number that measures nothing and hides that fact. A missing tasks file is a *question for the user*, not a gap for you to paper over. This is the single most important behavior of this phase. **RECOMMENDED** inputs have sane defaults and may be skipped — but log every skip in `PROJECT.md` (e.g. "num_trials defaulted to 1 — scores will be single-trial, so the significance gate will correctly reject marginal gains"), so the honesty cost of each default is visible at report time. ### Block on a missing NEEDED input (never fabricate) The action when a NEEDED input is absent depends on the run mode: - **Interactive / chat mode** — ASK THE USER and wait: quote *what* is needed, *why* it is needed (what breaks without it), and *how* to provide it (the exact path / command / option / alternatives from `INPUTS.md`). Do not proceed past the missing input. - **Non-interactive mode** (`cap-evolve run` / orchestrate, no human to ask) — do NOT fabricate. WRITE a clearly delimited section into `PROJECT.md`: `BLOCKED: — why it is needed — how to provide it`, then STOP with a non-zero exit. A blocked-but-honest stop is correct; a green run on a guessed input is not. This extends the ask-if-missing discipline above — it is the same rule, with an explicit non-interactive fallback so a headless run fails loud and recorded instead of silently inventing a dataset, scorer, trajectories path, or scoring source. ### Why a contract, not a guess The classic failure mode of "auto-optimize my agent" tooling is to start running with whatever it can find and backfill assumptions. That yields a green run and a worthless result. Splitting inputs into NEEDED (blocking → ask) vs RECOMMENDED (default → log) makes the *only* legitimate way to proceed-without-an-input an explicit, recorded default — never a silent fabrication. Treat `INPUTS.md` as the spec; this SKILL.md is just the procedure for honoring it. ## Dual-mode This phase runs two ways from the **same** SKILL.md: standalone as the slash command `/cap-evolve:intake` (the `argument-hint` shows its run.py args), and orchestrator-callable — `cap-evolve run` / the `orchestrate` skill invokes the same `scripts/run.py` headlessly and threads the run dir between phases. ## How to run ``` python scripts/run.py --base .capevolve # scaffold .capevolve/project ``` The script is purely mechanical: it copies the template and prints the next steps. The *judgment* — interviewing, choosing components, and running the ask-if-missing loop — is yours, driven by this SKILL.md and `inputs/INPUTS.md`. Then implement `adapters/adapter.py`, fill `capevolve.yaml`, and proceed to `implement-and-check`. Together, **intake → implement-and-check** is the *full integration*: scaffold → implement the 4 adapter methods → `cap-evolve check` green, before any budget is spent. The using-agent (e.g. the chosen optimizer) can run this whole integration autonomously. > **Worked example (onboard a new benchmark from a prompt):** see `examples/` for > an end-to-end onboarding. The intake/onboarding step **installs the benchmark** > (clones + installs it) and the **optimizer agent** wires the adapter from the > stub until `cap-evolve check` passes, then optimizes the selected capability. The > example's `setup.sh` is the executable transcript of that onboarding; `run.sh` > runs the full optimization with the live dashboard. ## What good vs bad intake looks like - **Good:** every NEEDED input resolved to a real path or `"adapter"`; splits and budget chosen deliberately; each defaulted RECOMMENDED input logged in `PROJECT.md`; `capevolve.yaml` fully filled; the user answered every blocking question before the scaffold was declared done. - **Bad:** a tasks file that "looked plausible" was synthesized; the scorer leaks the gold answer into feedback; test == train with no note; budget left at a default that cannot possibly find a gain; the run proceeded past a missing NEEDED input "to keep moving". - **Bad:** authored INSTRUCTIONS that let an iteration pass by adding one tool + rewording docstrings (leaving violated rules as prose) — or that omit the STEP-0 reading mandate, the existing-tool-code mandate, or the explicit two-phase (diagnose fan-out → implement fan-out → merge) subagent pattern. ## References - `references/concepts.md` — the inputs contract, NEEDED vs RECOMMENDED rationale, the four adapter methods, and split/trial/budget guidance with sources. ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [skillberry-ai](https://github.com/skillberry-ai) - **Source:** [skillberry-ai/cap-evolve](https://github.com/skillberry-ai/cap-evolve) - **License:** Apache-2.0 Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-skillberry-ai-cap-evolve-intake - Seller: https://agentstack.voostack.com/s/skillberry-ai - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.