# Test Builder

> >

- **Type:** Skill
- **Install:** `agentstack add skill-oleksiidotsenko-test-builder-test-builder`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [OleksiiDotsenko](https://agentstack.voostack.com/s/oleksiidotsenko)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [OleksiiDotsenko](https://github.com/OleksiiDotsenko)
- **Source:** https://github.com/OleksiiDotsenko/test-builder/tree/main/plugins/test-builder/skills/test-builder

## Install

```sh
agentstack add skill-oleksiidotsenko-test-builder-test-builder
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Test Builder

Build valid, fair, evidence-based assessments. Follow the working algorithm
below; consult the referenced file at each step for the detailed rules.

This skill's files live at `${CLAUDE_PLUGIN_ROOT}/skills/test-builder/` (the
"skill directory"). Reference paths like `references/x.md` are relative to it;
run the bundled scripts by absolute path from the user's working directory so
outputs land in the user's project, never in the plugin cache. § numbers cite
the master guide the references derive from.

## Entry points

- **Build a test / items from scratch** → Steps 0–7.
- **Review an existing test or items** → Steps 5–7 (reconstruct missing
  passports as you go; report per `references/review-checklist.md`).
- **Analyze response data after administration** → Steps 8–10.
- **A single quick item** still gets: mode, Bloom tag, format rules,
  anti-pattern screen, passport, and the human-review flag. No shortcuts.

## Non-negotiable guardrails (apply at every step)

1. **Human-in-the-loop.** Every AI-drafted item is flagged
   `source: AI-generated, reviewed by ` in its passport and must pass
   human expert review against the checklist before use. Never present a
   generated test as ready-to-administer — deliver it as
   "draft pending human review."
2. **Fact-check.** Verify any factual claim in a stem, key, or distractor
   against an authoritative source; fluent phrasing is not evidence. If you
   cannot verify a fact, say so in the passport and mark the item for expert
   verification.
3. **Ethics and child safety by default.** Screen out discriminatory,
   sensitive, dangerous, and elitist contexts; apply crisis/war caution
   (`references/anti-patterns.md` §10.2) unless the teacher has deliberately
   prepared such a learning goal.
4. **Scoring fairness.** Any multiple-response or partial-credit scheme is
   fixed and announced before administration. Refuse to leave scoring
   ambiguous — if the user won't choose, default to the guide's recommended
   scheme and state it explicitly.
5. **Construct purity.** Watch for construct-irrelevant variance (e.g.,
   reading load in a math item) and construct underrepresentation (matrix
   gaps); name them when found.

## The working algorithm

### Step 0 — Decide the mode FIRST

Ask whether the test is formative (low-stakes), diagnostic, or summative
(high-stakes), and identify subject, grade/level, topics, and outcome
strands. The mode dictates difficulty, psychometric rigor, feedback, and
ethics for everything downstream. In formative mode, never use negative
grading; build in spaced repetition and, where relevant, a hinge question
with misconception-mapped options.
→ read `references/principles-and-modes.md`

### Step 1 — Write the specification + matrix

Fill `assets/specification-template.md`: topics and outcome strands with
weights, formats and counts, max score, scoring scheme (announced in
advance), administration time. Build the "topics × Bloom × outcome strands"
matrix (`assets/matrix-template.md`); apply the summative difficulty
distribution (≈20–25% easy / 50–60% optimal / 20–25% hard) when relevant.
Scaffold the matrix with:

```
python3 "${CLAUDE_PLUGIN_ROOT}/skills/test-builder/scripts/new_matrix.py" --topics "Topic A,Topic B" --items 20 --weights "50,50"
```

→ read `references/specification-and-matrix.md`

### Step 2 — Bloom-tag every planned item

Use behavioral action verbs; build a "question ladder" per topic to reach
the higher levels. Keep difficulty and cognitive level distinct — each item
carries both tags independently.
→ read `references/taxonomy.md`

### Step 3 — Mine distractors from real errors

Where possible, pose the question in open form first, cluster the wrong
answers, promote the most frequent to distractors. Document for every
distractor the faulty mental model it embodies — this rationale goes in the
passport and is validated. If no real error data exists, use documented,
named misconceptions and treat the first administration as the mining round.
→ read `references/writing-rules.md` (§9.3)

### Step 4 — Draft items by format rules

Respect per-format constraints: single-best-answer MC (3 strong options
often beat 4 with a dead one; "all/none of the above" forbidden);
multiple-response scoring schemes fixed and announced; True/False in series
on a shared stimulus; matching with surplus responses; ordering unambiguous;
short answer with a full creditable-variants list; extended response with a
mandatory rubric. For competence-based curricula, include at least one
PISA-style stimulus cluster. Randomize key positions across the test.
→ read `references/item-formats.md` and `references/writing-rules.md`;
for subject-specific emphasis → `references/subject-adaptations.md`;
for online delivery → `references/online-and-ai.md`

### Step 5 — Screen against anti-patterns

Check every item, option by option, for the seven cue-giveaways, the
ethics/fairness constraints, and the accessibility/cognitive-load rules.
Cite violations by anchor (e.g., "violates §10.1.5").
→ read `references/anti-patterns.md`

### Step 6 — Complete a passport for every item

Copy `assets/item-passport-template.yaml`; fill id, subject, grade, topic,
outcome strand, Bloom level, mode, format, stem, options, key,
per-distractor rationale, scoring, source, reviewer. Flag AI-generated items
explicitly in `source`. Validate:

```
python3 "${CLAUDE_PLUGIN_ROOT}/skills/test-builder/scripts/validate_passport.py" item-001.yaml item-002.yaml
```

A failing passport is a defective item — fix or discard before review.

### Step 7 — Run the review checklist

Content, form, accessibility, ethics/fairness, and whole-test checks. A
second pair of eyes is mandatory; for AI-generated items, a human expert.
Discard unsound items — do not rescue them. Deliverable wording:
"PASS (pending human sign-off)," never "ready to administer."
→ read `references/review-checklist.md`

### Step 8 — After administration, compute statistics

Export responses to CSV (rows = takers, columns = items, plus a `KEY` row —
or an already-scored 0/1/points matrix) and run:

```
python3 "${CLAUDE_PLUGIN_ROOT}/skills/test-builder/scripts/psychometrics.py" responses.csv --out report.md
```

Interpret P-value, D-index, Rit, and distractor uptake against the guide's
interval tables and calibration anchors; give each item a verdict — bank,
revise, or retire — and record it in its passport.
→ read `references/psychometrics.md`

### Step 9 — Scale and report where needed

Raw score → scale → grade, with scale-type caveats (a 12-point grade is
ordinal; state conversion rules before seeing results).
→ read `references/psychometrics.md` (§12)

### Step 10 — Deliver feedback per the test's mode

Formative: immediate, descriptive, explaining every option, never negative
grading. Diagnostic: by outcome strand, misconception map from distractor
uptake. Summative: score plus item analysis where possible.
→ read `references/principles-and-modes.md` (§4)

## Output conventions

- Deliver: the specification, the matrix, then items — each item followed by
  its passport (YAML block). End with the review-checklist report and the
  human-review disclaimer.
- Scripts are Python 3, standard library only; they run offline and touch
  nothing outside the paths given to them.
- When reviewing, quote the violated anchor (checklist ID and/or guide §) so
  the teacher can look it up.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [OleksiiDotsenko](https://github.com/OleksiiDotsenko)
- **Source:** [OleksiiDotsenko/test-builder](https://github.com/OleksiiDotsenko/test-builder)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-oleksiidotsenko-test-builder-test-builder
- Seller: https://agentstack.voostack.com/s/oleksiidotsenko
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.