# Psy Exp Reviewer

> Use for reviewing psychological experiment code quality at any stage — from early design idea through completed code. Supports five modes: code-audit (full platform-aware code review with smoke test protocol), config-audit (pre-code design review), implementation-plan-review (architecture review), triage-only (missing-information checklist from natural-language idea), and blocked (insufficient in…

- **Type:** Skill
- **Install:** `agentstack add skill-soupandpsy-amazing-psycoder-skills-psy-exp-reviewer`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [soupandpsy](https://agentstack.voostack.com/s/soupandpsy)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [soupandpsy](https://github.com/soupandpsy)
- **Source:** https://github.com/soupandpsy/amazing-psycoder-skills/tree/main/amazing-psycoder/psy-exp-reviewer

## Install

```sh
agentstack add skill-soupandpsy-amazing-psycoder-skills-psy-exp-reviewer
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Psychological Experiment Code Reviewer

## Version

v1.3 — stable, 2026-06-10. Sub-skill of [amazing-psycoder](../SKILL.md).

## Purpose

Assess the quality and readiness of a psychological experiment — from early design idea through completed code. The reviewer adapts its mode to the input available. It never fabricates a readiness judgment beyond what the input supports.

This is the **final mandatory gate** in the experiment development chain. It evaluates code generated by psy-exp-coder, identifies issues, and enters a **check → fix → re-check loop** with the coder. Each audit round identifies remaining issues, the coder applies fixes, and the reviewer re-audits. This cycle repeats until zero Critical and zero Major issues remain. Only then is the final file delivered and data collection can begin. No experiment code proceeds to data collection without passing reviewer audit with `ready_for_collection` or `ready_after_minor_fixes`.

### What "Production-Ready" Means

Code that passes reviewer audit must meet five conditions:

1. **Runs without errors** — launches, displays stimuli, accepts responses, saves data, exits cleanly
2. **Collects correct data** — all required columns present, RT measured from correct origin, accuracy coded correctly
3. **Does not crash mid-experiment** — try/finally protects data, Escape works in every loop
4. **Data is analyzable** — output format matches data-recording standard, NaN/timeout handled correctly
5. **Experiment logic is correct** — trial sequence matches paradigm, response mapping unambiguous, condition ratios verified

## Integration with Coder Skill

The reviewer cross-references the [psy-exp-coder](../psy-exp-coder/SKILL.md) skill's artifacts:

| Coder artifact | Reviewer use |
|---------------|-------------|
| Platform spec Canonical Skeleton | Reference for correct API patterns — compare generated code against skeleton |
| Platform spec anti-pattern table | Checklist of forbidden patterns to scan for |
| Coder Quality Gate (9 items) | Minimum bar — if any gate fails, automatic `not_ready_for_collection` |
| Platform mapping README | Verify config→code mapping correctness |
| Paradigm reference files | Verify experiment logic (window sequence, accuracy rules) matches paradigm |

The reviewer also cross-references the [psy-exp-designer](../psy-exp-designer/SKILL.md) skill's artifacts:

| Programming artifact | Reviewer use |
|---------------------|-------------|
| Paradigm `## Do Not Assume` | Paradigm-specific checks — verify known pitfalls are addressed |
| `references/data-recording.md` | Data output column validation — all 10 base columns present |
| `references/config-schema.md` | Config validation rules — all 9 checks pass before code generation |

## Review Modes

Before reviewing, classify the request into one mode.

| Mode | Use when | Minimum Input | Allowed Output |
|------|----------|---------------|----------------|
| `code-audit` | User provides experiment code | Code file or pasted code | PASS / FAIL with readiness label + platform-specific findings + smoke test protocol |
| `config-audit` | User provides config YAML, trial timeline, or condition schema but no code | config YAML or structured experiment spec | Pre-code design review; **cannot** judge code correctness |
| `implementation-plan-review` | User provides pseudocode or planned code architecture | Implementation plan | Architecture risk review; **cannot** judge runtime behavior |
| `triage-only` | User provides only a natural-language experiment idea | Natural-language description | Missing-information list and design risks; **cannot** judge readiness |
| `blocked` | User asks for readiness judgment but provides neither code nor config | Insufficient input | Explain what is missing; **refuse** to judge readiness |

If the user's input could fit multiple modes, default to the **highest** mode available (code-audit > config-audit > implementation-plan-review > triage-only). If input is insufficient for any productive review, use `blocked`.

## Readiness Labels

| Label | Allowed in mode | Meaning |
|-------|----------------|---------|
| `ready_for_collection` | `code-audit` only | Zero critical or major issues; code matches platform spec skeleton; smoke test passed |
| `ready_after_minor_fixes` | `code-audit` only | Only minor issues remain; smoke-testable but fix before analysis |
| `not_ready_for_collection` | `code-audit`, `config-audit` | Critical or major issues exist; do NOT collect data |
| `pre_code_ready` | `config-audit` only | Config/spec complete and ready for code generation |
| `needs_experiment_info` | `triage-only`, `config-audit` | Key design information is missing |
| `blocked` | `blocked` | Cannot review; input is insufficient |

**Hard rule**: `ready_for_collection` and `ready_after_minor_fixes` require actual code review. If no code was provided, the highest possible label is `pre_code_ready`.

## Scope Limitation Rule

At the start of every review output, state what was and was not reviewed. If no code was provided:

> **Scope**: No experiment code was provided. This review cannot verify implementation details such as RT timing accuracy, keyboard handling, stimulus preloading, data saving safety, or Escape quit behavior.

## Platform Detection (code-audit only)

When code is provided, first detect the platform:

| Signature | Platform |
|-----------|----------|
| `from psychopy import` / `visual.Window` / `keyboard.Keyboard` | PsychoPy |
| `initJsPsych` / `jsPsych.run` / `jsPsychHtmlKeyboardResponse` | jsPsych |
| `PsychImaging` / `Screen('Flip'` / `KbQueueCreate` / `sca` | Psychtoolbox |

Once detected, load the corresponding coder spec for authoritative API patterns:
- PsychoPy → `../psy-exp-coder/psychopy/spec/README.md`
- jsPsych → `../psy-exp-coder/jspsych/spec/README.md`
- Psychtoolbox → `../psy-exp-coder/psychtoolbox/spec/README.md`

Also load the paradigm file from the programming layer for failure mode cross-reference:
- `../psy-exp-designer/paradigms/{paradigm_name}.md`

If the platform cannot be detected, flag this as a critical issue.

---

## Review Checklist — code-audit

### Gate 0: Coder Quality Gate (minimum bar)

Run the 9-item Quality Gate from the coder SKILL.md first. Any failure = automatic critical issue.

| # | Check | Pass condition | How to verify |
|---|-------|---------------|---------------|
| 1 | Skeleton match | Code structure matches platform Canonical Skeleton | Compare section-by-section: imports → params → display → preload → trial loop → data save → cleanup. Order and structure must match |
| 2 | No anti-patterns | Zero occurrences of any pattern in platform spec anti-pattern table | `grep` for each forbidden pattern. See §Platform Anti-Pattern Grep Patterns below |
| 3 | Spec API patterns used | API patterns in code come from platform spec canonical skeleton, not paradigms/demo files | Check RT source, keyboard API, timing loop, data save pattern against spec skeleton — not against paradigm .md examples |
| 4 | Parameters at top | All editable values in parameter block; no magic numbers | Scan for hardcoded numbers in trial loop: durations, colors, key names, file paths. Every number in trial logic must reference a variable from the params block |
| 5 | Escape in every loop | Every while loop that contains Flip/flip/frame-draw includes an escape/abort check | Count `while` loops in the trial section. Count escape checks. Must be equal |
| 6 | RT source | Platform-correct RT source (see §Platform-Specific Timing & RT) | Find where `rt` is assigned. Verify it comes from the correct source for the detected platform |
| 7 | Incremental save | Per-trial data flush; try/finally or try/catch/sca | Find the data save call. Is it inside the trial loop? Is it wrapped in try/finally? |
| 8 | Preload outside loop | No stimulus construction inside trial loop | Scan trial loop for `imread`, `ImageStim(`, `MakeTexture`, `Sound(` |
| 9 | FONT_CONFIG | CJK text → FONT_CONFIG toggle present | If text contains Chinese/Japanese/Korean characters, verify FONT_CONFIG block with OS auto-detect exists in params section |

### Platform Anti-Pattern Grep Patterns

When auditing code, scan for these exact patterns per platform:

**PsychoPy (14 patterns to scan):**
```
event.getKeys(           ← blocks event loop
event.waitKeys(          ← blocks event loop
kb.waitKeys(             ← blocks event loop in trial
time.sleep(              ← blocks event loop
core.wait(               ← blocks event loop (>5ms)
imread(                  ← inside trial loop = frame drop
ImageStim(               ← inside trial loop = jitter
time.time()              ← wrong RT source
clock.getTime()          ← wrong RT source (use key.rt)
waitRelease=True         ← adds release delay to RT
Keyboard()               ← missing backend='ptb'
.exec(                   ← Pavlovia incompatible
preBuffer=-1             ← missing on Sound()
addLoop(                 ← called before loop runs (not right before)
```

**Psychtoolbox (15 patterns to scan):**
```
WaitSecs(                ← blocks execution (except for ITI pre-trial)
KbWait                   ← blocking, no RT timestamp
KbCheck                  ← inside response loop (use KbQueueCheck)
GetSecs - stimOnset      ← manual RT calculation
input(                   ← invisible in fullscreen
imread(                  ← inside trial loop
MakeTexture(             ← inside trial loop
KbQueueCreate(           ← inside trial loop (should be before)
KbQueueStart(            ← inside trial loop (should be before)
KbQueueFlush             ← MISSING at trial start
SkipSyncTests, 1         ← sync tests skipped
Sound(                   ← high latency audio
DrawText(..., '+'        ← text-based fixation cross
Screen('Flip', w)        ← missing 'when' parameter
Screen('Flip', w, 0)     ← zero when parameter
```

**jsPsych (12 patterns to scan):**
```
jsPsych.init(            ← v7 removed
'html-keyboard-response' ← string plugin type (should be class)
jsPsych.NO_KEYS          ← v7 removed (use "NO_KEYS")
jsPsych.ALL_KEYS         ← v7 removed (use "ALL_KEYS")
timelineVariable('x')    ← missing second argument (should be true)
Date.now()               ← manual RT timing
setTimeout(              ← breaks event loop
setInterval(             ← breaks event loop
XMLHttpRequest           ← in-trial network call
fetch(                   ← in-trial network call
keyCode:                 ← hardcoded keyCode number
.select('col')           ← v7 removed method
```

### 1. Experiment Logic

- [ ] Trial window sequence matches the stated paradigm (compare against paradigm file `## Trial Window Timeline`)
- [ ] Each window has defined content, duration, response rule — no window is ambiguous
- [ ] Correctness rule is unambiguous for every trial type (including no-go/catch/timeout)
- [ ] Condition table covers all factorial combinations
- [ ] Block transitions handled (rest screens, instructions between blocks)
- [ ] Feedback only in practice blocks (unless explicitly designed otherwise)
- [ ] **NEW** Instruction text includes all key mapping information (which key = which answer)
- [ ] **NEW** Debrief/thank-you screen exists and provides clean exit path

### 2. Platform-Specific Timing & RT

**PsychoPy:**
- [ ] `keyboard.Keyboard(backend='ptb')` — not default backend (50-70ms RT error if missing)
- [ ] `win.callOnFlip(kb.clock.reset)` — clock reset at stimulus onset, not before flip
- [ ] `kb.getKeys(waitRelease=False)` — not default True (adds 100-200ms)
- [ ] `key.rt` used for RT — not `kb.clock.getTime()`, not `time.time()`
- [ ] `CountdownTimer` loop for response deadline — not `core.wait()`
- [ ] `win.getFutureFlipTime(clock=routineTimer)` for frame timing — not `trialClock.getTime()`
- [ ] **NEW** `kb.clearEvents()` BEFORE stimulus flip — prevents pre-stimulus key contamination
- [ ] **NEW** `sound.Sound(preBuffer=-1)` if audio used — PTB backend for low latency
- [ ] **NEW** `win.callOnFlip(port.setData, code)` if EEG trigger — trigger AFTER flip, not before
- [ ] **NEW** `thisExp.addLoop(trials)` called right before the loop runs, not at experiment start

**jsPsych:**
- [ ] `initJsPsych()` + `jsPsych.run()` — not `jsPsych.init()`
- [ ] Plugin types are class references — not strings
- [ ] `data.rt` used for RT — no manual `Date.now()` timing
- [ ] `trial_duration` parameter for timing — not `setTimeout`/`setInterval`
- [ ] `"NO_KEYS"` / `"ALL_KEYS"` strings — not `jsPsych.NO_KEYS`
- [ ] **NEW** `preload` plugin as FIRST timeline node — not after stimuli
- [ ] **NEW** `'escape'` in `choices` array for every response trial, or escape check in `on_finish`
- [ ] **NEW** `jsPsych.data.get().localSave('csv', filename)` in `on_finish` — not in trial callbacks
- [ ] **NEW** `timeline_variables` pre-computed at script load — not generated at runtime
- [ ] **NEW** `jsPsych.pluginAPI.compareKeys()` for accuracy — not manual `==` comparison

**Psychtoolbox:**
- [ ] `KbQueueCreate` + `KbQueueCheck` — not `KbCheck` for RT
- [ ] `VBLTimestamp` from `Screen('Flip')` as RT origin — not `GetSecs`
- [ ] `firstPress - stimOnset` for RT calculation (ms precision)
- [ ] `vbl + (waitframes - 0.5) * ifi` for frame timing — not `WaitSecs()`
- [ ] `KbQueueFlush([], 2)` at start of each trial
- [ ] `try/catch/sca/Priority(0)/ShowCursor` — not bare `sca`
- [ ] **NEW** KbQueue lifecycle: `Create` + `Start` before trial loop, `Stop` + `Release` after loop — never inside
- [ ] **NEW** `Screen('Flip', window, vbl + (wf-0.5)*ifi)` — every Flip has `when` parameter
- [ ] **NEW** `Screen('DrawingFinished')` before Flip if heavy drawing — prevents frame overrun
- [ ] **NEW** `InitializePsychSound(1)` + `PsychPortAudio('Open',...,2,...)` if audio — low-latency mode
- [ ] **NEW** `SkipSyncTests, 0` — production must not skip sync tests
- [ ] **NEW** `HideCursor` + `Priority(MaxPriority(window))` — called after window open, before trial loop

### 3. Response Collection

- [ ] Response keys validated against allowed set (not arbitrary keys)
- [ ] Anticipatory responses (RT  10,000ms trials deleted; subjects with >10% RT  "第 N 轮审计：发现 X 个问题。修复后进入第 N+1 轮。"

每次修复后必须重新审计。审计轮次和每轮问题数记录在最终报告中。

### After PASS (ready_for_collection)

When the audit passes with `ready_for_collection` or `ready_after_minor_fixes`:

> "审计通过。下一步:  
>  1. 完成 [First-Run Checklist](#first-run-checklist-pre-first-subject) 的所有项目  
>  2. 确认 Smoke Test Protocol 已通过，代码在目标机器上正常工作  
>  3. 保存审计报告作为文档  
>  4. 如需分析数据，使用 `/amazing-psycoder` 进入分析流水线"

---

## Related Files

| File | When to load |
|------|-------------|
| [../psy-exp-coder/psychopy/spec/README.md](../psy-exp-coder/psychopy/spec/README.md) | PsychoPy code audit — canonical skeleton + 14 anti-patterns |
| [../psy-exp-coder/jspsych/spec/README.md](../psy-exp-coder/jspsych/spec/README.md) | jsPsych code audit — canonical skeleton + 12 anti-patterns |
| [../psy-exp-coder/psychtoolbox/spec/README.md](../psy-exp-coder/psychtoolbox/spec/README.md) | PTB code audit — canonical skeleton + 15 anti-patterns |
| [../psy-exp-designer/paradigms/](../psy-exp-designer/paradigms/) | Paradigm failure mode cross-reference |
| [../psy-exp-designer/references/data-recording.md](../psy-exp-designer/references/data-recording.md) | Data output column validation |
| [../psy-exp-designer/references/config-schema.md](../psy-exp-designer/references/config-schema.md) | Config validation rules |

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [soupandpsy](https://github.com/soupandpsy)
- **Source:** [soupandpsy/amazing-psycoder-skills](https://github.com/soupandpsy/amazing-psycoder-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-soupandpsy-amazing-psycoder-skills-psy-exp-reviewer
- Seller: https://agentstack.voostack.com/s/soupandpsy
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.