Red Team

Preamble: Interaction Conventions

idstack skills are designed to run in multiple CLIs (Claude Code, OpenAI Codex CLI, and others). To stay portable, skill bodies use a few concept names that have a CLI-specific implementation:

• AskUserQuestion — when a skill says "ask via AskUserQuestion" or "using AskUserQuestion",

it means: present a single numbered multiple-choice question (e.g., "Which of these best describes X? 1) ..., 2) ..., 3) Other") and stop, waiting for the user's next message before proceeding. Ask one question at a time, never batch. In Claude Code this maps to the AskUserQuestion tool; in Codex CLI (which has no analog) just emit the numbered question as plain text and wait.

• Agent / Skill (sub-task dispatch) — when a skill says "if the Agent tool is available,

dispatch X as a sub-task," that's an optimization. If your CLI has no equivalent, fall through to the inline written-out steps that follow — every skill that uses Agent ships a sequential fallback alongside it.

• Skill (cross-skill invocation) — used only by /idstack:pipeline. In Claude Code this

invokes a child skill in-process via the Skill tool. In CLIs without that primitive, the pipeline degrades to prompting the user to type the next skill name explicitly.

• Skill invocation syntax in user-facing text — when a skill prints "run /idstack:foo"

or links to /idstack:foo in a "Next steps" section, that's Claude Code's /idstack: invocation. In Codex CLI, translate to $foo (drop the /idstack: prefix, replace the leading / with $). Apply this same translation when the skill emits handoff instructions in reports, manifests, or AskUserQuestion options. Same body text, two hosts; the model translates per-CLI on output.

These are directives to the model, not magic words — interpret them as the protocol above.

Preamble: Update Check

# Locate the idstack install. Supports Claude Code (default), Codex CLI, and a
# user override via $IDSTACK_HOME.
if [ -n "${CLAUDE_PLUGIN_ROOT:-}" ]; then
  _IDSTACK="$CLAUDE_PLUGIN_ROOT"
elif [ -n "${IDSTACK_HOME:-}" ]; then
  _IDSTACK="$IDSTACK_HOME"
elif [ -d "$HOME/.agents/plugins/idstack" ]; then
  _IDSTACK="$HOME/.agents/plugins/idstack"
elif [ -d "$HOME/.agents/skills/idstack" ]; then
  _IDSTACK="$HOME/.agents/skills/idstack"
else
  # Claude Code caches marketplace plugins under a versioned dir; take the
  # highest version present. Empty if idstack was never installed this way —
  # every "$_IDSTACK/bin/..." call below is guarded, so that degrades quietly.
  _IDSTACK=$(ls -d "$HOME"/.claude/plugins/cache/idstack/idstack/*/ 2>/dev/null | sort | tail -1)
  _IDSTACK="${_IDSTACK%/}"
fi
_UPD=$("$_IDSTACK/bin/idstack-update-check" 2>/dev/null || true)
[ -n "$_UPD" ] && echo "$_UPD"

If the output contains UPDATE_AVAILABLE: tell the user "A newer version of idstack is available. Run cd $_IDSTACK && git pull && ./setup to update. (The ./setup step is required — it cleans up legacy symlinks.)" Then continue normally.

Preamble: Project Manifest

Before starting, check for an existing project manifest.

if [ -f ".idstack/project.json" ]; then
  echo "MANIFEST_EXISTS"
  "$_IDSTACK/bin/idstack-migrate" .idstack/project.json 2>/dev/null || cat .idstack/project.json
else
  echo "NO_MANIFEST"
fi

If MANIFEST_EXISTS:

• Read the manifest. If the JSON is malformed, report the specific parse error to the

user, offer to fix it, and STOP until it is valid. Never silently overwrite corrupt JSON.

• Preserve all existing sections when writing back.

If NO_MANIFEST:

• This skill will create or update the manifest during its workflow.

Preamble: Preferences

if [ -f ".idstack/project.json" ] && command -v python3 &>/dev/null; then
  python3 -c "
import json, sys
try:
    data = json.load(open('.idstack/project.json'))
    prefs = data.get('preferences', {})
    v = prefs.get('verbosity', 'normal')
    if v != 'normal':
        print(f'VERBOSITY:{v}')
except: pass
" 2>/dev/null || true
fi

If VERBOSITY:concise: Keep explanations brief. Skip evidence citations inline (still follow evidence-based recommendations, just don't cite tier codes in output). If VERBOSITY:detailed: Include full evidence citations, alternative approaches considered, and rationale for each recommendation. If VERBOSITY:normal or not shown: Default behavior — cite evidence tiers inline, explain key decisions, skip exhaustive alternatives.

Preamble: Designer Profile

_PROFILE="$HOME/.idstack/profile.yaml"
if [ -f "$_PROFILE" ]; then
  # Simple YAML parsing for experience_level (no dependency needed)
  _EXP=$(grep -E '^experience_level:' "$_PROFILE" 2>/dev/null | sed 's/experience_level:[[:space:]]*//' | tr -d '"' | tr -d "'")
  [ -n "$_EXP" ] && echo "EXPERIENCE:$_EXP"
else
  echo "NO_PROFILE"
fi

If EXPERIENCE:novice: Provide more context for recommendations. Explain WHY each step matters, not just what to do. Define jargon on first use. Offer examples. If EXPERIENCE:intermediate: Standard explanations. Assume familiarity with instructional design concepts but explain idstack-specific patterns. If EXPERIENCE:expert: Be concise. Skip basic explanations. Focus on evidence tiers, edge cases, and advanced considerations. Trust the user's domain knowledge. If NO_PROFILE: On first run, after the main workflow is underway (not before), mention: "Tip: create ~/.idstack/profile.yaml with experience_level: novice|intermediate|expert to adjust how much detail idstack provides."

Preamble: Context Recovery

Check for session history and learnings from prior runs.

# Context recovery: timeline + learnings
_HAS_TIMELINE=0
_HAS_LEARNINGS=0
if [ -f ".idstack/timeline.jsonl" ]; then
  _HAS_TIMELINE=1
  if command -v python3 &>/dev/null; then
    python3 -c "
import json, sys
lines = open('.idstack/timeline.jsonl').readlines()[-200:]
events = []
for line in lines:
    try: events.append(json.loads(line))
    except: pass
if not events:
    sys.exit(0)

# Quality score trend
scores = [e for e in events if e.get('skill') == 'course-quality-review' and 'score' in e]
if scores:
    trend = ' -> '.join(str(s['score']) for s in scores[-5:])
    print(f'QUALITY_TREND: {trend}')
    last = scores[-1]
    dims = last.get('dimensions', {})
    if dims:
        tp = dims.get('teaching_presence', '?')
        sp = dims.get('social_presence', '?')
        cp = dims.get('cognitive_presence', '?')
        print(f'LAST_PRESENCE: T={tp} S={sp} C={cp}')

# Skills completed
completed = set()
for e in events:
    if e.get('event') == 'completed':
        completed.add(e.get('skill', ''))
print(f'SKILLS_COMPLETED: {','.join(sorted(completed))}')

# Last skill run
last_completed = [e for e in events if e.get('event') == 'completed']
if last_completed:
    last = last_completed[-1]
    print(f'LAST_SKILL: {last.get(\"skill\",\"?\")} at {last.get(\"ts\",\"?\")}')

# Pipeline progression
pipeline = [
    ('needs-analysis', 'learning-objectives'),
    ('learning-objectives', 'assessment-design'),
    ('assessment-design', 'course-builder'),
    ('course-builder', 'course-quality-review'),
    ('course-quality-review', 'accessibility-review'),
    ('accessibility-review', 'red-team'),
    ('red-team', 'course-export'),
]
for prev, nxt in pipeline:
    if prev in completed and nxt not in completed:
        print(f'SUGGESTED_NEXT: {nxt}')
        break
" 2>/dev/null || true
  else
    # No python3: show last 3 skill names only
    tail -3 .idstack/timeline.jsonl 2>/dev/null | grep -o '"skill":"[^"]*"' | sed 's/"skill":"//;s/"//' | while read s; do echo "RECENT_SKILL: $s"; done
  fi
fi
if [ -f ".idstack/learnings.jsonl" ]; then
  _HAS_LEARNINGS=1
  _LEARN_COUNT=$(wc -l /dev/null | tr -d ' ')
  echo "LEARNINGS: $_LEARN_COUNT"
  if [ "$_LEARN_COUNT" -gt 0 ] 2>/dev/null; then
    "$_IDSTACK/bin/idstack-learnings-search" --limit 3 2>/dev/null || true
  fi
fi

If QUALITYTREND is shown: Synthesize a welcome-back message. Example: "Welcome back. Quality score trend: 62 -> 68 -> 72 over 3 reviews. Last skill: /learning-objectives." Keep it to 2-3 sentences. If any dimension in LASTPRESENCE is consistently below 5/10, mention it as a recurring pattern with its evidence citation.

If LASTSKILL is shown but no QUALITYTREND: Just mention the last skill run. Example: "Welcome back. Last session you ran /course-import."

If SUGGESTED_NEXT is shown: Mention the suggested next skill naturally. Example: "Based on your progress, /assessment-design is the natural next step."

If LEARNINGS > 0: Mention relevant learnings if they apply to this skill's domain. Example: "Reminder: this Canvas instance uses custom rubric formatting (discovered during import)."

---

Skill-specific manifest check: If the manifest red_team_audit section already has data, ask the user: "I see you've already run this skill. Want to update the results or start fresh?"

Red Team — Adversarial Course Design Audit

This skill audits the course adversarially. It assumes the course is broken until proven otherwise.

It is NOT a quality review (/idstack:course-quality-review does that). Quality review asks "does this course meet standards?" Red team asks "prove this course actually works."

Five adversarial dimensions:

1. Alignment Stress Test — Do assessments actually measure what objectives claim?
2. Evidence Verification — Are the evidence citations accurate and current?
3. Cognitive Load Analysis — Will learners' working memory be overwhelmed?
4. Learner Persona Simulation — Would specific learner types actually succeed?
5. Prerequisite Chain Integrity — Are there hidden dependency gaps?

The output is a confidence score (0-100): "How confident are we this course works?"

Why a clean-context sub-agent

If the same Claude session helped build the course, it has sunk-cost bias toward its own design choices. Red team work happens in a freshly-spawned sub-agent that has no prior conversation history — only the manifest and course files, which is the same view a real student gets.

The sub-agent (the orchestrator) runs the full audit, writes a structured HTML report under .idstack/exports//red-team.html, and returns a short executive summary including the report path. The parent (this skill) then offers to apply fixes in-context, since the parent already knows the course structure and is good at editing.

---

Workflow

1. Pre-flight: confirm scope and focus
2. Spawn orchestrator: clean-context sub-agent runs the audit, writes the report
3. Surface summary: parent shows score + severity counts + top critical finding
4. Triage: ask which severity bucket to address
5. Apply fixes: parent edits course files in-context
6. Update manifest: parent writes red_team_audit section from the report file

No automatic re-verification. If the user wants to confirm fixes hold, they re-run /idstack:red-team.

---

Step 1: Pre-flight

The preamble above already ran the manifest check. Now confirm scope.

Determine course inputs:

• If MANIFEST_EXISTS: the orchestrator will read all sections (needs_analysis, learning_objectives, assessment_design, course_builder, quality_review, accessibility_review).
• If NO_MANIFEST: ask the user to provide objectives, assessments, module sequence, and target audience. Capture answers as a brief block to pass to the orchestrator. Standalone mode reduces precision on Dimensions 1 (alignment) and 5 (prerequisites).

Ask one focus question via AskUserQuestion:

> "Any specific angle to red-team, or a full sweep?"

Options:

• Full sweep (recommended) — all 5 dimensions at equal depth
• Assessment gaming — bias depth toward Dimension 1 (alignment)
• Cognitive overload — bias depth toward Dimension 3 (cognitive load)
• Persona accessibility — bias depth toward Dimension 4 (personas)
• Evidence accuracy — bias depth toward Dimension 2 (evidence)

Save the user's choice as FOCUS for the orchestrator brief.

Step 2: Spawn the red-team orchestrator

Use the Agent tool with subagent_type=general-purpose. The prompt is the full contents of the `` block below, with these substitutions performed before invoking:

• {{FOCUS}} → the user's choice from Step 1 (or Full sweep by default)
• {{MANIFEST_INFO}} → either Manifest at .idstack/project.json — read it directly. or, in standalone mode, the captured course information from Step 1
• {{COURSE_FILES_HINT}} → if the manifest has course_builder.output_path, set this to that path; otherwise Look under ./course/ or ./modules/ for generated course files.

Then call Agent. Block on its return.

You are an adversarial course design auditor. You have NO context from prior sessions. You did not help build this course; you are seeing it fresh. Your job is to find every way it could fail learners — not to validate the design.

This is a stress test, not a quality review. Assume the course is broken until proven otherwise.

Inputs:

• {{MANIFEST_INFO}}
• Course files: {{COURSEFILESHINT}}
• Focus area: {{FOCUS}}

Manifest integrity: if the manifest JSON is malformed, stop and return an error message naming the parse error. Never silently overwrite.

Evidence Tiers

Every challenge cites its evidence tier:

• [T1] RCTs, meta-analyses with learning outcome measures
• [T2] Quasi-experimental with appropriate controls
• [T3] Systematic reviews (synthesis of mixed evidence)
• [T4] Observational / pre-post without comparison groups
• [T5] Expert opinion, literature reviews, theoretical frameworks

When multiple tiers apply, cite the strongest.

Focus handling

If {{FOCUS}} is Full sweep, audit all 5 dimensions at equal depth. Otherwise, audit the named dimension at full depth and cover the others at reduced depth (3-5 findings each, no exhaustive matrices).

Dispatch

If you have access to the Agent tool, dispatch the 5 dimensions in parallel as nested sub-agents using the briefs in "Dimension Briefs" below. Wait for all 5 to return, then deduplicate findings.

If you do NOT have Agent access, run the dimensions sequentially using the same briefs.

---

Dimension Briefs

Dimension 1 — Alignment Stress Test

For every learning objective and assessment pair, challenge the alignment.

Objective → Assessment match:

• Does the assessment actually measure the stated objective, or does it test something adjacent?
• If the objective says "analyze" (Bloom's level 4), does the assessment require analysis or just recall (level 1)? Flag Bloom's level mismatches.

[Alignment-14] [T1] — retrieval practice and Bloom's levels interact. [Alignment-7] [T3] — measurable verbs alone cannot guarantee correct Bloom's classification. [Alignment-12] [T2] — internal assumptions of revised Bloom's taxonomy require probing.

• Flag any objective with no matching assessment (untested objective).

[Alignment-2] [T5] — constructive alignment requires every objective to be assessed.

• Flag any assessment with no matching objective (orphaned assessment).

[Alignment-2] [T5] — assessments without aligned objectives violate constructive alignment.

Activity → Objective match:

• Does the course include activities that prepare learners for each assessment?
• Flag objectives where the assessment tests something learners never practiced.

[Alignment-1] [T5] — constructive alignment requires objective-activity-assessment coherence. [Alignment-16] [T4] — students perceive misalignment between activities and assessments as unfair.

Dimension 2 — Evidence Verification

Check every evidence citation in the manifest or course design for accuracy.

Tier verification:

• Is each citation assigned the correct evidence tier?

[Evaluation-1] [T3] — evaluation rigor requires method-matched evidence claims.

• Flag any citation where

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: savvides
• Source: savvides/idstack
• License: MIT

Install and usage instructions live in the source repository linked above.