Exercise Gen

Exercise Generator

Produce scaffolded Jupyter notebooks that teach a codebase through hands-on exercises. Each notebook is self-contained, runnable on its own, and progresses from guided usage through independent creation.

This skill is manifest-driven. The manifest at learn/internals/exercise-plan.md is the source of truth for which exercises exist, where they slot into curriculum.md, and how each is rendered. Read the Shared Contracts in repo-learner/SKILL.md before starting.

Read references/exercise-types.md for the exercise taxonomy (use / modify / debug / create / compare) and when to use each. Read references/subagent-brief-template.md before fanning out to subagents.

Notebook-first, with narrow inline exceptions

Default emission is .ipynb. Two narrow exceptions allowed:

1. compare exercises — table or side-by-side prose comparisons fit

naturally inline.

1. Short copy-paste-run blocks — fewer than ~15 lines, no scaffold, no

validation, no solution. Pure "run this and observe."

Everything else is a notebook. The long-term direction is "everything becomes a notebook" — treat inline as an exception you should be reluctant to use, not a co-equal option.

Pipeline

Stages 1–2 produce the manifest (if it doesn't already exist). Stages 3–5 consume it.

Stage 1: Candidate Mining

Scan the codebase to identify "exercisable surfaces" — places where a learner can do something concrete that builds understanding.

What to scan for:

1. Public API functions with clear I/O. Natural "use" exercises.
2. Configuration and setup patterns. "Set up and configure" exercises.
3. Extension points. Subclass / register / implement — the best

exercises, because they force understanding of the abstraction.

1. Workflows visible in examples/tests. The authors already wrote these

— extract the pattern, strip to skeleton.

1. Error-handling paths and common mistakes. "Debug" exercises.
2. Comparison points. Two ways to do the same thing → "compare"

exercises.

Output: learn/internals/exercise-candidates.md. Per candidate:

### Candidate: [short name]
- **Surface:** [which file/function/class]
- **Type:** use | modify | create | debug | compare
- **Exercises:** [what the learner would do — 1-2 sentences]
- **Concepts tested:** [what they need to understand]
- **Dependencies:** [prerequisite candidates or curriculum sections]
- **Difficulty:** beginner | intermediate | advanced
- **Confidence:** high | medium | low

Aim for 12–18 candidates. Pre-filter during mining — skip candidates that obviously don't teach anything new. Cast a focused net, not a wide one.

Stage 2: Selection, Ordering, and Manifest Emission

Select 8–12 exercises from the candidates and sequence them.

Selection criteria (priority order):

1. Forces active understanding. The learner must think, not just copy.
2. Builds on previous exercises. Cumulative understanding.
3. Covers the important surfaces. Span the library's key capabilities.
4. Mix of exercise types. Progression ramps naturally (use → modify →

debug → create → optional compare).

1. Each exercise earns its place. If you can't articulate what NEW

understanding it produces beyond the previous ones, cut it.

Ordering rules:

• Prerequisites before dependents.
• Simpler before complex.
• Core API before extensions.
• Happy path before error handling.
• Align with curriculum sections: each exercise maps to one or more

sections. Skip curriculum sections that don't admit hands-on exercises.

Output: the manifest at learn/internals/exercise-plan.md. See the Shared Contracts in repo-learner/SKILL.md for the schema. Each exercise entry is a markdown section with a yaml code fence at the top.

If repo-analyzer already wrote a draft manifest with status: planned entries, treat that as ground truth — only add/modify entries if the analyzer's plan is genuinely incomplete. Do not silently replace the analyzer's choices.

Stage 2.5: Emission Decision

For each manifest entry, set emission: notebook (default) or emission: inline. Inline only when:

• type: compare AND the comparison fits in a markdown table or side-by-

side code blocks.

• type: use AND the exercise is a short copy-paste-run block (no scaffold,

no validation, no solution).

Otherwise: notebook. If unsure, choose notebook.

Stage 3: Notebook Scaffolding (or inline block design)

For emission: notebook

Design the cell-by-cell structure. This is the blueprint for Stage 4.

Cell 1 [markdown]: Title + curriculum back-link
  - Exercise number, name, type, estimated time
  - One-line goal
  - Back-link: "Context: you just read §X.Y in [curriculum.md](../curriculum.md#sX.Y)"
  - Prerequisites (link to earlier notebooks if any)

Cell 2 [code]: Setup
  - Imports
  - Data loading / fixture creation
  - Helper functions the learner needs but shouldn't have to write
  - Runs without error as-is

Cell 3+ [markdown + code]: Guided walkthrough (2-4 cells)
  - Alternate markdown (explanation) and code (working example)
  - Code cells are COMPLETE and RUNNABLE — not scaffolded
  - Include print()/display() so output is visible
  - Don't re-explain concepts from curriculum.md — link back instead

Cell N [markdown]: "Your turn"
  - Specific, unambiguous task
  - Expected output format described
  - Optional: hint dropdowns inside  tags

Cell N+1 [code]: Scaffold
  - Function signature with docstring
  - TODO comments marking where to fill in (WHAT, not HOW)
  - Type hints on parameters and return
  - Parses without syntax errors

Cell N+2 [code]: Validation
  - Assertions checking the learner's code works
  - Clear error messages: "Expected X, got Y"
  - At least 3 cases: basic, edge, non-trivial
  - Print success message when all pass

Cell N+3 [markdown]: Stretch goal (optional)
  - Harder extension, no scaffold

Cell N+4 [code]: Solution
  - Runnable code cell. scaffold_notebook.py sets source_hidden so it
    starts collapsed in JupyterLab. Other viewers show it expanded.

Rules for scaffold cells:

• Stub functions return None or raise NotImplementedError.
• Include type hints — they're documentation.
• TODO comments say WHAT, not HOW: # TODO: Filter items where score > threshold

not # TODO: Use list comprehension with if clause.

• Don't scaffold too much (no structural decisions left) or too little

(the learner can't tell what function to write).

Rules for validation cells:

• 3+ cases: basic, edge, non-trivial.
• Assertions have descriptive messages.
• Always print("All tests passed.") at the end.
• Visual check for any plots.

For emission: inline

Write the markdown block that will replace the curriculum placeholder. For compare: a table or side-by-side prose with code samples. For short use: a fenced code block + one expected-output line. Keep under ~20 lines of markdown total.

Stage 4: Generation, Validation, Insertion

This stage is gated. A notebook is not "done" until each gate passes.

Stage 4a: Generation

For each notebook exercise: construct an ExerciseSpec and call scripts/scaffold_notebook.py. Save to learn/notebooks/exercise-NN-title.ipynb (slug-stripped filename — no parens, no special chars).

For each inline exercise: prepare the markdown block as a string. Do NOT insert it yet.

After generation, update the manifest: status: scaffolded.

Subagent fan-out (when notebook count > 4): One subagent per notebook. Each gets references/subagent-brief-template.md + its single exercise spec. Main agent runs a stitcher pass after: dedupe setup code, verify cross-references, lint filenames. Below the threshold, generate sequentially.

Stage 4b: Env files

Before validation can run, the notebooks need installable env files.

• Read learn/internals/.config.json for user.env_manager (and

repo.language). Never mention any other manager in user-facing files.

• Call scaffold_notebook.py's `generateenvfiles(specs, output_dir,

language=...). For Python: writes requirements.txt + pyproject.toml. Adding a language extends LANGUAGE_PROFILES` and the corresponding generator branch.

Stage 4c: Env install (MANDATORY)

Run the user's chosen install command before any validation. The previous pipeline implicitly required this but never executed it — the result was nbconvert validation either skipping silently or failing on missing deps.

1. Look up the install command:

LANGUAGE_PROFILES[repo.language].env_managers[user.env_manager].

1. Run it via the Bash tool in learn/notebooks/. Use a generous timeout

(60–300s for a fresh project). Stream output so the user sees progress.

1. If it fails, diagnose:

• Network issue → retry once, then surface to the user.
• Missing system dep (e.g. compiler, system library) → tell the user

specifically what's missing, with the platform-specific install command.

• Version conflict → revisit dependencies in the manifest; loosen

pins or substitute. Don't silently skip — Stage 4d needs the env to exist.

1. Register the notebook env as a named Jupyter kernel. Without this

step, editors (VS Code in particular) have no idea which Python to use when the user opens a notebook, and they get dumped into a kernel picker with no obvious right answer. The notebook's kernelspec.name is set by scaffold_notebook.py to learn-; this step actually registers that name against the venv.

From learn/notebooks/, run (Python + uv example): `` uv run python -m ipykernel install --user --name=learn- --display-name="Learn " ` For pip: python -m ipykernel install --user --name=... after source .venv/bin/activate. For other env managers, the equivalent run python -m ipykernel ...` form.

Confirm by running jupyter kernelspec list — the new kernel name should appear.

Stage 4d: Validation (MANDATORY — two channels)

scaffold_notebook.py emits a learn/internals/validation/exercise-NN.validation.json stub per notebook. Both channels must fill it in:

Channel 1: Mock-student validation (pedagogical check). Spawn a fast-tier model as a mock-student subagent per notebook with this brief. Use the cheapest capable model available — on Claude Code, Haiku; on other agents, the equivalent fast tier. The point is fresh eyes, not raw capability.

You are a student learning a new codebase. You have access to ONLY the
information in this notebook — no prior knowledge of the library. Work
through it:

1. Run the setup cell. Report import errors or missing dependencies.
2. Read and run each guided cell. Note anything confusing.
3. Read "Your turn." Attempt using ONLY what the walkthrough taught.
   Do NOT look at the solution.
4. Run the validation cell against your attempt.
5. If stuck, explain exactly where and why.

Output JSON:
{
  "exercise": "",
  "setup_runs": true/false,
  "guided_cells_run": true/false,
  "could_complete_without_solution": true/false,
  "validation_passed": true/false,
  "stuck_points": [...],
  "ambiguities": [...],
  "missing_info": [...],
  "solution_matches": true/false
}

Fix-up rules:

• setup_runs: false → fix imports/deps. Hard blocker.
• could_complete_without_solution: false → walkthrough gap or ambiguity.
• validation_passed: false (mock student's approach was reasonable) →

loosen assertions.

• stuck_points → fix missing info and ambiguity; leave genuine difficulty.
• solution_matches: false → solution is wrong; fix it.

When the mock student passes, set validator_passed: true in validation_report.json (and attach the JSON report under validator_report).

Apply fixes silently. Only surface to the user if an exercise had to be cut entirely.

Channel 2: nbconvert execute (environment check). This channel verifies the environment, not solution correctness — that's Channel 1's job. In the venv installed by Stage 4c, run:

jupyter nbconvert --to notebook --execute --allow-errors --output exercise-NN.executed learn/notebooks/exercise-NN-*.ipynb

--allow-errors is required. The scaffold cell intentionally raises NotImplementedError — it's a placeholder for the learner. The validation cell will also fail because the scaffold isn't filled in. Those failures are expected and must NOT fail this check.

Then classify cell outcomes with the deterministic helper:

python /scripts/check_executed_notebook.py \
    learn/notebooks/exercise-NN.executed.ipynb \
    learn/internals/validation/exercise-NN.nbconvert.json

The helper reads each cell's role:* metadata tag (set by scaffold_notebook.py at generation time) and emits a per-cell verdict. Exit code 0 means env-healthy; nonzero means at least one expected-success cell errored. Drop the JSON it produces straight into validation_report.json under nbconvert_log.

If env-healthy: set nbconvert_passed: true in validation_report.json.

If env-unhealthy (setup or guided-code cell errored): the env is broken. Fix the underlying issue (missing dep, wrong wheel for the platform, OS-specific code path) and re-run. Don't paper over it. Solution correctness is Channel 1's job, not this one.

When both channels write to validation_report.json and pass, update the manifest: status: validated.

The orchestrator's reconciliation pass will refuse to finish if any validation_report.json is empty or marks failure. Do not skip this stage.

Stage 4e: Insertion

For each entry with status: validated, edit learn/curriculum.md:

1. Find the line containing `` (where SLOT is

the manifest's slot field).

1. The next line is the placeholder: - [ ] Exercise NN — pending or

similar.

1. Replace ONLY that one line with the real content:

• For emission: notebook:

```markdown

• [ ] [Exercise NN — title](notebooks/exercise-NN-title.ipynb)

```

• For emission: inline: replace the one placeholder line with the

inline markdown block (which may be multiple lines).

Update the manifest: status: inserted.

You MUST NOT touch any other line. The marker is invariant: leave it in place above the inserted content.

Stage 5: Notebook README

After all entries are inserted, generate learn/notebooks/README.md: setup instructions leading with the detected env_manager (from .config.json), plus the Jupyter kernel registration hint appropriate to the language (for Python: python -m ipykernel install ...).

The env files (requirements.txt + pyproject.toml for Python; other files when additional languages are added) were generated in Stage 4b; this stage only writes the README that points at them.

Light mode

If .config.json:tuning.depth == "light", three things change. Comprehensive mode (the default) is unaffected.

• Candidates (Stage 1): mine 5–8 candidates instead of 12–18. Be more

ruthless about pre-filtering.

• Selection (Stage 2): select 3–5 exercises instead of 8–12. Skew toward

high-value use and modify exercises tied to Phase 1; drop compare and most architectural candidates.

• Validation (Stage 4d): skip Channel 1 (mock-student validation).

Keep Channel 2 (nbconvert execute) — it's cheap and catches the real env bugs. Still write the validation_report.json with validator_passed: true and a validator_skipped_reason: "light mode" field so reconciliation passes.

Stage 4c (env install) still runs in Light mode — without it, the nbconvert check in Stage 4d cannot run.

Subagent fan-out, marker discipline, manifest insertion, and reconciliation are identical to comprehensive mode.

Test mode

If .config.json:tuning.mode == "test", run a smoke-test variant. The output directory is learn_test/; all paths shift accordingly. Light mode's behavior applies on top (depth is forced to light in test mode).

• Stages 1–2 (Mining + Selection): skipped entirely. repo-an

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: PranitMohnot
• Source: PranitMohnot/repo-learner-suite
• License: MIT

Install and usage instructions live in the source repository linked above.