Experiment Planner
Use when turning an ML hypothesis, paper claim, or model idea into a reproducible experiment plan with baselines, metrics, seeds, risks, and expected artifacts.
Install
$ agentstack add skill-maximussthegreat-ml-researcher-os-experiment-planner ✓ scanned · ✓ verified, works with Claude Code, Cursor, and more.
Security review
✓ PassedNo issues found. Passed automated security review. · v0.1.0 How review works →
- ✓ Prompt-injection patterns
- ✓ Secret / credential exfiltration
- ✓ Dangerous shell & filesystem operations
- ✓ Untrusted network calls
- ✓ Known-malicious package signatures
What it can access
- ✓ Network access No
- ✓ Filesystem access No
- ✓ Shell / process execution No
- ✓ Environment & secrets No
- ✓ Dynamic code execution No
From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.
Verified badge
Passed review? Show it. Paste this badge into your README, it links to the public security report.
Links to your public security report.[](https://agentstack.voostack.com/security/report/skill-maximussthegreat-ml-researcher-os-experiment-planner) Reliability & compatibility
Declared compatibility
Compatibility is declared by the source manifest. End-to-end runtime verification is coming, see below.
We're building live execution health for every listing: tool-call success rate, median latency, uptime, and last-checked timestamps, measured, not self-reported. It isn't live yet, so we don't show numbers we can't stand behind.
How agent discovery & health will work →About
Experiment Planner
Use this skill after claim extraction and before writing training code.
Goal
Convert a research claim into the smallest controlled experiment that can produce useful evidence.
Required inputs
- Hypothesis or paper claim
- Dataset or proposed dataset
- Target metric
- Compute budget
- Existing baseline, if any
If any input is missing, ask for it or mark it as unknown and make a conservative plan.
Required output
Write an experiment plan with:
- Hypothesis
- Minimum viable experiment
- Baselines
- Ablations
- Data split strategy
- Metrics
- Seed policy
- Failure modes
- Logging and artifacts
- Stop criteria
Baseline rules
Every plan needs at least one baseline. Prefer:
- simplest non-neural baseline
- standard library baseline
- previously reported baseline from the source paper
- ablation of the proposed method
Failure modes to check
- data leakage
- train/validation split mismatch
- metric mismatch
- hidden preprocessing fit on validation or test data
- seed sensitivity
- tiny test set
- weak baseline
- cherry-picked run
- missing negative result
Rules
- Do not plan an experiment without at least one meaningful baseline.
- Do not compare a heavily tuned model against an untuned baseline.
- Do not claim reproduction when the dataset, split, or metric differs from the source claim.
- Prefer three small controlled runs over one expensive ambiguous run.
Output style
Be concrete. Prefer a small experiment that can run today over an impressive plan that cannot be verified.
Source & license
This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.
- Author: maximussthegreat
- Source: maximussthegreat/ml-researcher-os
- License: MIT
Install and usage instructions live in the source repository linked above.
Reviews
No reviews yet, be the first.
Write a review
Versions
- v0.1.0 Imported from the upstream source.