Research Experiment

Experiment Execution and Recording

Guide the design, execution tracking, and result recording of experiments that test research hypotheses.

Context

Each iteration in the methodology tests one hypothesis through an experiment. The experiment must produce a clear signal: the hypothesis is confirmed, refuted, or the result is inconclusive. The emphasis is on minimality — the goal is to get an answer, not to build a product.

Experiment Design Principles

1. Minimality: create the smallest prototype or analysis that can

confirm or refute the hypothesis. Resist scope creep.

1. Reproducibility: key experiments (those that feed into the final

report) must be reproducible by other engineers without the author's involvement. This means: documented environment, launch instructions, and setup scripts.

1. Timebox awareness: the experiment runs within the hypothesis timebox.

If the timebox is about to expire and the result is unclear, that itself is a result — do not extend without a deliberate decision.

Procedure

Step 1 — Read the hypothesis card

Locate and read the hypothesis card (hypotheses/H-NNN.md). Extract:

• The statement being tested.
• The verification criterion (what counts as confirmation).
• The timebox (how long is allocated).
• The current status (must be open or in-progress).

If the status is not open or in-progress, stop and inform the researcher that this hypothesis is not in an experimentable state.

Step 2 — Plan the experiment

Help the researcher design a minimal experiment:

1. Approach: what will be built, analyzed, or measured? Suggest the

simplest viable approach.

1. Environment: what OS, language versions, tools, and dependencies are

needed? Capture these now — they form the reproducibility requirements.

1. Expected output: what artifact will demonstrate the result (PoC,

benchmark data, log output, counterexample)?

1. Completion signals: specifically, what outcome confirms the hypothesis

and what outcome refutes it? Map the verification criterion to concrete observables.

If the verification criterion in the card is ambiguous, propose a more precise operationalization and update the card with the researcher's approval.

Step 3 — Set up the environment

Assist with environment setup:

• Suggest Dockerfile, Makefile, shell scripts, or other setup automation

as appropriate for the experiment's complexity.

• For trivial experiments (single script, no unusual dependencies), a note

in the card's Experiment Notes section is sufficient.

• For complex setups, recommend creating a dedicated directory in the

project's prototype/tool area and versioning it.

Document the environment in the hypothesis card's Experiment Notes section.

Step 4 — Track progress

During the experiment, help the researcher track progress:

• Periodically check timebox status. If more than 75% of the timebox has

elapsed without a clear signal, alert the researcher and suggest:

• Simplifying the experiment scope.
• Identifying what is blocking progress.
• Considering whether the timebox should trigger a kill or pivot

decision.

• Record intermediate findings in the Experiment Notes section of the

hypothesis card. These notes are free-form but should capture key observations, dead ends, and unexpected discoveries.

Step 5 — Record the result

When the experiment concludes (by result or by timebox expiry):

1. Determine the outcome:

• Confirmed — the verification criterion is met.
• Refuted — the verification criterion is demonstrably not met.
• Inconclusive — neither confirmed nor refuted within the timebox.

Document what would be needed for a definitive result.

1. Update the hypothesis card via the research-hypothesis skill:

• Set Status to the appropriate value.
• Fill the Result section with: findings, link to prototype/proof,

artifact list.

• Set the Completed date.

1. Document reproducibility (for experiments that will enter the final

report):

• Environment description (OS, language versions, dependencies).
• Launch instructions (step-by-step).
• Setup scripts or configuration files.

Step 6 — Hand off to decision

After recording the result, inform the researcher that the iteration is ready for the decision step. Suggest invoking the research-decision skill to determine the next action (continue / pivot / kill / fork).

Reproducibility Checklist

For experiments that are candidates for the final report, verify:

• [ ] Environment versions documented (OS, runtime, key libraries).
• [ ] Dependencies listed (package file, requirements.txt, go.mod, etc.).
• [ ] Launch instructions written and tested.
• [ ] Setup automation exists if the environment is non-trivial.
• [ ] Output artifacts are saved and linked from the hypothesis card.

Intermediate experiments that were discarded do not need to meet this bar.

Relation to Other Skills

• Follows research-hypothesis (a hypothesis card must exist before an

experiment can begin).

• Results feed into research-decision for next-action determination.
• Reproducibility artifacts are inventoried by research-synthesis.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: v0lka
• Source: v0lka/skills
• License: MIT

Install and usage instructions live in the source repository linked above.