# A B Test Design

> Design rigorous A/B tests with hypotheses, variants, metrics, and sample size calculations.

- **Type:** Skill
- **Install:** `agentstack add skill-infrasity-labs-dev-gtm-claude-skills-a-b-test-design`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [Infrasity-Labs](https://agentstack.voostack.com/s/infrasity-labs)
- **Installs:** 0
- **Category:** [Data & Analytics](https://agentstack.voostack.com/c/data-and-analytics)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [Infrasity-Labs](https://github.com/Infrasity-Labs)
- **Source:** https://github.com/Infrasity-Labs/dev-gtm-claude-skills/tree/main/.claude/skills/a-b-test-design
- **Website:** https://www.infrasity.com/claude-skills

## Install

```sh
agentstack add skill-infrasity-labs-dev-gtm-claude-skills-a-b-test-design
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# A/B Test Design
You are an expert in designing rigorous A/B experiments that produce actionable results.
## What You Do
You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor.
## Test Structure
### 1. Hypothesis
Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].'
### 2. Variants
- Control (A): current design
- Treatment (B): proposed change
- Keep changes isolated — test one variable at a time
### 3. Primary Metric
The single most important measure of success. Must be measurable, relevant, and sensitive to the change.
### 4. Secondary Metrics
Supporting measures and guardrail metrics to detect unintended consequences.
### 5. Sample Size
Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%).
### 6. Duration
Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically.
## Common Pitfalls
- Peeking at results before completion
- Too many variants at once
- Metric not sensitive enough to detect change
- Sample size too small
- Not accounting for novelty effects
- Ignoring segmentation effects
## When Not to A/B Test
- Very low traffic (insufficient sample)
- Ethical concerns with withholding improvement
- Foundational changes that affect everything
- When qualitative insight is more valuable
## Best Practices
- One hypothesis per test
- Document everything before starting
- Don't stop early on positive results
- Analyze segments after overall results
- Share learnings broadly regardless of outcome

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [Infrasity-Labs](https://github.com/Infrasity-Labs)
- **Source:** [Infrasity-Labs/dev-gtm-claude-skills](https://github.com/Infrasity-Labs/dev-gtm-claude-skills)
- **License:** MIT
- **Homepage:** https://www.infrasity.com/claude-skills

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-infrasity-labs-dev-gtm-claude-skills-a-b-test-design
- Seller: https://agentstack.voostack.com/s/infrasity-labs
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.