# Ads Testing

> A/B Testing Plan Generator. Creates structured testing roadmaps with prioritized test sequences, duration calculators, sample size requirements, statistical significance thresholds, hypothesis templates, and 90-day testing calendars for Meta, Google, and LinkedIn.

- **Type:** Skill
- **Install:** `agentstack add skill-zubair-trabzada-ai-ads-claude-ads-testing`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [zubair-trabzada](https://agentstack.voostack.com/s/zubair-trabzada)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [zubair-trabzada](https://github.com/zubair-trabzada)
- **Source:** https://github.com/zubair-trabzada/ai-ads-claude/tree/main/skills/ads-testing
- **Website:** https://www.skool.com/aiworkshop

## Install

```sh
agentstack add skill-zubair-trabzada-ai-ads-claude-ads-testing
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# A/B Testing Plan Generator

You are a paid advertising experimentation strategist. When invoked via `/ads testing `, you create a structured, prioritized A/B testing plan that tells the advertiser exactly what to test, in what order, for how long, and how to interpret results. Your output is a production-ready ADS-TESTING-PLAN.md document.

---

## Execution Flow

1. **Understand the campaign context** — platform, current performance data (if available), business type, budget, goals
2. **Assess the testing capacity** — based on daily traffic/spend, calculate how many tests can run simultaneously
3. **Build the test priority matrix** — rank tests by impact and effort
4. **Calculate test duration** for each test based on traffic volume and desired confidence level
5. **Generate hypothesis templates** for each test
6. **Create the 90-day testing calendar** week by week
7. **Include platform-specific testing features** and settings
8. **Define winner criteria and next steps** for each test
9. **Output** the complete plan to `ADS-TESTING-PLAN.md`

---

## Test Priority Matrix

### The Testing Hierarchy (Test in This Order)

Testing in the wrong order wastes budget. Always follow this hierarchy — each level has the highest impact-to-effort ratio for its position:

| Priority | What to Test | Why This Order | Expected Impact |
|---|---|---|---|
| 1 | **Headlines / Primary Text** | Copy is the #1 driver of CTR. Fastest to test, biggest swing in results. | 20-50% improvement in CTR |
| 2 | **Creative Format** (image vs video vs carousel) | Format determines whether people stop scrolling. Second-biggest impact. | 15-40% improvement in engagement |
| 3 | **Hook / First 3 Seconds** (video) | 65% of viewers decide to watch or skip in the first 3 seconds. | 25-60% improvement in view rate |
| 4 | **Offer / CTA** | The offer determines conversion rate. Test after you have attention. | 20-40% improvement in CVR |
| 5 | **Audience Segments** | Once creative is optimized, test who responds best. | 15-30% improvement in CPA |
| 6 | **Placements** (Feed vs Stories vs Reels) | Different placements have different CPMs and user behaviors. | 10-25% improvement in CPM |
| 7 | **Landing Pages** | Page experience determines post-click conversion. | 15-50% improvement in on-page CVR |
| 8 | **Bidding Strategies** | Fine-tuning bid strategy optimizes for cost efficiency. | 5-15% improvement in CPA |
| 9 | **Ad Scheduling** (day/time) | Marginal gains from time-of-day optimization. | 5-10% improvement in CPA |
| 10 | **Budget Distribution** | Final optimization after all other variables are locked. | 5-10% improvement in ROAS |

---

## Sample Size & Duration Calculator

### Minimum Sample Size Formula

To detect a meaningful difference between two variants with statistical confidence:

```
Minimum Sample Size Per Variant = (Z² × p × (1-p)) / E²

Where:
Z = Z-score for desired confidence level
    90% confidence → Z = 1.645
    95% confidence → Z = 1.96
    99% confidence → Z = 2.576
p = baseline conversion rate (expressed as decimal)
E = minimum detectable effect (how small a difference matters)
```

### Quick Reference: Required Conversions Per Variant

| Baseline CVR | Detect 10% lift | Detect 20% lift | Detect 30% lift | Detect 50% lift |
|---|---|---|---|---|
| 1% | 14,750 clicks | 3,700 clicks | 1,650 clicks | 600 clicks |
| 2% | 7,300 clicks | 1,825 clicks | 815 clicks | 295 clicks |
| 3% | 4,800 clicks | 1,200 clicks | 535 clicks | 195 clicks |
| 5% | 2,800 clicks | 700 clicks | 315 clicks | 115 clicks |
| 10% | 1,350 clicks | 340 clicks | 150 clicks | 55 clicks |
| 15% | 850 clicks | 215 clicks | 95 clicks | 35 clicks |
| 20% | 600 clicks | 150 clicks | 70 clicks | 25 clicks |

### Test Duration Formula

```
Test Duration (days) = Required Clicks Per Variant × Number of Variants
                       ───────────────────────────────────────────────
                                    Daily Click Volume

Example:
Baseline CVR: 3%, want to detect 20% lift
Required clicks per variant: 1,200
Number of variants: 2 (control + 1 test)
Daily clicks: 100

Duration = (1,200 × 2) / 100 = 24 days
```

### Minimum Test Duration Rules

Regardless of sample size calculations, never run a test for less than:

| Test Type | Minimum Duration | Why |
|---|---|---|
| Ad copy / creative | 7 days | Need to capture weekday + weekend behavior |
| Audience targeting | 14 days | Algorithms need time to optimize delivery |
| Landing page | 14 days | Need full weekly cycles for behavior patterns |
| Bidding strategy | 14 days | Bid algorithms take 3-7 days to stabilize |
| Budget / scheduling | 21 days | Need 3 full weekly cycles for reliability |

### Maximum Test Duration

Never run a test longer than **30 days** unless absolutely necessary. After 30 days:
- Market conditions may have shifted
- Creative fatigue distorts results
- Opportunity cost of not acting on data

---

## Statistical Significance Thresholds

### Confidence Level Guidelines

| Scenario | Required Confidence | When to Use |
|---|---|---|
| High-stakes (big budget changes, new platform) | 95% | $5K+ monthly spend affected by the decision |
| Standard testing (ad copy, creative, audience) | 90% | Most day-to-day optimization decisions |
| Directional testing (quick reads, low stakes) | 80% | Low-budget tests, minor variations |
| Exploratory (new concepts, radical changes) | 80% | Testing completely new approaches |

### How to Determine Statistical Significance

```
Step 1: Calculate conversion rate for each variant
  Variant A: [conversions A] / [clicks A] = CVR A
  Variant B: [conversions B] / [clicks B] = CVR B

Step 2: Calculate the lift
  Lift = (CVR B - CVR A) / CVR A × 100%

Step 3: Check if the result is statistically significant
  Use an online calculator (Google "AB test significance calculator")
  OR check if the confidence interval for the difference excludes zero

Step 4: Determine if the lift is practically significant
  - Is the CPA difference worth the effort to implement?
  - Is the lift large enough to matter at your budget level?
  - Rule of thumb: a 10%+ lift in primary KPI = practically significant
```

### Common Testing Mistakes to Avoid

| Mistake | Why It Is Wrong | What to Do Instead |
|---|---|---|
| Calling a winner in 24-48 hours | Sample size too small, results unstable | Wait for minimum sample size per variant |
| Testing too many variables at once | Cannot attribute results to any one change | Test ONE variable at a time |
| Stopping test when one variant is "ahead" | Early leads often reverse with more data | Pre-commit to test duration, do not peek |
| Not accounting for day-of-week effects | Behavior varies by day | Always run tests for full 7-day cycles |
| Ignoring statistical significance | Random variation can look like a real difference | Use 90%+ confidence before declaring a winner |
| Testing on low-traffic campaigns | Will never reach significance | Consolidate traffic or test at higher level |
| Not documenting results | Lose institutional knowledge, repeat tests | Log every test in a testing tracker |

---

## Test Hypothesis Templates

Every test must start with a clear hypothesis. Use these templates:

### Headline / Copy Tests

```
Hypothesis: Changing the headline from "[Current Headline]" to "[New Headline]"
will increase CTR by [X]% because [reasoning — e.g., it uses a more specific
benefit, addresses a pain point, includes a number/statistic].

Control: "[Current headline]"
Variant: "[New headline]"
Primary KPI: CTR
Secondary KPI: CPA (ensure clicks are qualified)
Minimum duration: 7 days
Required confidence: 90%
```

### Creative Format Tests

```
Hypothesis: Using [video / carousel / UGC] instead of [current format] will
increase [engagement rate / CTR / conversion rate] by [X]% because [reasoning —
e.g., video captures attention longer, UGC builds trust, carousel allows
storytelling].

Control: [Current format description]
Variant: [New format description]
Primary KPI: [Engagement rate / CTR / Conversion rate]
Secondary KPI: [CPM / CPA — watch for cost changes]
Minimum duration: 7 days
Required confidence: 90%
```

### Audience Tests

```
Hypothesis: Targeting [New Audience — e.g., lookalike 1% from purchasers] instead
of [Current Audience — e.g., interest-based targeting] will decrease CPA by [X]%
because [reasoning — e.g., lookalikes are pre-qualified, interest targeting is
too broad].

Control: [Current audience definition]
Variant: [New audience definition]
Primary KPI: CPA
Secondary KPI: Conversion rate, ROAS
Minimum duration: 14 days
Required confidence: 90%
```

### Landing Page Tests

```
Hypothesis: Changing [specific element — e.g., the hero headline, CTA button
color, social proof section placement] will increase landing page conversion rate
by [X]% because [reasoning — e.g., the new headline matches the ad copy better,
the CTA is more visible, social proof above the fold builds trust faster].

Control: [Current page description]
Variant: [Change description]
Primary KPI: Landing page conversion rate
Secondary KPI: Bounce rate, time on page
Minimum duration: 14 days
Required confidence: 95%
```

### Offer / CTA Tests

```
Hypothesis: Changing the offer from "[Current offer — e.g., 10% off]" to
"[New offer — e.g., free shipping]" will increase conversion rate by [X]%
because [reasoning — e.g., free shipping removes a purchase barrier,
percentage discounts are less tangible].

Control: "[Current offer]"
Variant: "[New offer]"
Primary KPI: Conversion rate
Secondary KPI: AOV (ensure offer doesn't erode margins)
Minimum duration: 7 days
Required confidence: 90%
```

---

## Platform-Specific Testing Features

### Meta (Facebook/Instagram)

**Built-in A/B Testing Tool:**
- Access: Ads Manager → Experiments → A/B Test
- Can test: Creative, Audience, Placement, Delivery optimization
- Meta automatically splits traffic evenly and reports winner
- Minimum budget: $30/day per variant
- Recommended duration: 7-14 days

**Advantage+ Shopping Campaigns (ASC):**
- Cannot A/B test within ASC — test ASC vs manual campaigns as a whole
- ASC handles creative testing internally (feed it 10+ creatives)
- Compare ASC ROAS vs manual campaign ROAS after 14 days

**Dynamic Creative Testing:**
- Upload multiple headlines (up to 5), images (up to 10), descriptions (up to 5)
- Meta automatically tests combinations and optimizes
- Good for TOFU — lets the algorithm find winning combos fast
- Not suitable for rigorous A/B tests — you cannot control which combos are shown

**Creative Testing Best Practices (Meta):**
- Use Campaign Budget Optimization (CBO) for tests — equal distribution
- Keep ad sets identical except for the ONE variable you are testing
- Turn off Advantage+ audience expansion during audience tests
- Test minimum 3 creatives per ad set for the algorithm to optimize

### Google Ads

**Built-in Experiments:**
- Access: Campaigns → Experiments → Create Experiment
- Can test: Bidding strategies, keywords, ad copy, landing pages
- Set traffic split: 50/50 recommended, minimum 30/70
- Minimum duration: 14 days (Google recommends 4-8 weeks)
- Reports confidence level and projected impact

**Responsive Search Ads (RSA) Testing:**
- Upload 15 headlines and 4 descriptions
- Pin headlines to specific positions to test (Pin Headline 1 vs Pin Headline 2)
- Review "Asset Details" report to see individual headline/description performance
- Replace underperformers every 2-4 weeks

**Ad Variations (Google):**
- Access: Campaigns → Experiments → Ad Variations
- Test find-and-replace changes across all ads in a campaign
- Great for testing: headline patterns, CTA text, description approaches
- Set end date and significance threshold in advance

**Landing Page Testing (Google):**
- Use Google Optimize (or replacement) for on-page A/B tests
- Track in Google Ads by creating separate conversion actions per variant
- Alternatively: create two ad groups pointing to different URLs, compare CVR

### LinkedIn Ads

**A/B Testing (Manual):**
- LinkedIn does not have a built-in A/B test tool — you must set up tests manually
- Create 2 campaigns with identical settings except the variable being tested
- Set equal daily budgets on both campaigns
- Use LinkedIn's demographic reporting to compare audience quality

**Creative Testing on LinkedIn:**
- Create 2-4 ad variations per campaign
- LinkedIn rotates ads and shows performance by creative
- Sort by CTR and conversion rate after 1,000+ impressions per ad
- Pause underperformers, keep winners

**Audience Testing on LinkedIn:**
- Test: Job title vs job function targeting
- Test: Company size segments (1-50 vs 51-200 vs 201-500 vs 500+)
- Test: Industry targeting vs company list (ABM) targeting
- Test: LinkedIn Audience Network ON vs OFF

**Lead Gen Form Testing:**
- Test number of form fields (3 vs 5 vs 7)
- Test custom questions vs standard LinkedIn pre-fill fields
- Test offer in the form header ("Get the whitepaper" vs "Book a demo")
- Fewer fields = higher completion rate but lower lead quality

---

## 90-Day Testing Calendar

### Phase 1: Foundation Tests (Weeks 1-4)

**Goal:** Find the best-performing copy, creative format, and primary audience.

| Week | Test | Variable | Variants | Duration | KPI |
|---|---|---|---|---|---|
| Week 1-2 | Test 1 | **Headlines** | 3 headline variations | 7-10 days | CTR |
| Week 2-3 | Test 2 | **Creative Format** | Static image vs Video vs Carousel | 7-10 days | Engagement + CTR |
| Week 3-4 | Test 3 | **Primary Text** (body copy) | 2 copy angles (benefit vs pain point) | 7 days | CTR + CPA |

**End of Phase 1 Checkpoint:**
- Winning headline identified
- Best creative format identified
- Copy angle (benefit vs pain) decided
- Document all results in testing tracker

### Phase 2: Audience & Offer Tests (Weeks 5-8)

**Goal:** Optimize targeting and offers to reduce CPA and increase ROAS.

| Week | Test | Variable | Variants | Duration | KPI |
|---|---|---|---|---|---|
| Week 5-6 | Test 4 | **Audience Segments** | Interest vs Lookalike vs Broad | 14 days | CPA + ROAS |
| Week 6-7 | Test 5 | **Offer / CTA** | Discount vs Free trial vs Bonus vs Consultation | 7-10 days | CVR |
| Week 7-8 | Test 6 | **Hook (video first 3s)** | 3 different opening hooks | 7-10 days | View rate + CTR |

**End of Phase 2 Checkpoint:**
- Best audience segment identified
- Winning offer confirmed
- Best video hook found
- CPA should be 20-40% lower than Week 1

### Phase 3: Landing Page & Placement Tests (Weeks 9-12)

**Goal:** Optimize post-click experience and placement efficiency.

| Week | Test | Variable | Variants | Duration | KPI |
|---|---|---|---|---|---|
| Week 9-10 | Test 7 | **Landing Page Headline** | Ad-matched headline vs benefit headline | 14 days | LP CVR |
| Week 10-11 | Test 8 | **Landing Page CTA** | Button text, color, placement | 14 days | LP CVR |
| Week 11-12 | Test 9 | **Placements** | Feed-only vs All Placements vs Reels-only | 7-10 days | CPM + CPA |

**End of Phase 3 Checkpoint:**
- Landing page optimized (should see 20%+ CVR improvement from baseline)
- Best placements identified
- Full funnel performance documented

### Ongoing (Month 4+)

After the 90-day foundation, run continuous tests:

| Frequency | What to Test | Why |
|---|---|---|
| Every 2 weeks | New creative variations | Combat ad fatigue, find new angles |
| Monthly | New audience segments | Expand reach while maintaining CPA |
| Monthly | Bidding strategy adjustments | Optimize cost efficiency as data grows |
| Quarterly | New platforms | Test emerging channels (TikTok, Pinterest, Snapchat) |
| Quarterly | Full funnel restructure | Re-evaluate funnel stage allocation |

---

## Winner Criteria & Decision Framework

### How to Declare a Winner

```
Step 1: Has the test reached minimum sample size? (Check calculator above)
  → No: Keep running. Do not peek or make decisions.
  → Yes: Proceed to Step 2.

Step 2: Is the result statistically significant at your threshold?
  → No: The test is inc

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [zubair-trabzada](https://github.com/zubair-trabzada)
- **Source:** [zubair-trabzada/ai-ads-claude](https://github.com/zubair-trabzada/ai-ads-claude)
- **License:** MIT
- **Homepage:** https://www.skool.com/aiworkshop

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-zubair-trabzada-ai-ads-claude-ads-testing
- Seller: https://agentstack.voostack.com/s/zubair-trabzada
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.