# Rct Power Calculations > Calculate statistical power and sample sizes for RCTs. Use when user mentions: power analysis, sample size calculation, minimum detectable effect, MDE, statistical power, effect size, clustering effects, design effect, ICC, intra-cluster correlation. - **Type:** Skill - **Install:** `agentstack add skill-sshtomar-claude-code-skills-social-science-rct-power-calculations` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [sshtomar](https://agentstack.voostack.com/s/sshtomar) - **Installs:** 0 - **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [sshtomar](https://github.com/sshtomar) - **Source:** https://github.com/sshtomar/claude-code-skills-social-science/tree/main/skills/rct-power-calculations ## Install ```sh agentstack add skill-sshtomar-claude-code-skills-social-science-rct-power-calculations ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About Power analysis determines the sample size required to detect treatment effects of a given magnitude with specified probability. Adequate power prevents Type II errors (failing to detect real effects). Power calculations must account for clustering (design effects), non-compliance (reduces effective sample), attrition (reduces final sample), and multiple testing (inflates α). Underpowered studies waste resources and risk being misinterpreted as evidence of no effect. Power = 80% is conventional. Higher power (90%) is better when study is expensive or one-shot. Conduct Power Analysis Before Data Collection Calculate required sample size BEFORE starting enrollment or randomization Post-hoc power calculations are meaningless. Power must guide design, not rationalize results (Hoenig & Heisey 2001) Underpowered studies waste resources, fail to detect real effects, mislead policy with false nulls Account for Clustering Design Effects When using cluster randomization, multiply base sample size by design effect: DE = 1 + (m-1) × ICC Clustering reduces effective sample size due to within-cluster correlation. Ignoring this causes severe underpowering (Donner & Klar 2000) Studies underpowered by 2-4x, inability to detect real effects despite large nominal sample Inflate for Attrition and Non-Compliance Adjust sample size for expected attrition and imperfect compliance before finalizing design Attrition reduces final sample. Non-compliance attenuates ITT effects. Both reduce power substantially Final analysis underpowered even if initial enrollment meets naive power target Use Realistic Effect Sizes Base MDE on pilot data, similar interventions, or smallest policy-relevant effect, not on wishful thinking Overoptimistic effect size assumptions guarantee underpowered study. True effects in social science typically 0.1-0.3 SD (Vivalt 2020) Study designed to detect implausibly large effects, fails to detect realistic effects, wasted resources Adjust for Multiple Comparisons When testing multiple outcomes, increase sample size or reduce α to maintain family-wise error rate Multiple testing inflates Type I error. With 5 tests at α=0.05, false positive rate is 26% not 5% Study powered for single test but not for actual analysis plan, false discoveries Assumed MDE or effect size is realistic based on prior evidence or theory Review similar interventions, conduct pilot, consult domain experts, use smallest policy-relevant effect Study either grossly overpowered (wasteful) or underpowered (fails to detect real effects) Assumed intra-cluster correlation reflects true within-cluster similarity Use baseline data from same clusters, similar studies, or conservative upper bound (ICC=0.10-0.20) Design effect wrong → sample size wrong → power incorrect Expected attrition and take-up rates match what actually occurs Base on pilot data, similar studies in same context, plan retention strategies Actual power lower than designed power, inability to detect effects When conducting power analysis: 1. Specify primary outcome and effect size (MDE or expected effect) 2. Choose significance level (α=0.05 typical) and desired power (0.80-0.90) 3. Calculate base sample size for simple design 4. Apply design effect if cluster randomization (multiply by 1+(m-1)×ICC) 5. Inflate for expected non-compliance (divide by compliance²) 6. Inflate for expected attrition (divide by (1-attrition_rate)) 7. Adjust for multiple testing if applicable 8. Check feasibility; if infeasible, reduce scope or accept larger MDE ```python @app.cell def comprehensive_power_calculation(): # Full power calculation with all real-world adjustments # Demonstrates compounding inflation factors for RCT sample size import numpy as np from scipy.stats import norm # Parameters alpha = 0.05 power = 0.80 mde = 0.25 # Effect size in SD units (realistic for social programs) p = 0.50 # Treatment allocation # Base calculation z_alpha = norm.ppf(1 - alpha/2) z_power = norm.ppf(power) n_base = ((z_alpha + z_power)**2) / (p * (1-p) * mde**2) print(f"COMPREHENSIVE POWER ANALYSIS") print(f"=" * 60) print(f"Parameters: α={alpha}, power={power}, MDE={mde} SD") print(f"\n1. Base sample size: {int(np.ceil(n_base))}") # Adjustment 1: Clustering icc = 0.05 cluster_size = 30 design_effect = 1 + (cluster_size - 1) * icc n_after_clustering = n_base * design_effect print(f"\n2. Clustering adjustment:") print(f" ICC={icc}, avg cluster size={cluster_size}") print(f" Design effect: {design_effect:.2f}") print(f" After clustering: {int(np.ceil(n_after_clustering))}") # Adjustment 2: Non-compliance compliance_rate = 0.70 n_after_compliance = n_after_clustering / (compliance_rate**2) print(f"\n3. Non-compliance adjustment:") print(f" Expected compliance: {compliance_rate:.0%}") print(f" Inflation factor: {1/compliance_rate**2:.2f}") print(f" After compliance: {int(np.ceil(n_after_compliance))}") # Adjustment 3: Attrition attrition_rate = 0.20 n_final = n_after_compliance / (1 - attrition_rate) print(f"\n4. Attrition adjustment:") print(f" Expected attrition: {attrition_rate:.0%}") print(f" Inflation factor: {1/(1-attrition_rate):.2f}") print(f" FINAL REQUIRED N: {int(np.ceil(n_final))}") print(f"\n" + "=" * 60) print(f"Total inflation: {n_final/n_base:.2f}x base sample size") print(f"Baseline N needed: {int(np.ceil(n_final))}") print(f"Endline N expected: {int(np.ceil(n_final * (1-attrition_rate)))}") return int(np.ceil(n_final)), ``` Calculate power for cluster-randomized trial with design effect ```python @app.cell def cluster_rct_power(): # Cluster RCT power accounting for design effect # Shows how clustering can reduce power by 2-4x import numpy as np from scipy.stats import norm alpha = 0.05 power = 0.80 mde = 0.30 # Individual randomization (baseline) z_a = norm.ppf(1 - alpha/2) z_p = norm.ppf(power) n_individual = ((z_a + z_p)**2) / (0.25 * mde**2) print(f"CLUSTER vs. INDIVIDUAL RANDOMIZATION") print(f"=" * 60) print(f"Individual randomization: {int(np.ceil(n_individual))} participants") # Cluster randomization scenarios scenarios = [ {"icc": 0.05, "cluster_size": 20}, {"icc": 0.05, "cluster_size": 50}, {"icc": 0.10, "cluster_size": 30}, {"icc": 0.20, "cluster_size": 30}, ] for scenario in scenarios: icc = scenario["icc"] m = scenario["cluster_size"] de = 1 + (m - 1) * icc n_cluster = n_individual * de n_clusters_needed = int(np.ceil(n_cluster / m)) print(f"\nICC={icc}, cluster size={m}:") print(f" Design effect: {de:.2f}") print(f" Total N needed: {int(np.ceil(n_cluster))}") print(f" Clusters needed: {n_clusters_needed}") print(f" Inflation: {de:.2f}x individual design") print("\nKey lesson: Higher ICC or larger clusters → much larger N needed") return () ``` Design effect = 1 + (m-1) × ICC. With ICC=0.10 and m=30, DE=3.9, meaning you need 4x the individual-randomized sample size. Cluster randomization trades power for validity when spillovers are a concern. Conducting power analysis after data collection (post-hoc power) Meaningless exercise, doesn't inform anything, misinterpreted as evidence quality Power analysis is for design phase only. Never calculate "observed power" after study Ignoring clustering design effect in sample size calculation Study underpowered by 2-4x, fails to detect real effects despite large nominal sample ALWAYS multiply by design effect DE = 1+(m-1)×ICC for cluster randomization Using overoptimistic effect size assumptions Study designed to detect implausibly large effects, fails for realistic effects Use conservative estimates from pilots, literature, or smallest policy-relevant effect Not inflating for attrition and non-compliance Final analysis underpowered even if enrollment meets naive target Always inflate for expected attrition (÷ (1-attrition_rate)) and compliance (÷ compliance²) Treating power as binary threshold (80% good, 79% bad) Arbitrary decisions, missing that power is continuous and context-dependent Power is a continuum. Consider costs, effect importance, and Type I vs. II error tradeoffs For 80% power, α=0.05, 50/50 allocation: - MDE = 0.10 SD: ~3,140 total - MDE = 0.20 SD: ~786 total - MDE = 0.30 SD: ~350 total - MDE = 0.40 SD: ~198 total - MDE = 0.50 SD: ~128 total Then multiply by: - Design effect if clustering: 1+(m-1)×ICC - 1/compliance² for non-compliance - 1/(1-attrition_rate) for attrition If power calculation shows infeasible N: - Increase MDE (accept detecting only larger effects) - Reduce scope (fewer outcomes, simpler design) - Improve efficiency (stratification, ANCOVA with baseline) - Pool with other studies (consortium approach) - Use alternative design (regression discontinuity, DID) - Accept higher Type II error risk (document explicitly) Do NOT: proceed with underpowered study and hope for best. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Erlbaum. Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research. Handbook of Development Economics, 4, 3895-3962. Donner, A., & Klar, N. (2000). Design and Analysis of Cluster Randomization Trials in Health Research. Arnold Publishers. Hoenig, J. M., & Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. American Statistician, 55(1), 19-24. Vivalt, E. (2020). How much can we generalize from impact evaluations? Journal of the European Economic Association, 18(6), 3045-3089. ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [sshtomar](https://github.com/sshtomar) - **Source:** [sshtomar/claude-code-skills-social-science](https://github.com/sshtomar/claude-code-skills-social-science) - **License:** MIT Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-sshtomar-claude-code-skills-social-science-rct-power-calculations - Seller: https://agentstack.voostack.com/s/sshtomar - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.