Rct Core Design

Randomized Controlled Trials (RCTs) are the gold standard for causal inference when randomization is feasible. Random assignment eliminates selection bias and balances observed and unobserved confounders in expectation, enabling clean identification of treatment effects. Strong RCT design requires careful attention to power, ethics, implementation fidelity, and threats to validity.

RCTs answer "does X cause Y?" with minimal assumptions.

Pre-Registration and Pre-Analysis Plan Register trial and file pre-analysis plan BEFORE data collection or analysis begins Pre-registration prevents p-hacking, specification searching, and publication bias. Credibility revolution in social science demands transparency (Christensen & Miguel 2018) Results perceived as data-mined, journals may reject, inability to make credible causal claims

Adequate Statistical Power Conduct power analysis to ensure sufficient sample size for detecting meaningful effects Underpowered studies waste resources and risk null findings being misinterpreted as "no effect exists" (Button et al. 2013) Failure to detect true effects (Type II error), wasted intervention resources, misleading conclusions

IRB Approval Before Enrollment Obtain Institutional Review Board approval before recruiting any participants Ethical research requires informed consent, risk minimization, and protection of human subjects (Belmont Report 1979) Research misconduct, legal liability, inability to publish, harm to participants

Clear Theory of Change Document logical pathway from intervention inputs to expected outcomes before implementation Theory of change guides measurement, timing decisions, and outcome selection. Makes assumptions testable Measuring wrong outcomes at wrong times, inability to interpret null results, missing mechanisms

Randomization at Appropriate Level Choose randomization unit (individual, cluster, geographic) to minimize spillovers while maintaining power Wrong randomization level causes contamination (spillovers violate SUTVA) or severe power loss from clustering Biased estimates from spillovers OR inability to detect effects due to power loss

Treatment of one unit doesn't affect outcomes of other units Consider geographic proximity, social networks, market equilibrium effects Use cluster randomization, measure spillovers explicitly, or accept partial equilibrium estimates

Units assigned to treatment actually receive it (or model non-compliance) Monitor take-up rates during implementation, plan for imperfect compliance Report ITT (policy-relevant) and use IV for LATE (efficacy for compliers)

Treatment delivered consistently across units and time Implementation fidelity checks, standardized protocols, training Document variation, test for heterogeneous effects, consider implementation design

Loss to follow-up is unrelated to treatment or would-be outcomes Track attrition rates by treatment arm, test for differential attrition Use Lee bounds, inverse probability weighting, or report bounds on treatment effects

When designing an RCT:

1. Articulate research question and theory of change
2. Identify outcomes and measurement strategy (primary vs. secondary)
3. Conduct power analysis for required sample size
4. Choose randomization level (individual vs. cluster) based on spillovers
5. Design ethical treatment assignment (phase-in, lottery, encouragement)
6. Prepare IRB protocol and obtain approval
7. Register trial and file pre-analysis plan
8. Implement with fidelity monitoring
9. Plan for compliance, attrition, and threats to validity

@app.cell
def power_analysis_rct():
    # Calculate required sample size for RCT design
    # Based on Duflo et al. (2007) power calculation framework

    import numpy as np
    from scipy.stats import norm

    # Parameters
    alpha = 0.05  # Significance level
    power = 0.80  # Statistical power
    mde = 0.25  # Minimum detectable effect (SD units)
    p = 0.50  # Proportion assigned to treatment

    # Calculate critical values
    z_alpha = norm.ppf(1 - alpha/2)  # Two-tailed test
    z_power = norm.ppf(power)

    # Basic sample size
    n_base = ((z_alpha + z_power)**2) / (p * (1-p) * mde**2)

    # Adjust for clustering if applicable
    icc = 0.05  # Intra-cluster correlation
    cluster_size = 30  # Average cluster size
    design_effect = 1 + (cluster_size - 1) * icc
    n_clusters = n_base * design_effect / cluster_size

    # Adjust for attrition
    attrition_rate = 0.20
    n_final = n_base * design_effect / (1 - attrition_rate)

    print(f"POWER ANALYSIS FOR RCT")
    print(f"Parameters: α={alpha}, power={power}, MDE={mde} SD")
    print(f"\nBase sample size: {int(np.ceil(n_base))}")
    print(f"Design effect: {design_effect:.2f}")
    print(f"Clusters needed: {int(np.ceil(n_clusters))}")
    print(f"Final N (with attrition): {int(np.ceil(n_final))}")

    return int(np.ceil(n_final)),

Design RCT when direct denial of treatment raises ethical concerns

@app.cell
def ethical_rct_design():
    # Use phase-in design when denying treatment is ethically problematic
    # All units eventually receive treatment, but timing is randomized

    import pandas as pd
    import numpy as np

    # Context: Limited program slots, everyone gets treatment eventually
    # Solution: Randomize TIMING rather than ACCESS

    n_units = 300
    n_cohorts = 3  # Roll out in 3 waves

    # Assign units to cohorts randomly
    cohorts = np.random.choice(range(n_cohorts), size=n_units)

    # Cohort 1: Immediate (Year 1)
    # Cohort 2: Delayed (Year 2)
    # Cohort 3: Delayed (Year 3)

    df = pd.DataFrame({
        'unit_id': range(n_units),
        'cohort': cohorts,
        'year_treated': cohorts + 1  # Everyone gets treatment
    })

    print("PHASE-IN / STEPPED WEDGE DESIGN")
    print("=" * 50)
    print(df.groupby('cohort').size())

    print("\nEthical advantages:")
    print("- No one permanently denied treatment")
    print("- Addresses capacity constraints")
    print("- Still enables causal inference")

    print("\nAnalysis approach:")
    print("- Compare early vs. late cohorts")
    print("- Use DID or event study framework")
    print("- Longer follow-up for early cohorts")

    return df,

When direct denial is unethical: (1) Phase-in/stepped wedge (randomize timing), (2) Encouragement design (randomize encouragement, not access), (3) Lottery (when slots limited), (4) Oversubscription (randomize among eligible). Never deny established entitlements.

Not conducting power analysis before starting Underpowered study wastes resources, fails to detect real effects, misleads policy Always run power calculations with realistic effect sizes and account for clustering/attrition

Starting data collection before IRB approval and registration Research misconduct, inability to publish, ethical violations IRB approval and trial registration are prerequisites, not afterthoughts

Individual randomization when spillovers likely SUTVA violation, biased estimates, contaminated control group Use cluster randomization or buffer zones when spillovers expected

Measuring outcomes before treatment fully delivered Premature measurement finds null effects when treatment hasn't had time to work Theory of change specifies timing - measure outcomes after sufficient exposure period

Not planning for non-compliance Surprised by low take-up, insufficient power for actual treatment received Pilot to estimate take-up, power for ITT given expected compliance, design encouragement

Before launching RCT:

• [ ] Research question clearly specified
• [ ] Theory of change documented
• [ ] Power analysis conducted (accounts for clustering, attrition)
• [ ] Outcomes and measurement plan specified
• [ ] Randomization method chosen (individual vs. cluster)
• [ ] Ethical design confirmed (no unjustified denial)
• [ ] IRB protocol submitted and approved
• [ ] Trial registered (AEA RCT Registry or equivalent)
• [ ] Pre-analysis plan filed
• [ ] Implementation fidelity plan created
• [ ] Compliance monitoring plan ready
• [ ] Attrition tracking procedures established

Avoid RCTs when:

• Randomization is unethical (denying life-saving treatment)
• Spillovers are unavoidable and large (market equilibrium effects)
• Sample size insufficient for adequate power
• Treatment is national-level policy (no counterfactual)
• Cost vastly exceeds value of information gained
• Results won't inform decisions (no policy window)

Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research: A toolkit. Handbook of Development Economics, 4, 3895-3962. Glennerster, R., & Takavarasha, K. (2013). Running Randomized Evaluations: A Practical Guide. Princeton University Press. Christensen, G., & Miguel, E. (2018). Transparency, reproducibility, and the credibility of economics research. Journal of Economic Literature, 56(3), 920-980. Button, K.S., et al. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365-376. J-PAL Research Resources: https://www.povertyactionlab.org/research-resources

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: sshtomar
• Source: sshtomar/claude-code-skills-social-science
• License: MIT

Install and usage instructions live in the source repository linked above.