# Rct Data Analysis

> Analyze data from randomized controlled trials. Use when user mentions: treatment effects, ITT analysis, LATE, TOT, compliance analysis, attrition, balance checks, heterogeneous effects, RCT results, experimental analysis.

- **Type:** Skill
- **Install:** `agentstack add skill-sshtomar-claude-code-skills-social-science-rct-data-analysis`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [sshtomar](https://agentstack.voostack.com/s/sshtomar)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [sshtomar](https://github.com/sshtomar)
- **Source:** https://github.com/sshtomar/claude-code-skills-social-science/tree/main/skills/rct-data-analysis

## Install

```sh
agentstack add skill-sshtomar-claude-code-skills-social-science-rct-data-analysis
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

RCT analysis leverages randomization to identify causal effects with minimal assumptions. Intention-to-treat (ITT) provides policy-relevant estimates of assignment effects. Local average treatment effects (LATE/IV) recover efficacy for compliers. Balance checks verify randomization, attrition analysis guards against selection bias, and heterogeneity analysis reveals for whom treatment works.

Analysis must preserve the integrity of randomization through proper inference and transparent reporting.

  Report ITT as Primary Result
  Intention-to-treat analysis MUST be reported as the main result, even with non-compliance
  ITT maintains randomization, provides policy-relevant parameter (effect of offering treatment), and prevents selection bias from endogenous compliance (Angrist & Pischke 2009)
  Selection bias, loss of causal interpretation, inability to make policy recommendations

  Cluster-Robust Standard Errors
  Use cluster-robust (or HC3-robust) standard errors, never classical SEs
  Clustered/robust SEs account for within-cluster correlation and heteroskedasticity. Classical SEs severely underestimate uncertainty (Bertrand et al. 2004)
  Type I error rates of 45% instead of 5%, false positives, invalid inference

  Balance Table Before Outcomes
  Check and report covariate balance on baseline characteristics before analyzing outcomes
  Balance checks verify randomization worked. Large imbalances suggest implementation problems or sampling variation requiring controls
  Hidden confounding, invalid attribution of effects, inability to diagnose randomization failures

  Address Attrition Explicitly
  Report attrition rates by treatment arm, test for differential attrition, bound effects if substantial
  Differential attrition creates selection bias that undermines randomization. Must be addressed or results are invalid (Lee 2009)
  Selection bias, invalid causal claims, inability to distinguish true effects from attrition-driven patterns

  Adjust for Multiple Testing When Examining Multiple Outcomes
  Use Bonferroni, Holm, or FDR adjustment when testing multiple hypotheses
  Multiple testing inflates Type I error rate. Without adjustment, 5% significance means 26% false positive rate with 5 tests
  False discoveries, overstated evidence, publication of spurious results

  Loss to follow-up is unrelated to treatment assignment and potential outcomes
  Compare attrition rates by treatment arm, test for correlation with baseline characteristics
  Use Lee bounds, IPW, or explicitly bound treatment effects under worst-case scenarios

  Random assignment affects outcome only through treatment receipt (not directly)
  Conceptual argument, check for alternative mechanisms (Hawthorne effects, etc.)
  ITT is still valid and policy-relevant; LATE estimates are biased

  Assignment doesn't flip treatment direction (no defiers)
  Check if always-takers exist in control group and never-takers in treatment
  LATE estimates are weighted average that may be misleading

When analyzing RCT data:
1. Check data quality and completeness
2. Verify balance on baseline covariates
3. Calculate and report attrition rates (overall and differential)
4. Estimate ITT (primary analysis)
5. Estimate LATE if non-compliance exists (secondary)
6. Test for heterogeneous effects (pre-specified subgroups)
7. Conduct robustness checks (alternative specifications)
8. Report all results transparently (including null findings)

```python
@app.cell
def itt_analysis_rct(df):
    # Intention-to-treat analysis: Effect of assignment to treatment
    # This is the policy-relevant parameter even with imperfect compliance

    import statsmodels.formula.api as smf
    import pandas as pd

    # Verify data structure
    required = {'outcome', 'treatment', 'unit_id'}
    assert required.issubset(df.columns), f"Missing columns: {required - set(df.columns)}"

    # CRITICAL: Use cluster-robust or HC3-robust standard errors
    if 'cluster_id' in df.columns:
        # Cluster-robust (for cluster-randomized trials)
        model = smf.ols("outcome ~ treatment", data=df).fit(
            cov_type='cluster',
            cov_kwds={'groups': df['cluster_id']}
        )
        se_type = "Cluster-robust"
    else:
        # HC3-robust (for individual randomization)
        model = smf.ols("outcome ~ treatment", data=df).fit(cov_type='HC3')
        se_type = "HC3-robust"

    # Extract results
    itt_effect = model.params['treatment']
    itt_se = model.bse['treatment']
    itt_pval = model.pvalues['treatment']
    ci_low, ci_high = model.conf_int().loc['treatment']

    # Context for interpretation
    control_mean = df[df['treatment'] == 0]['outcome'].mean()
    control_sd = df[df['treatment'] == 0]['outcome'].std()
    effect_size = itt_effect / control_sd

    print("INTENTION-TO-TREAT (ITT) ANALYSIS")
    print("=" * 60)
    print(f"Outcome: {df.columns[df.columns.get_loc('outcome')]}")
    print(f"N: {len(df)} ({df['treatment'].sum()} treatment, {(~df['treatment'].astype(bool)).sum()} control)")
    print(f"Standard errors: {se_type}")
    print(f"\nITT Effect: {itt_effect:.4f}")
    print(f"Std. Error: {itt_se:.4f}")
    print(f"95% CI: [{ci_low:.4f}, {ci_high:.4f}]")
    print(f"p-value: {itt_pval:.4f}")
    print(f"\nControl mean: {control_mean:.4f}")
    print(f"Effect size: {effect_size:.3f} SD")
    print(f"Relative effect: {100*itt_effect/control_mean:.1f}%")

    return model,
```

Verify randomization balance before analyzing outcomes

```python
@app.cell
def check_balance(df):
    # Balance checks verify randomization succeeded
    # Large imbalances may require controls or suggest implementation issues

    import pandas as pd
    import numpy as np
    from scipy import stats

    # Baseline covariates (measured before randomization)
    baseline_vars = ['age', 'education', 'income', 'baseline_outcome']

    results = []
    for var in baseline_vars:
        # Means by treatment group
        control_mean = df[df['treatment'] == 0][var].mean()
        treat_mean = df[df['treatment'] == 1][var].mean()

        # Normalized difference (Imbens & Rubin 2015)
        control_var = df[df['treatment'] == 0][var].var()
        treat_var = df[df['treatment'] == 1][var].var()
        norm_diff = (treat_mean - control_mean) / np.sqrt((control_var + treat_var) / 2)

        # T-test
        t_stat, p_val = stats.ttest_ind(
            df[df['treatment'] == 0][var].dropna(),
            df[df['treatment'] == 1][var].dropna()
        )

        results.append({
            'Variable': var,
            'Control': f"{control_mean:.3f}",
            'Treatment': f"{treat_mean:.3f}",
            'Norm Diff': f"{norm_diff:.3f}",
            'p-value': f"{p_val:.3f}"
        })

    balance_table = pd.DataFrame(results)

    print("BALANCE TABLE")
    print("=" * 60)
    print(balance_table.to_string(index=False))

    # Joint F-test
    from statsmodels.formula.api import ols
    formula = "treatment ~ " + " + ".join(baseline_vars)
    joint_model = ols(formula, data=df).fit()

    print(f"\nJoint F-test: F={joint_model.fvalue:.3f}, p={joint_model.f_pvalue:.4f}")

    # Flag concerns
    large_diffs = sum(abs(float(r['Norm Diff'])) > 0.25 for r in results)
    if large_diffs > 0:
        print(f"\nNote: {large_diffs} variables have |norm diff| > 0.25")
        print("Consider including these as controls in regression")

    return balance_table,
```

Balance checks use baseline (pre-randomization) covariates only. Large imbalances (|norm diff| > 0.25) don't invalidate randomization but suggest including those variables as controls to improve precision. Never use post-treatment variables for balance checks.

  Not using cluster-robust or heteroskedasticity-robust standard errors
  Type I error rates can be 45% instead of 5%, leading to massive false positive rates
  ALWAYS use cov_type='cluster' or cov_type='HC3'. Never use default SEs.

  Not reporting ITT when non-compliance exists
  Selection bias from analyzing compliers only, loss of causal interpretation
  ALWAYS report ITT as primary result. LATE/TOT are secondary sensitivity analyses.

  Ignoring attrition or not testing for differential attrition
  Selection bias can completely invalidate results if attrition is differential
  Report attrition by arm, test differential attrition, use Lee bounds if substantial

  P-hacking through subgroups without adjustment
  Finding "significant" effects that are just Type I errors from multiple testing
  Pre-specify subgroups in PAP, use Bonferroni/Holm adjustment, report all tests

  Using baseline covariates that were measured post-randomization
  Conditioning on post-treatment variables creates bias (bad controls)
  Only use covariates measured before randomization in balance checks and controls

Minimum reporting for RCT results:
- Sample size (by treatment arm)
- Balance table on baseline characteristics
- Attrition rates (overall and by arm)
- ITT estimates with cluster-robust/HC3 SEs
- 95% confidence intervals
- Control group mean (for context)
- Effect size in SD units
- P-values (exact, not 

Results are questionable if:
- No balance table provided
- Classical (non-robust) standard errors used
- ITT not reported for study with non-compliance
- Attrition >20% without bounding analysis
- Multiple outcomes tested without adjustment
- Post-treatment covariates included as controls
- Results only reported for "compliers" without ITT

Angrist, J. D., & Pischke, J. S. (2009). Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press.
Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How much should we trust differences-in-differences estimates? Quarterly Journal of Economics, 119(1), 249-275.
Lee, D. S. (2009). Training, wages, and sample selection: Estimating sharp bounds on treatment effects. Review of Economic Studies, 76(3), 1071-1102.
Imbens, G. W., & Rubin, D. B. (2015). Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge University Press.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [sshtomar](https://github.com/sshtomar)
- **Source:** [sshtomar/claude-code-skills-social-science](https://github.com/sshtomar/claude-code-skills-social-science)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-sshtomar-claude-code-skills-social-science-rct-data-analysis
- Seller: https://agentstack.voostack.com/s/sshtomar
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.