StatsPAI

English | 中文

StatsPAI: Agent-Native Causal Inference & Econometrics Toolkit for Python

[](https://pypi.org/project/StatsPAI/) [](https://pypi.org/project/StatsPAI/) [](https://github.com/brycewang-stanford/statspai/blob/main/LICENSE) [](https://github.com/brycewang-stanford/statspai/actions) [](https://brycewang-stanford.github.io/StatsPAI/) [](https://pepy.tech/projects/statspai) [](https://joss.theoj.org/papers/9f1c837b1b1df7adfcdd538c3698e332) [](https://doi.org/10.5281/zenodo.19933900)

StatsPAI is a validation-tiered Python library for causal inference and applied econometrics. One import, 1,000+ registered functions across 80+ submodules (live count: python scripts/registry_stats.py), spanning classical econometrics, ML/AI causal methods, and reporting utilities for Word, Excel, and LaTeX.

Built for AI agents: every registered function has machine-readable discovery metadata (list_functions(), describe_function(), function_schema()), and parity-backed functions expose an explicit validation_status so agents and humans can distinguish certified numerical evidence from API-stable breadth.

It places Python-native implementations, selected R-style causal workflows, and Stata-style econometric interfaces behind a shared import, registry, and reporting layer.

---

For Journal Reviewers

StatsPAI is currently under review at JOSS. Use the path that matches your role:

1. Current JOSS review — install and smoke-test the package with [docs/joss_reviewer_guide.md](docs/jossreviewerguide.md), then inspect validation evidence in [docs/joss_validation_dossier.md](docs/jossvalidationdossier.md). Design rationale and frequently-raised questions — scope, state of the field, and the build-vs-contribute case — are answered in [docs/joss_reviewer_qa.md](docs/jossreviewerqa.md).
2. Run representative offline examples from [examples/](examples/).
3. Check contribution and support pathways in [CONTRIBUTING.md](CONTRIBUTING.md), [SUPPORT.md](SUPPORT.md), and the GitHub issue templates under [.github/ISSUE_TEMPLATE/](.github/ISSUE_TEMPLATE/).

The examples use bundled teaching datasets and require no network access after installation.

Reproduce the high-dimensional / double-ML numerical evidence (the part most relevant to that literature): StatsPAI's rigorous-Lasso port reproduces hdm's three vignette applications — Barro–Lee growth convergence, Acemoglu–Johnson–Robinson institutions IV, and the CPS 2012 gender wage gap — provides a faithful port of hdm's logistic high-dimensional effects (rlassologit_effect), and pins its double-machine-learning estimators against DoubleML.

# rigorous-Lasso (hdm) parity — no R or extras needed (reference committed as JSON)
pytest tests/reference_parity/test_rlasso_parity.py \
       tests/reference_parity/test_rlasso_vignette_parity.py \
       tests/reference_parity/test_rlassologit_effect_parity.py -q

# DoubleML parity (optional: needs the `parity` extra = doubleml-for-py)
pip install -e ".[dev,parity]" && pytest tests/external_parity/test_dml_python_parity.py -q

See [docs/guides/rigorous_lasso_hdm.md](docs/guides/rigorouslassohdm.md) for the full hdm ↔ StatsPAI function map and [docs/guides/case_study_401k.md](docs/guides/casestudy401k.md) for the canonical 401(k) reproduction with sp.dml.

---

Quick Start — 60 seconds

pip install statspai, then run any of the four canonical causal-inference exercises below. StatsPAI ships the classic teaching datasets bundled under sp.datasets — Callaway–Sant'Anna mpdta, Card (1995) returns-to-schooling, Abadie–Diamond–Hainmueller California Prop 99, Lee (2008) Senate RD, LaLonde / NSW–DW, Angrist–Krueger (1991) QOB, Basque terrorism, German reunification — so every snippet runs offline with no data wrangling.

import statspai as sp

sp.datasets.list_datasets()   # name / design / n_obs / paper / expected_main

DiD — Callaway & Sant'Anna staggered ATT on mpdta

Minimum-wage effect on teen employment (the canonical example used in R's did package).

import statspai as sp

df = sp.datasets.mpdta()
cs = sp.callaway_santanna(data=df, y='lemp', t='year',
                          i='countyreal', g='first_treat')
print(sp.aggte(cs, type='simple').summary())
# Simple ATT ≈ -0.033, bootstrap SE ≈ 0.008, p ` tag in
[`README.md`](README.md) and [`README_CN.md`](README_CN.md) to the
absolute raw GitHub URL so the banner loads on PyPI / TestPyPI / any
off-GitHub README renderer. **No code changes** — all shipped module
files match v1.15.2 exactly; only the long-description metadata baked
into the wheel + sdist regenerates.

**📦 v1.15.2 (2026-05-17) — Strict-JSON MCP wire + dual-track replicate guides + release-packaging hardening**

Patch release on top of v1.15.1 with **no estimator numerical change**.
Three independent hardening tracks land together: (1) `sp.agent.mcp_server`
now produces strict-JSON-clean output — native `NaN` / `±Infinity` floats
are walked to `null` before serialisation so RFC 8259 parsers
(including Claude Desktop) never see a token they reject; (2)
`sp.replicate` graduates four canonical replications — Card (1995),
Abadie-Diamond-Hainmueller (2010) California Prop 99, Lalonde (1986) /
DW (1999), and Lee (2008) Senate RD — from single-track stubs to full
**classic + modern** recipes on bundled real CSVs with pinned golden
numbers; (3) release packaging tightens — wheel smoke tests fail loudly
on `ImportError`, `py.typed` ships in the wheel, the result
`_repr_html_` path escapes user-controlled cells (notebook XSS-safety),
and a new `[text]` extra makes `sentence-transformers` an explicit
opt-in for `sp.causal_text`. Install with
`pip install --upgrade statspai`. Full notes in
[`CHANGELOG.md`](CHANGELOG.md) under `[1.15.2]`.

**📦 v1.15.1 (2026-05-07) — R-parity RD opt-in + negative-binomial implementation notes**

Patch release preparing the PyPI cut after v1.15.0. `sp.rdrobust`
now accepts `bwselect='cct'`, an opt-in path that delegates bandwidth
selection and robust bias-corrected inference to the official
`rdrobust>=1.3` Python port for bit-equal R `rdrobust::rdrobust`
replications. The default `bwselect='mserd'` remains unchanged, so
existing RD pipelines keep their numbers unless they explicitly opt in.
Install the exact-parity path with `pip install statspai[rd-cct]`.

This release also documents the negative-binomial count-regression
surface. `sp.nbreg` is a log-link MLE with NB2 by default
(`Var[Y|X] = μ + αμ²`) and an NB1 option via `dispersion='constant'`
(`Var[Y|X] = μ(1 + δ)`). The optimizer starts from Poisson IRLS, then
alternates NB-weighted IRLS for the coefficients with scalar profile-
likelihood optimization for the dispersion parameter. It supports
offsets, exposure, weights, IRR reporting, HC/cluster SEs, a likelihood-
ratio test against Poisson, and formula fixed effects such as
`y ~ x | id` via explicit dummy expansion for moderate panels.
`sp.xtnbreg(model='fe')` wraps that fixed-effect path and clusters by
entity by default; `model='re'` dispatches to `sp.menbreg`, the
random-intercept NB2 GLMM. Full notes in [`CHANGELOG.md`](CHANGELOG.md)
under `[1.15.1]`.

**📦 v1.15.0 (2026-05-06) — Five polish waves (IV / synth / decomposition / ML+causal / RDD)**

A bundled minor release covering five module-level polish waves on top
of v1.13.1 (1.14.0 was an internal cut never released to PyPI). **IV**:
a new `sp.iv.iv_diag` reporting bundle (Olea-Pflueger effective F,
Lee-McCrary-Moreira-Porter `tF` adjusted CI, Anderson-Rubin / Moreira
CLR / Kleibergen K weak-IV-robust sets, Conley-Hansen-Rossi LTZ
sensitivity, Blandhol-Mogstad-Słoczyński TSLS-as-LATE caveat) plus
`sp.iv.iv_compare` forest comparison and four IV diagnostic plots.
**Synth**: supported synthetic-control result objects gain a publication-oriented
`.to_latex()` / `.to_excel()` / `.to_word()` table-export pipeline,
trajectory and gap plots get prediction-interval / pre-RMSPE ribbons
(Cattaneo-Feng-Titiunik 2021 / Cattaneo-Feng-Palomba-Titiunik 2025), and
the SDID schema is canonicalised so `sp.synth_report(method='sdid')`
produces a bundled report. **Decomposition**: a new Yu-Elwert (2024)
distributional-decomposition module + a unified `sp.decompose()`
dispatcher and shared influence-function / WLS / statistic-value
backbone. **ML+causal**: `sp.dml_sensitivity` (Chernozhukov-Cinelli-
Newey-Sharma-Syrgkanis 2022 long-story-short framework),
`sp.dml_diagnostics` (DoubleML-style 2×2 panel), `sp.cate_eval`
(Yadlowsky-et-al 2025 RATE / AUTOC / Qini decoupled from the forest
backbone), causal-discovery DAG visualisation, `PolicyTreeResult`
promotion, and ⚠️ correctness fixes in
`forest.CausalForest.best_linear_projection` (now AIPW + HC1 SEs) and
`mediation.mediate` (no silent bootstrap-failure substitution). **RDD**:
three new estimators (`sp.rd_flex` cross-fit ML adjustment,
`sp.rd_bias_aware_fuzzy` Anderson-Rubin-style weak-IV-robust fuzzy CI,
`sp.rd_discrete` Kolesár-Rothe honest CIs for discrete RVs), three
reporting helpers (`sp.rd_dashboard`, `sp.rd_compare`,
`sp.rd_robustness_table`), `rho` parameter + discrete-RV / weak-first-
stage warnings on `sp.rdrobust`, and a Cattaneo-Jansson-Ma (2020)
boundary-adaptive `sp.rdplotdensity` upgrade. Also: BJS imputation
(Borusyak-Jaravel-Spiess 2024) DiD support repaired, neural-causal /
synth / spatial-DID export modules added, and a `sp.iv(absorb=...)`
HDFE 2SLS path. Full notes in [`CHANGELOG.md`](CHANGELOG.md) under
`[1.15.0]`.

**📦 v1.13.1 (2026-05-05) — Stability tiers + external-validity dossier + cold-start surgery**

v1.13 stamps every `FunctionSpec` with a `stability` tier (`stable` /
`experimental` / `deprecated`) plus per-function `limitations`,
surfaced through `sp.describe_function`, `sp.list_functions(stability=...)`,
the `statspai list` CLI, and the LLM-facing `sp.function_schema`;
`sp.recommend` / `sp.causal` / `sp.paper` default to dropping
`experimental` / `deprecated` entries unless `allow_experimental=True`
is passed. Eight high-impact estimators (`aipw`, `aggte`,
`pretrends_test`, `sensitivity_rr`, `mccrary_test`, `oster_bounds`,
`wild_cluster_bootstrap`, `rd_honest`) are upgraded from
auto-registered stubs to hand-written specs. A weak-instrument
preflight gate in `sp.preflight(... "ivreg", formula=...)` flags
first-stage F below the Staiger–Stock (1997) / Stock–Yogo (2005)
thresholds, and `sp.recommend(... design='iv')` adaptively reorders
LIML / AR ahead of 2SLS on weak first stages. v1.13 also ships a
36-module R parity harness (`tests/r_parity/`), a 21-module Stata
parity harness (`tests/stata_parity/`), 4 canonical-dataset
original-paper replays (Card 1995, Callaway–Sant'Anna `mpdta`, Abadie
Basque, LaLonde NSW + PSID-1 — all bit-equal to the published headline
numbers), a Track-C performance harness (HDFE / CS-DiD / SCM / DML
log-log scaling), a B=1000 Monte-Carlo coverage run on
`tests/coverage_monte_carlo/` (seven materialized nominal rows:
OLS 0.952 / 2×2 DiD 0.955 / strong-Z IV 0.962 / Callaway-Sant'Anna
0.946 / entropy balancing 1.000 / DML 0.969 / causal-forest AIPW
0.977; rows above the 99% Wilson band [0.935, 0.967] are reported as
conservative over-coverage), and a 900-trial
CausalAgentBench prompt suite (mock mode shipped, `--api` one switch
away). Three new top-level meta-APIs — `sp.validation_report()`,
`sp.coverage_matrix()`, `sp.reproduce_jss_tables()` — let referees
verify StatsPAI's external-validity claims without leaving Python. Cold
start: `statspai.forest` is lazy-loaded (Step 1B), 18 estimator files
import sklearn lazily (Step 1C), and HAL TMLE drops sklearn class
inheritance (Step 1D); `import statspai` now pulls **0** sklearn
submodules (down from 245). **⚠️ Correctness fix** —
`sp.callaway_santanna(method='reg')` had a latent influence-function
scaling bug; `'ipw'` and `'dr'` are unchanged but **re-run any
v1.10–v1.13 CS-DiD analyses that used `method='reg'`**. Full notes in
[`CHANGELOG.md`](CHANGELOG.md) under `[1.13.1]`.

**📦 v1.12.2 (2026-05-01) — ML routing for `sp.causal_question` + shared robustness battery + weighted PLIV/IIVM**

Patch release on top of v1.12.0's DML hardening: `sp.causal_question`
now routes `design='dml'|'tmle'|'metalearner'|'causal_forest'` directly
into the correct estimator with the right identification story; both
`sp.paper(...)` entry points share a new design-aware robustness battery
(`workflow/_robustness.py`); `sp.llm_annotator_correct` graduates to
multi-class with bias-corrected bootstrap + SE-inflation diagnostic;
`sp.dml(model='pliv'|'iivm')` now honours `sample_weight` end-to-end.
No estimator output changes for v1.12.0/1.12.1 call sites. Full notes
in [`CHANGELOG.md`](CHANGELOG.md) under `[1.12.2]`.

**📦 v1.12.0 (2026-04-30) — DML hardening + TMLE correctness pass**

A two-workstream maintenance release; full release notes in
[`CHANGELOG.md`](CHANGELOG.md) under `[1.12.0]` and breaking-change
migration in [`MIGRATION.md`](MIGRATION.md#v111--v112--dml-module-hardening).

- **⚠️ Correctness — DML.** `sp.dml(model='irm' | 'iivm')` now uses
  `StratifiedKFold` (was `KFold`); empty subgroup folds raise
  `IdentificationFailure` instead of silently filling AIPW scores with
  zeros. `sp.dml_panel(binary_treatment=True)` is a deprecated no-op
  (the old classifier-on-within-demeaned path produced a propensity
  with no clean `E[D̃|X̃]` interpretation). `sp.dml_model_averaging`
  default `weight_rule` is now `"short_stacking"` (Ahrens, Hansen,
  Schaffer & Wiemann 2025 *JAE* eq. 7) — pass `weight_rule="inverse_risk"`
  to recover the v1.11 behaviour. PLIV weak-IV partial-correlation
  floor tightened from `1e-6` to `1e-3` and a residual-variance-ratio
  guard catches the perfectly-collinear-instrument case.
- **⚠️ Correctness — TMLE.** `sp.tmle.SuperLearner` now solves a proper
  simplex-constrained QP (was NNLS with post-hoc rescaling — off-simplex
  except by coincidence). `sp.tmle.ltmle` censoring half-implementation
  closed; `sp.tmle.ltmle_survival` separates RMST and terminal-RD
  influence functions (was using a single non-target-functional EIF
  for both). `sp.hal_tmle(variant='projection')` raises
  `NotImplementedError` honestly until the Riesz-projection step is
  ported (was a silent no-op shrinkage on the post-fit ε).
- **Added — DML.** `random_state=` and `sample_weight=` on every
  `sp.dml(model=...)` call. `sample_weight=` is fully supported on
  PLR / IRM / PLIV / IIVM / `sp.dml_panel` / `sp.dml_model_averaging`
  (Z-estimator sandwich variance throughout). Every variant now
  populates `model_info["diagnostics"]` (propensity range, n clipped,
  subgroup-fallback counts, partial corr, approximate first-stage F).

---

**🎉 NEW in v1.8 — Native Rust IRLS for `sp.fast.fepois` · `sp.prod_fn` production functions · `regtable` Rounds 1-4 · estimator provenance 142/925**

StatsPAI 1.8.0 (2026-04-28) lands a 3× wall-clock speed-up on the medium HDFE benchmark and a brand-new structural-estimation module. **No numerical changes** to existing v1.7.x estimators — the Rust IRLS path is bit-for-bit identical to the NumPy fallback (verified by `test_fepois_native_irls_vs_python_irls_parity`).

| Area | v1.8 Highlights |
| --- | --- |
| **Native Rust IRLS** | `sp.fast.fepois` runs at **0.855 s** on the standard medium dataset (n=1M, fe1=100k, fe2=1k) vs the v1.7.x baseline 2.61 s — **3.05× speed-up**, and **1.34× of R `fixest::fepois`** (well under the ≤ 1.5× target). Closes the long-standing wall-clock gap to `fixest`. Three orthogonal contributions: Phase A (Rust scatter), Phase B0 (sort-by-FE sequential sweep + dispatcher cache), Phase B1 (single-PyO3-call `fepois_irls` state machine), Path A (Rust separation pre-pass). All audited at `benchmarks/hdfe/AUDIT.md`. |
| **Production functions** | New `sp.prod_fn` dispatcher: `olle

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [brycewang-stanford](https://github.com/brycewang-stanford)
- **Source:** [brycewang-stanford/StatsPAI](https://github.com/brycewang-stanford/StatsPAI)
- **License:** MIT

Install and usage instructions live in the source repository linked above.