Synthetic Data

Synthetic Data Generation

> Generate structurally realistic synthetic datasets for pilot testing, power analysis, and method development.

Modes

| Mode | What it produces | Entry point | |------|-----------------|-------------| | From design | Synthetic data matching an existing experiment design document | "Generate test data for my experiment" | | From schema | Synthetic data from a user-described structure | "Create a dataset with these variables" | | Calibrated | Synthetic data calibrated to published summary statistics | "Make fake data matching these descriptives" |

Default: From schema. If an experiment design document exists in docs/, auto-select From design. If user provides published statistics, auto-select Calibrated.

When to Use

• Testing analysis code before real data collection
• Power analysis via simulation (complements /experiment-design Power mode)
• Method development and debugging estimation pipelines
• Generating pilot data for grant proposals or ethics applications
• Teaching demonstrations with realistic-looking data

When NOT to Use

• Designing the experiment itself --> /experiment-design
• Running analysis on real data --> /data-analysis
• Auditing identification strategy --> /causal-design

---

Workflow

Step 1: Detect Mode

Detect from context or ask:

| Signal | Mode | |--------|------| | docs/experiment-design.md exists | From design | | User describes variables, types, relationships | From schema | | User provides means, SDs, correlations from a paper | Calibrated | | Ambiguous | Ask |

Step 2: Interview for Data Structure

Gather the following (adapt questions to mode):

| Parameter | Question | Default | |-----------|----------|---------| | Variables | What variables do you need? | — | | Types | Continuous, binary, ordinal, categorical? | Infer from name | | Sample size | How many observations? | 500 | | Treatment | Is there a treatment variable? How many arms? | — | | Effect size | Expected treatment effect (Cohen's d, OR, etc.)? | 0.3 (small-medium) | | Correlations | Which variables should be correlated? How strongly? | — | | Clustering | Are observations nested (e.g., students in classrooms)? | No | | Panel structure | Multiple time periods? How many? | Cross-section | | Missing data | Should the data include realistic missingness? | No | | Language | R or Python? | Detect from project or ask |

For From design mode, extract most parameters from the design document automatically and confirm with the user.

For Calibrated mode, require the user to provide published summary statistics. Read references/calibration-targets.md for the calibration procedure.

Step 3: Generate Script

Read references/dgp-recipes.md for code patterns matching the requested design.

Read shared/multi-language-conventions.md for language-specific code style.

Generate a self-contained script that:

1. Sets a random seed (document the seed value)
2. Defines the data generating process with clear comments
3. Generates the dataset
4. Adds realistic noise and distributional features
5. Introduces missing data patterns if requested
6. Saves the dataset as CSV (and optionally .rds/.parquet)
7. Prints a summary of the generated data

Step 4: Execute

For large Monte Carlo sweeps (10k+ simulations, multi-condition grids, or long-running bootstrap): run on your HPC cluster instead of locally. Drop the generation script into hpc/ with submit.sbatch (compute partition) or sweep.sbatch (array over seeds/conditions) — templates at Task Management templates/slurm/, guide at [docs/guides/hpc.md](../../docs/guides/hpc.md). All SLURM templates log git-sha.txt to OUT_DIR so synthetic datasets remain traceable to the DGP code version.

Run the script and verify:

• Dataset has the expected dimensions
• Variable types are correct
• Treatment/control groups are balanced (if applicable)
• Summary statistics are plausible
• No degenerate columns (all zeros, all missing, zero variance)

Step 5: Save Output

Output routing:

| File | Location | |------|----------| | Generated dataset | data/synthetic/{name}.csv | | Generation script | code/generate_synthetic_{name}.R (or .py) | | Data dictionary | data/synthetic/{name}_dictionary.md |

NEVER write to data/raw/. The data-sensitivity rule applies -- synthetic data is not raw data and must be clearly separated.

Create data/synthetic/ if it does not exist.

Step 6: Data Dictionary

Produce a markdown data dictionary alongside the dataset:

# Data Dictionary: {name}

Generated: {date}
Script: `code/generate_synthetic_{name}.R`
Seed: {seed}
N: {sample_size}

| Variable | Type | Description | Distribution | Parameters |
|----------|------|-------------|-------------- |------------|
| id | integer | Unique identifier | Sequential | 1 to N |
| treatment | binary | Treatment assignment | Bernoulli | p = 0.5 |
| outcome | continuous | Primary outcome | Normal | mu = 0, sigma = 1 (control) |
| ... | ... | ... | ... | ... |

## Treatment Effects
- True ATE: {value}
- True CATE by subgroup: {if applicable}

## Missing Data
- Mechanism: {MCAR/MAR/MNAR}
- Rate: {percentage} of {variable}

---

Key Principles

Reproducibility

• Always set random seeds. Document the seed in the script header and the data dictionary.
• Script must be fully self-contained -- running it again with the same seed produces identical output.

Realism

• Match variable types to realistic distributions. Not everything is normal:
• Income, reaction times --> log-normal or gamma
• Likert scales --> ordinal with floor/ceiling effects
• Count data --> Poisson or negative binomial
• Proportions --> beta
• Binary outcomes --> Bernoulli with realistic base rates
• Include realistic noise. Treatment effects should be embedded in noisy data, not clean signal.
• Add heterogeneity. Real data has heterogeneous treatment effects -- consider adding subgroup variation.

Missing Data

• If the real data will have missingness, simulate it:
• MCAR: Random deletion at a fixed rate
• MAR: Missingness depends on observed variables (e.g., higher income --> less missing)
• MNAR: Missingness depends on the missing value itself (e.g., depressed people less likely to respond)
• Document the missingness mechanism in the data dictionary.

Calibration (Calibrated Mode)

• Read references/calibration-targets.md for the full procedure.
• Require published summary statistics as input (means, SDs, correlations, effect sizes).
• Validate that the synthetic data matches the targets within tolerance.
• Report discrepancies between target and achieved statistics.

---

Mode: From Design

Workflow

1. Locate design document -- check docs/experiment-design.md, docs/pre-analysis-plan.md, log/plans/
2. Extract parameters:

• Treatment arms and assignment mechanism
• Primary and secondary outcomes (types, expected distributions)
• Covariates and their roles
• Sample size and any clustering/stratification
• Expected effect sizes from the power analysis

1. Confirm with user -- present extracted parameters and ask for adjustments
2. Generate -- follow Steps 3-6 above

Integration with /experiment-design

The design document produced by /experiment-design Design mode contains everything needed:

• Hypotheses with expected signs --> treatment effect directions
• Conditions and randomization --> treatment assignment DGP
• Outcome measures and scales --> variable types and distributions
• Power analysis results --> sample size and effect size

---

Mode: From Schema

Workflow

1. Interview -- ask user to describe each variable (or provide a sketch)
2. Infer structure -- detect relationships from the description:

• "X causes Y" --> include causal pathway
• "A and B are correlated" --> induce correlation via shared latent factor
• "C moderates the effect" --> include interaction term

1. Generate -- follow Steps 3-6 above

Common Requests

| User says | Interpret as | |-----------|-------------| | "A standard survey dataset" | Demographics + Likert DVs + treatment + attention checks | | "Panel data" | Units x time with unit and time fixed effects | | "Something like [paper name]" | Switch to Calibrated mode, use paper's descriptives |

---

Mode: Calibrated

Workflow

1. Collect targets -- user provides published summary statistics:

• Means and standard deviations
• Correlation matrix (or key correlations)
• Effect sizes from regression tables
• Sample size
• Distribution shapes (skew, kurtosis) if reported

1. Read references/calibration-targets.md for the calibration procedure
2. Generate with matching -- use the Cholesky decomposition or copula method to match the target correlation structure
3. Validate -- compare synthetic descriptives to targets, report match quality
4. Save -- follow Steps 5-6 above, include calibration targets in the data dictionary

---

Cross-References

| Resource | When read | |----------|-----------| | references/dgp-recipes.md | All modes (code patterns for each design type) | | references/calibration-targets.md | Calibrated mode (matching procedure) | | shared/multi-language-conventions.md | All modes (code style) | | /experiment-design skill | From design mode (reads its design documents) | | /data-analysis skill | Consumes synthetic data for pipeline testing | | data-sensitivity rule | Never write to data/raw/ | | design-before-results rule | Synthetic data supports locking the design before real data |

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: flonat
• Source: flonat/claude-research
• License: MIT

Install and usage instructions live in the source repository linked above.