Processed Data Export

Processed Data Export Skill

Purpose

This skill creates plotting-ready processed data from raw research data.

The standard output is a matched pair:

processed/
├── zth_curve.csv
└── zth_curve.yaml

The CSV is for Python, MATLAB, Origin, or other plotting programs. The YAML records provenance: source files, source datasets, processing script, creation time, column meanings, units, intended use, and processing history.

This skill does not draw figures and does not modify raw data.

When to Use

Use this skill when the user asks to:

• extract selected datasets from raw HDF5 into CSV
• generate processed plotting data from raw data
• export curves such as time-temperature, Zth, IV, power, voltage, or current traces
• create YAML metadata for a processed CSV
• make plotting data traceable to a raw HDF5 file
• convert an existing processing script to write standardized CSV + YAML
• prepare tidy data for multi-curve plotting

Typical requests:

• "从 raw HDF5 里导出画图用 CSV"
• "把 /data/times 和 /data/temperatureK 提取出来"
• "生成 zth_curve.csv 和对应 metadata"
• "把这个处理脚本改成输出 processed CSV + YAML"
• "保证 processed data 可追溯到原始 HDF5"

Scope Boundary

This skill is responsible for:

• reading raw source files, especially HDF5
• selecting and optionally transforming datasets
• writing processed CSV files
• writing paired YAML metadata files
• validating CSV/YAML consistency
• preserving traceability to raw data

This skill is not responsible for:

• plotting figures
• fitting physical models unless needed to produce an exported column
• changing numerical simulation algorithms
• modifying raw HDF5 files
• deleting or overwriting raw data

Never modify original raw files. Avoid overwriting processed files unless the user asks or the script has an explicit overwrite=True path.

Recommended Project Structure

Prefer:

project/
├── raw/
│   └── hdf5/
│       └── 2026-05-30_90nmMOSFET_pulse_10K_1mW_run001.h5
├── processed/
│   ├── zth_curve.csv
│   └── zth_curve.yaml
├── scripts/
│   └── process_zth.py
└── figures/

Input HDF5 Convention

Default input is an HDF5 file from raw/ or raw/hdf5/.

Common structure:

/
├── data/
│   ├── time_s
│   ├── temperature_K
│   ├── power_W
│   ├── voltage_V
│   └── current_A
└── metadata/

Always allow the user or project code to specify actual dataset paths, for example:

/data/time_s
/data/temperature_K
/data/power_W

CSV Rules

CSV files must:

• use the first row as column names
• use clear physical quantity names
• include units in column names when practical
• use snake_case
• avoid spaces
• contain only plotting-ready values, labels, and identifiers

Good column names:

time_s
time_us
temperature_K
delta_temperature_K
power_W
zth_K_per_W
voltage_V
current_A
device
condition
run_id

Avoid vague names:

T
x
data
result
temp
new

Tidy Data Preference

For multi-curve plotting, default to tidy long-form data unless the user explicitly requests wide-form output.

Recommended:

time_us,zth_K_per_W,device,condition,run_id
0.1,0.02,90nmMOSFET,10K_1mW,run001
1.0,0.05,90nmMOSFET,10K_1mW,run001
0.1,0.01,FinFET,10K_1mW,run002
1.0,0.03,FinFET,10K_1mW,run002

Avoid unless requested:

time_us,mosfet_zth,finfet_zth
0.1,0.02,0.01
1.0,0.05,0.03

YAML Metadata Rules

Every processed CSV must have a same-stem YAML file:

zth_curve.csv
zth_curve.yaml

The YAML must include at least:

name:
description:
created_at:
created_by:
source_files:
source_datasets:
processing_script:
processing_steps:
columns:
intended_use:
notes:

Each entry in columns should record:

• description
• unit
• source

Use source: computed for derived columns and an HDF5 dataset path for copied or transformed raw datasets.

Naming Rules

Name processed CSV files by figure or analysis purpose:

zth_curve.csv
peak_temperature.csv
structure_comparison.csv
duty_cycle_sweep.csv
foster_fit_parameters.csv
cauer_network_parameters.csv
temperature_profile.csv

If a date or condition is needed:

YYYY-MM-DD___.csv

Example:

2026-05-30_zth_curve_90nmMOSFET_10K_1mW.csv

Avoid:

data.csv
result.csv
new.csv
final.csv

Workflow

1. Identify the raw source file and required dataset paths.
2. Inspect HDF5 keys and metadata before assuming names.
3. Decide processed columns and units.
4. Prefer tidy long-form layout for multi-condition or multi-device data.
5. Implement or adapt a processing script using templates/export_processed_csv.py when useful.
6. Write CSV and YAML together.
7. Validate that CSV columns match YAML columns.
8. Confirm raw files were not modified.

Reusable Template

Use templates/export_processed_csv.py for deterministic CSV + YAML export utilities. It provides:

• read_hdf5_datasets
• export_processed_csv_with_metadata
• validate_processed_pair
• make_processed_name

For a minimal end-to-end HDF5 example, use:

examples/minimal_hdf5_to_csv_example.py

For a synthetic raw HDF5 generator, use:

examples/example_raw_data_generator.py

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Narroog
• Source: Narroog/research-skills
• License: MIT

Install and usage instructions live in the source repository linked above.