Trackio

Trackio is natively integrated with Transformers Trainer and all TRL trainers; the built-in TrackioCallback handles init, log, and finish. In TrainingArguments, SFTConfig, DPOConfig, or GRPOConfig set:

report_to="trackio"
run_name="sft_qwen3-4b_lr2e-5_bs128"
project=""
trackio_space_id="/ml-intern-a1b2c3d4"

Replace the bracketed values with real ones before the script leaves your hands: the namespace comes from hf-whoami --brief, the project groups related runs for comparison, and the space id suffix is any 8 characters. TRACKIO_PROJECT and TRACKIO_SPACE_ID env vars work too. Always give the user the dashboard URL: https://huggingface.co/spaces/.

Alerts drive iteration

Use trackio.alert(title, text, level) at every decision point in training. Levels:

• ERROR: stop and change approach (divergence, NaN, OOM)
• WARN: tweak hyperparameters (overfitting, early stopping, KL spike, reward collapse, slow convergence)
• INFO: milestones (training complete, target reached, checkpoint saved)

Always include numeric values and an actionable suggestion in the text, for example "loss=12.4 at step 200, lr likely too high, try x0.1". A future run must be able to parse it and act.

Wire alerts through a custom TrainerCallback passed via callbacks=[...]: on_log for training metrics (loss, reward, kl), on_evaluate for eval metrics (only available there). Keep each condition simple: one metric, one threshold.

Reading alerts back

Read alerts between runs instead of parsing thousands of metric points. CLI, always with --json:

trackio get alerts --project  --run  --json
trackio get alerts --project  --since  --json
trackio get run    --project  --run  --json
trackio get metric --project  --run  --metric  --json
trackio list runs  --project  --json

Python: api = trackio.Api(); api.alerts(p, run=r, since=ts); api.runs(p) where each run carries .name, .config, .alerts().

The plugin's job monitor watches job states and failure logs for you; read alerts with the CLI between runs and react to ERROR alerts immediately instead of waiting for the job to finish.

Next config from prior alerts

• diverged: lr x 0.1
• overfitting: weight_decay x 10 or reduce capacity
• early stopping: lr x 0.5 or adjust the schedule
• high accuracy: refine around the current config

Read the prior config via api.runs(...).config and only mutate keys the alerts justify changing.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: boeschj
• Source: boeschj/ml-intern-plugin
• License: Apache-2.0
• Homepage: https://github.com/boeschj/ml-intern-plugin#quickstart

Install and usage instructions live in the source repository linked above.