# Trackio

> Configure trackio experiment monitoring for training runs, add alert callbacks, read alerts back between runs, and derive the next run's config from prior alerts. Use for any training job monitoring or when iterating on hyperparameters.

- **Type:** Skill
- **Install:** `agentstack add skill-boeschj-ml-intern-plugin-trackio`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [boeschj](https://agentstack.voostack.com/s/boeschj)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [boeschj](https://github.com/boeschj)
- **Source:** https://github.com/boeschj/ml-intern-plugin/tree/main/skills/trackio
- **Website:** https://github.com/boeschj/ml-intern-plugin#quickstart

## Install

```sh
agentstack add skill-boeschj-ml-intern-plugin-trackio
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

Trackio is natively integrated with Transformers Trainer and all TRL trainers; the built-in TrackioCallback handles init, log, and finish. In TrainingArguments, SFTConfig, DPOConfig, or GRPOConfig set:

```python
report_to="trackio"
run_name="sft_qwen3-4b_lr2e-5_bs128"
project=""
trackio_space_id="/ml-intern-a1b2c3d4"
```

Replace the bracketed values with real ones before the script leaves your hands: the namespace comes from `hf-whoami --brief`, the project groups related runs for comparison, and the space id suffix is any 8 characters. `TRACKIO_PROJECT` and `TRACKIO_SPACE_ID` env vars work too. Always give the user the dashboard URL: `https://huggingface.co/spaces/`.

## Alerts drive iteration

Use `trackio.alert(title, text, level)` at every decision point in training. Levels:
- ERROR: stop and change approach (divergence, NaN, OOM)
- WARN: tweak hyperparameters (overfitting, early stopping, KL spike, reward collapse, slow convergence)
- INFO: milestones (training complete, target reached, checkpoint saved)

Always include numeric values and an actionable suggestion in the text, for example "loss=12.4 at step 200, lr likely too high, try x0.1". A future run must be able to parse it and act.

Wire alerts through a custom TrainerCallback passed via `callbacks=[...]`: `on_log` for training metrics (loss, reward, kl), `on_evaluate` for eval metrics (only available there). Keep each condition simple: one metric, one threshold.

## Reading alerts back

Read alerts between runs instead of parsing thousands of metric points. CLI, always with --json:

```
trackio get alerts --project  --run  --json
trackio get alerts --project  --since  --json
trackio get run    --project  --run  --json
trackio get metric --project  --run  --metric  --json
trackio list runs  --project  --json
```

Python: `api = trackio.Api(); api.alerts(p, run=r, since=ts); api.runs(p)` where each run carries `.name`, `.config`, `.alerts()`.

The plugin's job monitor watches job states and failure logs for you; read alerts with the CLI between runs and react to ERROR alerts immediately instead of waiting for the job to finish.

## Next config from prior alerts

- diverged: lr x 0.1
- overfitting: weight_decay x 10 or reduce capacity
- early stopping: lr x 0.5 or adjust the schedule
- high accuracy: refine around the current config

Read the prior config via `api.runs(...).config` and only mutate keys the alerts justify changing.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [boeschj](https://github.com/boeschj)
- **Source:** [boeschj/ml-intern-plugin](https://github.com/boeschj/ml-intern-plugin)
- **License:** Apache-2.0
- **Homepage:** https://github.com/boeschj/ml-intern-plugin#quickstart

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-boeschj-ml-intern-plugin-trackio
- Seller: https://agentstack.voostack.com/s/boeschj
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.