# Compare Models

> Compare 2+ YOLO models side-by-side on the same dataset — mAP, per-class AP, speed, size.

- **Type:** Skill
- **Install:** `agentstack add skill-macroman5-autotrain-yolo-compare-models`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [MacroMan5](https://agentstack.voostack.com/s/macroman5)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [MacroMan5](https://github.com/MacroMan5)
- **Source:** https://github.com/MacroMan5/autotrain-yolo/tree/master/.claude/skills/compare-models

## Install

```sh
agentstack add skill-macroman5-autotrain-yolo-compare-models
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Compare Models

Compare 2+ YOLO models on the same validation dataset. All models are evaluated with the same `data` and `imgsz` for a fair comparison.

## Inputs

- **Model paths**: 2 or more `.pt` file paths (required)
- **Dataset**: path to `data.yaml` (optional — falls back to `yolo-project.yaml` defaults.dataset)
- **imgsz**: image size (optional — falls back to `yolo-project.yaml` defaults.imgsz, then 640)

## Pre-checks

1. **Verify every model path exists** before running any validation. If any path is missing, stop and report which files were not found.
2. **Warn if >5 models** are requested — total validation time may be significant. Ask the user to confirm before proceeding.
3. **Resolve dataset and imgsz** from `yolo-project.yaml`:

```bash
python -c "
import yaml, pathlib
cfg_path = None
for p in [pathlib.Path('yolo-project.yaml')] + list(pathlib.Path('.').resolve().parents):
    candidate = p / 'yolo-project.yaml' if p.is_dir() else p
    if candidate.exists():
        cfg_path = candidate
        break
if cfg_path:
    cfg = yaml.safe_load(cfg_path.read_text())
    defaults = cfg.get('defaults', {})
    print(f\"dataset: {defaults.get('dataset', 'NOT SET')}\")
    print(f\"imgsz: {defaults.get('imgsz', 640)}\")
else:
    print('No yolo-project.yaml found')
"
```

If no dataset is provided and none is in the project config, stop and ask the user.

4. **Locate data.yaml**: The dataset path from the config points to a YOLO dataset directory. The `data.yaml` file is at `{dataset}/data.yaml`. Verify it exists.

## Validation

For each model, run validation with identical settings:

```bash
python -c "
import json, os
from ultralytics import YOLO

model_path = ''
data_yaml = ''
imgsz = 
device = 'cuda' if __import__('torch').cuda.is_available() else 'cpu'

model = YOLO(model_path)
results = model.val(data=data_yaml, imgsz=imgsz, device=device, verbose=False)

# File size in MB
size_mb = os.path.getsize(model_path) / (1024 * 1024)

# Inference speed (ms per image, preprocess + inference + postprocess)
speed = results.speed
total_ms = speed.get('preprocess', 0) + speed.get('inference', 0) + speed.get('postprocess', 0)

# Per-class AP50 values
class_names = results.names
per_class_ap50 = {}
per_class_ap50_95 = {}
if results.box.ap_class_index is not None:
    for i, cls_idx in enumerate(results.box.ap_class_index):
        name = class_names[int(cls_idx)]
        per_class_ap50[name] = round(float(results.box.ap50[i]), 4)
        per_class_ap50_95[name] = round(float(results.box.ap[i]), 4)

out = {
    'model': model_path,
    'mAP50': round(float(results.box.map50), 4),
    'mAP50-95': round(float(results.box.map), 4),
    'precision': round(float(results.box.mp), 4),
    'recall': round(float(results.box.mr), 4),
    'speed_ms': round(total_ms, 1),
    'size_mb': round(size_mb, 1),
    'per_class_ap50': per_class_ap50,
    'per_class_ap50_95': per_class_ap50_95,
}
print(json.dumps(out))
"
```

Collect the JSON output from each run.

## Output

### 1. Overall Comparison Table

Build a markdown table from the collected results:

```
| Model | mAP50 | mAP50-95 | Precision | Recall | Speed (ms) | Size (MB) |
|-------|-------|----------|-----------|--------|------------|-----------|
| ...   | ...   | ...      | ...       | ...    | ...        | ...       |
```

Sort by mAP50-95 descending. Bold the best value in each column.

### 2. Per-Class AP Delta Table (if >1 class)

Show per-class AP50 for each model. If exactly 2 models, add a delta column. If >2 models, show raw values and bold the best per class.

```
| Class | Model A | Model B | Delta |
|-------|---------|---------|-------|
| cat   | 0.92    | 0.88    | +0.04 |
| dog   | 0.85    | 0.91    | -0.06 |
```

### 3. Recommendations

State plainly:

- **Best overall**: Model with highest mAP50-95
- **Best per-class**: For each class, which model wins (only if models disagree)
- **Speed/accuracy tradeoff**: If a smaller/faster model is within 1-2% mAP of the best, call it out as a viable deployment option
- **Notable gaps**: Any class where a model drops >5% AP compared to the best — flag it

## Important

- All models MUST be validated on the same dataset with the same imgsz. Do not mix settings.
- Do not install anything. `ultralytics` and `torch` are already available.
- Print the comparison tables directly — do not write to a file unless the user asks.
- Keep commentary factual. No speculation about why a model is better — just report the numbers and let the user decide.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [MacroMan5](https://github.com/MacroMan5)
- **Source:** [MacroMan5/autotrain-yolo](https://github.com/MacroMan5/autotrain-yolo)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** yes
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-macroman5-autotrain-yolo-compare-models
- Seller: https://agentstack.voostack.com/s/macroman5
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.