# Simulation Failure Triage

> >

- **Type:** Skill
- **Install:** `agentstack add skill-heshamfs-materials-simulation-skills-simulation-failure-triage`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [HeshamFS](https://agentstack.voostack.com/s/heshamfs)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [HeshamFS](https://github.com/HeshamFS)
- **Source:** https://github.com/HeshamFS/materials-simulation-skills/tree/main/skills/robustness/simulation-failure-triage

## Install

```sh
agentstack add skill-heshamfs-materials-simulation-skills-simulation-failure-triage
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Simulation Failure Triage

## Goal

Classify common simulation failure signatures and return immediate actions, retry ladders, and stop conditions.

## Requirements

- Python 3.10+
- No external dependencies
- Works on Linux, macOS, and Windows

## Inputs to Gather

| Input | Description | Example |
|-------|-------------|---------|
| Code | Simulation code | `LAMMPS`, `VASP`, `MOOSE`, `QE` |
| Stage | Setup, runtime, postprocess | `runtime` |
| Symptoms | Failure signs | `nan,pressure-blowup` |
| Log text or file | Error evidence | `Lost atoms`, `ZBRENT` |
| Recent change | Last modified setting | `larger timestep` |

## Decision Guidance

- First preserve evidence: logs, inputs, executable version, and scheduler output.
- Separate setup errors from numerical instability and physical model issues.
- Retry with a single controlled change.
- Stop retrying when the result becomes scientifically meaningless or a required model input is missing.

## Script Outputs

`scripts/failure_triage.py` emits:

- `likely_causes`
- `immediate_actions`
- `retry_ladder`
- `stop_conditions`
- `evidence`

## Workflow

```bash
python3 skills/robustness/simulation-failure-triage/scripts/failure_triage.py \
  --code LAMMPS \
  --stage runtime \
  --symptoms nan,pressure-blowup \
  --recent-change "increased timestep" \
  --json
```

## Error Handling

Invalid stages or oversized log files stop with exit code 2. Unknown symptoms are retained as custom evidence.

## Limitations

This skill gives first-response triage. It does not guarantee that a failed simulation can be repaired.

## Verification checklist

- [ ] Recorded the preserved-evidence set BEFORE any rerun: copied logs, input decks, scheduler output, and the executable/version string (per the first `immediate_actions` item) so the original failure is reproducible.
- [ ] Quoted the FIRST warning/error from the run, not just the final crash line, and confirmed `evidence.log_excerpt` (first 500 chars, original casing) captures it; re-run with `--log-file`/`--log-text` if the excerpt misses the root signature.
- [ ] Logged every `likely_causes` entry with its `symptom`, `category`, and `first_action`, and verified no real signature landed in the `category: custom` bucket because its keyword was absent from `LOG_HINTS`/`PATTERNS`.
- [ ] Confirmed the failure category is correct rather than mislabeled: a segfault/SIGSEGV/signal 11/core-dumped maps to `crash` (not `corrupted-output`), and an OOM/`bad_alloc`/`oom-kill` maps to `out-of-memory` (not `incomplete-run`).
- [ ] Applied the `retry_ladder` strictly one change at a time, recording the single parameter changed and its effect at each rung (including the memory rung when memory-bound).
- [ ] Checked every `stop_conditions` entry and halted the retry loop instead of stacking arbitrary stabilizing tweaks when any condition is met (e.g. unverifiable potential/pseudopotential, results dependent on arbitrary stabilizers).
- [ ] After any numerical recovery, handed off to the `simulation-validator` skill for conservation / physical-bounds / "can I trust these results" checks rather than declaring success here.

## Common pitfalls & rationalizations

| Tempting shortcut | Why it's wrong / what to do |
|-------------------|-----------------------------|
| "The last line of the log is the error, so I'll fix that." | The final crash line is often a downstream symptom; the script's second `immediate_actions` item exists because the FIRST warning/error is the real cause. Capture it in `evidence.log_excerpt` and triage from there. |
| "It segfaulted, so it's a corrupted-output / disk issue." | A `crash` (SIGSEGV/signal 11/core dumped) is a memory-fault category pointing at out-of-bounds/null-pointer/ABI bugs and a gdb/valgrind/ASan repro — not I/O. Do not treat it as `corrupted-output`. |
| "The job got killed, so I'll just bump the walltime." | `killed` alone is genuinely ambiguous and stays `incomplete-run`; an OOM signature (`bad_alloc`, `oom-kill`, "out of memory") is a distinct `out-of-memory` cause needing memory reduction / `--mem` increase, not more walltime. |
| "I'll lower dt AND change the preconditioner AND adjust the barostat to get it running." | The retry ladder is one controlled change per rung; stacking fixes makes results "depend on arbitrary stabilizing changes" — an explicit `stop_conditions` trigger. Change one parameter, record its effect, then proceed. |
| "I added the missing PAIR coeff path and it ran, so the physics is fine." | A clean run after fixing `missing-potential`/`bad-pseudopotential` resolves setup, not validity; species mapping, valence, or functional may still be wrong. Run completion is not correctness. |
| "It runs now, triage is done." | Triage only restores execution. Conservation, physical bounds, and convergence are out of scope here — defer to `simulation-validator` before trusting any number. |

## Security

### Input Validation

- `--stage` is checked against a fixed allowlist (`setup`, `runtime`, `postprocess`, `unknown`); any other value exits with code 2.
- `--code` must be non-empty and at most 100 characters; an empty or oversized value exits with code 2.
- `--symptoms` are split on commas and capped at 50 entries, each at most 100 characters; exceeding either limit exits with code 2.
- Unknown symptoms are not rejected; they are retained as `custom` evidence rather than silently dropped.
- There are no numeric arguments, so no finite/positive checks apply.
- All input-validation failures (and log read errors) are caught and exit with code 2 and a stderr message.

### File Access

- The only file read is the optional `--log-file` path; when omitted, the script reads no files.
- Before reading, the log file's size is checked via `stat`; if it exceeds the 10 MB cap (`MAX_LOG_SIZE`) the script raises and exits with code 2. Both file content and `--log-text` are then truncated to 10 MB.
- The script writes no files; all results go to stdout (JSON with `--json`, otherwise plain text).
- Path-sandboxing caveat: the log path is not restricted to a working directory, so the script can read any file the invoking process has permission to read; only pass trusted paths.

### Tool Restrictions

- `allowed-tools` is `Read, Bash, Write, Grep, Glob`.
- `Bash` runs only the bundled `scripts/failure_triage.py`.
- `Read` opens the skill's own files and a user-provided log to inspect failure evidence; `Grep`/`Glob` locate log files, inputs, and prior output; `Write` records triage notes or preserved evidence at the user's direction. The bundled script itself does not write files.

### Safety Measures

- No `eval`/`exec` and no dynamic code execution; log text is treated as data and never run.
- The script spawns no subprocesses and runs no external solvers.
- Output is structured JSON (`--json`), making results machine-checkable.
- DoS caps bound resource use: 10 MB log size/text limit, 50 symptoms, and 100-character limits on `code` and each symptom.

## References

- See `references/failure_patterns.md` for common failure signatures and retry ladders.

## Version History

- 1.1.3: Add `Verification checklist` (evidence-based checks tied to `failure_triage.py` outputs — preserved-evidence set, first-error excerpt, category-mislabel guards, one-change retry ladder, stop conditions, simulation-validator handoff) and `Common pitfalls & rationalizations` table covering last-line-only triage, segfault-as-I/O, OOM-vs-walltime, stacked stabilizing changes, and "it ran so it's valid".
- 1.1.1: Strengthen evals to be discriminating — each case now pins the exact `failure_triage.py` JSON output via `script_checks` (specific `category`/`first_action` strings, log-hint symptom inference, fixed retry-ladder/stop-condition text, recent-change immediate action), so the suite measures the skill's actual output rather than generic triage knowledge.
- 1.1.0: Add dedicated `out-of-memory` and `crash` (segfault) classifications; segfaults and OOM are no longer mislabeled as I/O or interrupted-execution. Preserve original-case log excerpt. Add input-validation caps (symptoms, code length). Sharpen description to defer deep validation to simulation-validator. Fix `missing-potential` eval token.
- 1.0.0: Initial cross-code simulation failure triage skill.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [HeshamFS](https://github.com/HeshamFS)
- **Source:** [HeshamFS/materials-simulation-skills](https://github.com/HeshamFS/materials-simulation-skills)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-heshamfs-materials-simulation-skills-simulation-failure-triage
- Seller: https://agentstack.voostack.com/s/heshamfs
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.