# Training Neural Networks > Use when training, fine-tuning, or evaluating any neural network or ML model, before writing a training loop, when a model trains but underperforms, or when a metric looks suspiciously good or suspiciously bad - **Type:** Skill - **Install:** `agentstack add skill-umaraslam66-ml-superpowers-training-neural-networks` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [Umaraslam66](https://agentstack.voostack.com/s/umaraslam66) - **Installs:** 0 - **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [Umaraslam66](https://github.com/Umaraslam66) - **Source:** https://github.com/Umaraslam66/ml-superpowers/tree/main/skills/training-neural-networks ## Install ```sh agentstack add skill-umaraslam66-ml-superpowers-training-neural-networks ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # Training Neural Networks ## Overview Two things are true about neural net training, and everything here follows from them. **1. It is a leaky abstraction.** `model.fit(data)` looks like an API. It is not. The abstraction leaks constantly, and training only works to the degree you understand what is underneath it. **2. It fails silently.** This is the one that hurts. Wrong code raises an exception. A wrong neural net trains to completion, prints a decreasing loss, and hands you a number. You permuted your labels. You forgot to `.zero_grad()`. You augmented at eval time. Your padding mask is inverted. **All of these train fine.** Everything is syntactically correct, the loss goes down, and the whole thing is silently, confidently wrong. **Core principle:** Be patient and paranoid. Build simple to complex. At every step, make a concrete prediction about what will happen, then verify it before moving on. **Violating the letter of this process is violating the spirit of it.** ## The Iron Law ``` NO UNVERIFIED COMPLEXITY. ADD ONE THING AT A TIME. VERIFY IT. THEN ADD THE NEXT. ``` If you wrote data loading, augmentation, the model, the scheduler, mixed precision, EMA, and logging before running anything — you have built a haystack and hidden an unknown number of needles in it. Delete down to the skeleton and build back up one verified piece at a time. ## When to Use Use for ANY model-training work: - Writing a training loop from scratch - Fine-tuning a pretrained model (LoRA, full FT, SFT, adapters) - Building or changing an eval harness - "The model trains but the results are bad" - "The results are much better than expected" - Reproducing a paper or a baseline - Anything where a loss curve exists **Use this ESPECIALLY when:** - You are about to launch a long or expensive run - The task "seems standard" (standard tasks have standard silent bugs) - Someone is waiting on a number - You already tried three hyperparameter changes and nothing moved **Don't skip when:** - It's "just a fine-tune" — fine-tunes fail silently too, and more often - You're using a trusted framework — trusted frameworks have footguns - You're in a hurry — the fast-and-furious approach to training does not work and only leads to suffering ## The Six Stages You MUST pass each stage's gate before starting the next. The gates are the whole point: they are the concrete hypotheses you validate. | # | Stage | Skill | Gate to pass | |---|-------|-------|--------------| | 1 | Become one with the data | `ml-superpowers:becoming-one-with-the-data` | You can predict which examples the model will get wrong, and why | | 2 | Skeleton + dumb baselines | `ml-superpowers:building-the-training-skeleton` | All sanity checks pass, including overfitting a single batch to ~0 loss | | 3 | Overfit | `ml-superpowers:overfitting-first` | A model large enough to drive training loss very low | | 4 | Regularize | `ml-superpowers:regularizing-a-model` | Validation loss improves while you give up training loss | | 5 | Tune | `ml-superpowers:tuning-hyperparameters` | Best config found by random (not grid) search | | 6 | Squeeze | `ml-superpowers:tuning-hyperparameters` | Ensembles and longer training applied last | Stage 2 is where nearly all silent bugs are caught. It is the stage agents skip. Do not skip it. ## Why the Order Is Non-Negotiable The most common catastrophic mistake is **regularizing before overfitting**. An agent adds dropout, augmentation, weight decay, and label smoothing to a model that has never once demonstrated it can fit the training set. Now underfitting and overfitting look identical from the outside: a mediocre number. There is no signal left to debug with, and every subsequent change is a guess. Overfit first. A model that cannot overfit a tiny training set has a **bug**, not a capacity problem, and no amount of regularization will fix a bug. The same logic gives you the rest of the order: - Look at data before building, or you will build the wrong thing - Get a skeleton and dumb baselines before a real model, or you will not know whether your number is good - Overfit before regularizing, or you cannot tell under- from over-fitting - Regularize before tuning, or you tune on a model that doesn't generalize - Squeeze last, because ensembles and long runs are expensive and hide bugs ## Red Flags — STOP If you catch yourself thinking or writing any of these, stop and return to the stage you skipped: - "Let me write the full training pipeline, then run it" - "Loss went down, so it's working" - "The accuracy is 94%, training successful" (94% vs *what* baseline?) - "Let me add augmentation and dropout to improve results" - "I'll tune the learning rate and see if that helps" - "The data looks fine" (did you *look* at it, or assume?) - "I'll just use the defaults" - "It's a standard dataset, no need to inspect it" - "Let me try a bigger model" (before overfitting a small one) - "Let me change the LR, the batch size, and add warmup" (three at once) - Reporting a metric you have not sanity-checked against chance **All of these mean: you are guessing. Go back to the gate you skipped.** ## Common Rationalizations | Excuse | Reality | |--------|---------| | "It's a standard task, the recipe is overkill" | Standard tasks have standard silent bugs. The recipe takes 20 minutes and catches them. | | "Loss is decreasing, it's fine" | Loss decreases for models with permuted labels too. Decreasing loss proves almost nothing. | | "I don't need a baseline, I know what's good" | Then state the number *before* you run. If you can't, you need the baseline. | | "Looking at data is not my job" | It is the highest-value hour in the whole project. Karpathy's words, not a suggestion. | | "Let me add regularization to improve val" | You cannot regularize a model that hasn't overfit. You're masking a bug. | | "The framework handles that" | Frameworks silently apply LR decay, dropout at eval, and label smoothing you didn't ask for. | | "I'll verify after it finishes training" | A 6-hour run that was misconfigured at step 0 wasted 6 hours. Verify in the first 60 seconds. | | "One more hyperparameter tweak" | 3+ failed tweaks means a bug, not a hyperparameter. Go back to stage 2. | | "It's a fine-tune, not real training" | Fine-tunes have *more* silent failure modes: template mismatch, truncation, masked-loss bugs. | ## Beyond From-Scratch Training The failure mode this defends against — silent misconfiguration producing a plausible number — is identical in modern work. The gates are the same; the instruments change. | Gate | Fine-tuning | Eval harness | LLM / RAG | |---|---|---|---| | See the data | Read 200 prompts *post-template* | Check train/test contamination | Read retrieved chunks, not queries | | Loss @ init | Match the base model's loss | Score a known-perfect answer | Answer with no context | | Input-independent | Shuffle labels; must get worse | Grade a constant answer | Retrieve random chunks | | Overfit one batch | 10 examples → proves the loss mask | Hand-grade 10; harness must agree | 5 cases end-to-end | Each stage skill expands its own row. ## Related Skills - `ml-superpowers:debugging-silent-training-failures` — when a run is already broken - `ml-superpowers:designing-ml-experiments` — is this difference real? - `ml-superpowers:choosing-what-to-fix` — what to work on next - `superpowers:systematic-debugging` — non-ML bugs found along the way ## The Bottom Line You are not trying to train a model. You are trying to **never be fooled by your own pipeline**. The model is the easy part. At the end you have a training and evaluation infrastructure you trust, and a series of results you *predicted before you saw them*. That last one is what separates an ML engineer from someone turning knobs. ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [Umaraslam66](https://github.com/Umaraslam66) - **Source:** [Umaraslam66/ml-superpowers](https://github.com/Umaraslam66/ml-superpowers) - **License:** MIT Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-umaraslam66-ml-superpowers-training-neural-networks - Seller: https://agentstack.voostack.com/s/umaraslam66 - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.