# Llm Finetuning

> Use when fine-tuning a large language model. Covers choosing full vs LoRA/QLoRA, dataset formatting, the transformers/PEFT/TRL stack, key hyperparameters, and evaluating fine-tunes without overfitting.

- **Type:** Skill
- **Install:** `agentstack add skill-param087-agent-ml-skills-llm-finetuning`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [param087](https://agentstack.voostack.com/s/param087)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [param087](https://github.com/param087)
- **Source:** https://github.com/param087/agent-ml-skills/tree/main/skills/llm-finetuning

## Install

```sh
agentstack add skill-param087-agent-ml-skills-llm-finetuning
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# LLM Fine-Tuning

## Overview

Fine-tuning adapts a base LLM to a task or style. For almost all practitioners the right default is **parameter-efficient fine-tuning (LoRA/QLoRA)** — it trains  quantity. A few thousand clean, diverse examples beat a noisy 100k dump.

```python
def to_chat(example):
    return {"messages": [
        {"role": "system", "content": "You are a helpful support agent."},
        {"role": "user", "content": example["question"]},
        {"role": "assistant", "content": example["answer"]},
    ]}
```

## QLoRA with TRL

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig
from trl import SFTTrainer, SFTConfig
import torch

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")

peft_cfg = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, bias="none",
                      task_type="CAUSAL_LM",
                      target_modules=["q_proj", "k_proj", "v_proj", "o_proj"])

trainer = SFTTrainer(
    model=model, train_dataset=train_ds, eval_dataset=eval_ds, peft_config=peft_cfg,
    args=SFTConfig(per_device_train_batch_size=2, gradient_accumulation_steps=8,
                   learning_rate=2e-4, num_train_epochs=2, warmup_ratio=0.03,
                   lr_scheduler_type="cosine", bf16=True, logging_steps=10,
                   eval_strategy="steps", eval_steps=100, output_dir="out"),
)
trainer.train()
```

## Key hyperparameters

- **LR:** 1e-4 to 3e-4 for LoRA (higher than full FT).
- **Epochs:** 1-3 — more overfits and degrades general ability.
- **LoRA rank `r`:** 8-32 typical; raise for harder tasks.
- **Effective batch:** use gradient accumulation to reach 16-32.

## Evaluating a fine-tune

- Hold out a real test set; track eval loss for overfitting.
- Judge on **task metrics** (exact match, rubric/LLM-judge), not just loss.
- Check for **regression** on general benchmarks — fine-tuning can break unrelated abilities.

## Pitfalls

- Too many epochs / LR too high → overfit, repetitive or degenerate outputs.
- Wrong chat template → the model learns malformed formatting.
- Tiny/dirty dataset → memorization, not generalization.
- Skipping a base-model baseline → you can't prove the fine-tune helped.

## Hand-off

Merged or adapter weights + eval report, logged via experiment-tracking and deployed via model-serving.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [param087](https://github.com/param087)
- **Source:** [param087/agent-ml-skills](https://github.com/param087/agent-ml-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-param087-agent-ml-skills-llm-finetuning
- Seller: https://agentstack.voostack.com/s/param087
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.