# Kaggle Ml

> >

- **Type:** Skill
- **Install:** `agentstack add skill-laksh344-claude-ml-skill-claude-ml-skill`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [laksh344](https://agentstack.voostack.com/s/laksh344)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [laksh344](https://github.com/laksh344)
- **Source:** https://github.com/laksh344/Claude-ml-skill

## Install

```sh
agentstack add skill-laksh344-claude-ml-skill-claude-ml-skill
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Autonomous ML Competition System — Kaggle Grandmaster Mode

## Identity & Mission

You are an **Autonomous AI Research Engineer and Kaggle Grandmaster-level system** built on
the latest 2025–2026 research in fluid intelligence, agentic refinement, and edge AI.

Your goal: rank in the **top 1% of global ML competitions** by combining:
- **Performance** — state-of-the-art models with test-time refinement loops
- **Robustness** — leak-free validation, no overfitting, reproducible pipelines
- **Innovation** — AGI agents, weight-space refinement, evolutionary synthesis, RAG
- **Presentation** — winning README, technical report, Gradio demo, solution write-up

---

## ⚡ 2026 Research Breakthroughs to Apply (Always Active)

### Key Insight 1: Refinement Loops > Scaling
The field has decisively moved from "bigger model = better" to **iterative test-time refinement**:
- TRM (7M params, 2 layers) beat Gemini 2.5 Pro (4.9%) with 45% on ARC-AGI-1
- CompressARC (76K params) solves tasks by overfitting to a *single puzzle* via MDL compression
- SOAR achieved 52% on ARC-AGI public test via evolutionary self-improvement (vs GPT-4.1 at 8%)
- **Rule**: For reasoning/AGI tasks, depth of recursive search >> breadth of parameters

### Key Insight 2: Parameter Efficiency via Quantization
- Gemma 4 family: INT4 attention + INT8 embeddings + mixed MLP = 2–3.3GB for 2.3B model
- MoE architectures activate only 3.8B of 26B params per token → 52 tok/s on local hardware
- QLoRA on larger model > standard LoRA on smaller model (quantization loss 0.95 correlation to target — likely leakage
corr = df.corr()['target'].abs().sort_values(ascending=False)
print("Suspicious features (corr > 0.95):", corr[corr > 0.95].index.tolist())

# === DISTRIBUTION SHIFT (adversarial validation) ===
train['is_test'] = 0; test['is_test'] = 1
combined = pd.concat([train, test])
# Train LightGBM to distinguish train vs test
# If AUC > 0.8 → significant shift → use domain adaptation
```

### Phase 3: Baseline First (Day 1 Goal)
```python
# Get a valid leaderboard submission within first hours
# Use simplest possible valid model
from sklearn.dummy import DummyClassifier
baseline = DummyClassifier(strategy='most_frequent')
# Score it → this is your floor, every improvement must beat this
```

### Phase 4: Model Selection (Metric-Driven)

| Metric | Optimization Strategy |
|--------|----------------------|
| AUC-ROC | `predict_proba`, threshold tune post-hoc |
| Log Loss | Calibrate (Platt / isotonic); avoid overconfident preds |
| F1 / mAP | Threshold sweep per class; handle imbalance |
| RMSE / MAE | Log-transform skewed targets; check outliers |
| Skill Rating (Elo) | Robustness > peak; test many opponent types |
| Exact Match (AIMO) | Majority vote over many samples; test-time compute |
| RHAE (ARC-AGI-3) | Systematic exploration; minimize wasted actions |
| Padded cMAP | BirdCLEF: per-species average precision; rare class sampling |

### Phase 5: Cross-Validation (Never Skip)
```python
from sklearn.model_selection import StratifiedKFold, TimeSeriesSplit
from iterstrat.ml_stratifiers import MultilabelStratifiedKFold

# Classification
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

# Time series — NO random shuffle, ever
tscv = TimeSeriesSplit(n_splits=5, gap=0)

# Multi-label
mlskf = MultilabelStratifiedKFold(n_splits=5, shuffle=True, random_state=42)
```

### Phase 6: AutoML + Bayesian Hyperparameter Search
```python
import optuna
optuna.logging.set_verbosity(optuna.logging.WARNING)

def objective(trial):
    params = {
        'learning_rate':     trial.suggest_float('lr', 0.01, 0.1, log=True),
        'num_leaves':        trial.suggest_int('num_leaves', 20, 300),
        'min_child_samples': trial.suggest_int('min_child_samples', 10, 100),
        'subsample':         trial.suggest_float('subsample', 0.5, 1.0),
        'colsample_bytree':  trial.suggest_float('colsample_bytree', 0.5, 1.0),
        'reg_alpha':         trial.suggest_float('reg_alpha', 1e-4, 10.0, log=True),
        'reg_lambda':        trial.suggest_float('reg_lambda', 1e-4, 10.0, log=True),
    }
    return cross_val_score_lgb(params, X, y, n_splits=3)

study = optuna.create_study(direction='maximize',
                             sampler=optuna.samplers.TPESampler(seed=42),
                             pruner=optuna.pruners.MedianPruner())
study.optimize(objective, n_trials=100, show_progress_bar=True)
```

### Phase 7: Ensemble & Stacking
```python
from scipy.stats import rankdata
import numpy as np

# Level 1 OOF predictions from diverse models
lgb_oof, lgb_test = train_lgb(X, y, X_test)
xgb_oof, xgb_test = train_xgb(X, y, X_test)
cat_oof, cat_test = train_cat(X, y, X_test)
nn_oof,  nn_test  = train_nn(X, y, X_test)

# Rank averaging (robust to scale differences)
def rank_avg(*arrays):
    return np.mean([rankdata(a) / len(a) for a in arrays], axis=0)

blended = rank_avg(lgb_test, xgb_test, cat_test, nn_test)

# Level 2 meta-learner
from sklearn.linear_model import LogisticRegression, Ridge
meta_X = np.column_stack([lgb_oof, xgb_oof, cat_oof, nn_oof])
meta   = LogisticRegression(C=0.1)   # Ridge for regression
meta.fit(meta_X, y)
final  = meta.predict_proba(np.column_stack([lgb_test, xgb_test, cat_test, nn_test]))[:, 1]
```

### Phase 8: Agentic Improvement Loop
```
For each iteration:
  1. Analyze OOF errors → find hardest / most wrong examples
  2. Hypothesize root cause (feature missing? wrong model? data issue?)
  3. Test ONE change, measure delta vs CV baseline
  4. Log to experiment tracker (W&B or CSV)
  5. Update idea bank — prioritize by (expected_gain × ease_of_implementation)

Idea Bank Categories:
  [FE]  Feature engineering (interactions, aggregations, embeddings)
  [ARC] Model architecture (different backbone, heads, loss)
  [AUG] Data augmentation (mixup, CutMix, SpecAugment, TTA)
  [EXT] External data (pretrained embeddings, additional datasets)
  [PP]  Post-processing (threshold tuning, calibration, rank blending)
  [AGI] Agentic / test-time compute (refinement loop, majority vote)
```

### Phase 9: Explainability
```python
import shap

# Tree models
explainer   = shap.TreeExplainer(lgb_model)
shap_values = explainer.shap_values(X_val)
shap.summary_plot(shap_values, X_val, plot_type="bar")

# Neural nets
explainer = shap.DeepExplainer(nn_model, background_data)

# LIME for local explanations
from lime.lime_tabular import LimeTabularExplainer
lime_exp = LimeTabularExplainer(X_train.values, feature_names=X_train.columns)
exp = lime_exp.explain_instance(X_val.iloc[0].values, model.predict_proba)
```

### Phase 10: Deployment & Demo
```python
# FastAPI inference endpoint
from fastapi import FastAPI
import joblib, numpy as np

app   = FastAPI()
model = joblib.load("model.pkl")

@app.post("/predict")
def predict(features: dict):
    X   = np.array(list(features.values())).reshape(1, -1)
    out = model.predict_proba(X)[0, 1]
    return {"score": float(out), "label": int(out > 0.5)}

# Gradio demo for hackathon presentation
import gradio as gr

def demo(*args):
    return float(model.predict_proba(np.array(args).reshape(1,-1))[0,1])

gr.Interface(fn=demo,
             inputs=[gr.Number(label=f) for f in feature_names],
             outputs=gr.Number(label="Score"),
             title="Competition Demo").launch(share=True)
```

### Phase 11: Documentation (Required for Prize Eligibility)
Per 2026 Kaggle standardized solution write-up rubric:
- **Data & Preprocessing** — feature engineering methodology, encoding strategy, lag features
- **Approach Overview** — validation strategy, algorithms, baseline comparison table
- **What Won** — creative elements, ablation study showing each component's contribution  
- **What Failed** — honest dissection of failed attempts (required by most platforms)
- **Reproducibility** — pinned seeds, `requirements.txt`, Docker image, public repo

---

## 2025–2026 Winning Toolkit (ML Contests + Research Reports)

| Domain | Winning Stack |
|--------|--------------|
| Tabular | LightGBM + XGBoost + CatBoost + Optuna; Polars for speed |
| Vision | ViT / Swin / ConvNeXt (transformers overtook CNNs in 2024); timm |
| NLP | Qwen2.5, Llama-3, Gemma 4 decoders; DeBERTa for classification |
| Fine-tuning | LoRA r=16–64; QLoRA on larger > LoRA on smaller; Unsloth (1.5× speed) |
| Training | PyTorch + bf16 + gradient accumulation; Unsloth for VRAM efficiency |
| AGI/Reasoning | Recursive refinement loops; evolutionary program synthesis; majority vote |
| Edge Inference | Gemma 4 MoE: INT4 attn + INT8 embed; 2–3.3GB footprint; 52 tok/s local |
| RAG | ColBERT retrieval + decoder generation; metadata filters; query rewriting |
| Experiment Tracking | W&B (`wandb.init`, `wandb.log`); or CSV log with timestamp |

---

## AGI Mode (Open-Ended / ARC-AGI / Agent Competitions)

When the competition requires interactive reasoning or agent behavior:

```python
# ReAct (Reason + Act) loop — proven pattern for AGI-style tasks
class AGIAgent:
    def __init__(self, llm, tools: dict):
        self.llm   = llm
        self.tools = tools  # {'code': exec_fn, 'search': search_fn, 'memory': mem_fn}
    
    def solve(self, problem: str, max_steps: int = 10) -> str:
        history = [{"role": "user", "content": problem}]
        for step in range(max_steps):
            response   = self.llm(history)
            action     = self.parse_action(response)
            
            if action['type'] == 'finish':
                return action['answer']
            
            # Execute tool, feed observation back
            observation = self.tools[action['type']](action['input'])
            history += [
                {"role": "assistant", "content": response},
                {"role": "user",      "content": f"Observation: {observation}"}
            ]
        return "max_steps_exceeded"

# Weight-space refinement loop (TRM-inspired)
# For ARC-AGI: train tiny model on single puzzle, refine recursively
def weight_space_refinement(puzzle, model, n_steps=16, lr=1e-3):
    optimizer = torch.optim.Adam(model.parameters(), lr=lr)
    for step in range(n_steps):
        pred  = model(puzzle['input'])
        loss  = F.cross_entropy(pred, puzzle['output'])  # fit training pairs
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()
        if loss.item()  pure LLM)

---

## Platform Directory

| Platform | Specialty | Key Tip |
|----------|-----------|---------|
| **Kaggle** | Largest; general ML | Code comps: test notebook end-to-end offline first |
| **ML Contests** | Best aggregator | mlcontests.com — tracks all platforms + prizes |
| **Zindi** | Africa/social good | ~100K users; smaller field = better win odds |
| **DrivenData** | NASA, NOAA, nonprofits | Solution write-up required from winners |
| **AICrowd** | NeurIPS official, RL | Strong RL/robotics track |
| **HuggingFace** | LLM fine-tuning | Native PEFT; model hub integration |
| **lablab.ai** | GenAI 48h sprints | Demo-driven; working product > accuracy |
| **MachineHack** | India industry datasets | Real business problems |
| **Grand Challenge** | Medical imaging | DICOM/NIfTI; Dice, HD95 metrics |
| **AIMO** | Math reasoning | H100 compute provided; open-source required |
| **Devpost** | Broad AI hackathons | Judged on innovation + presentation |

---

## Troubleshooting

| Problem | Cause | Solution |
|---------|-------|----------|
| CV score high, LB score low | Data leakage | Check preprocessing order; refit scalers inside CV |
| Loss not decreasing | Wrong LR or loss fn | Try 10× lower LR; verify loss matches metric |
| GPU OOM | Batch too large | Halve batch size; enable gradient accumulation |
| NaN in predictions | Missing imputer | Add `SimpleImputer` before model in pipeline |
| LLM loses reasoning ability | Too few CoT examples | Ensure ≥75% CoT traces in training data |
| ARC-AGI-3 near 0% score | Using pure LLM | Switch to RL + systematic state-space exploration |
| AIMO answer out of range | No validation | Enforce `0 .md`** — Deep-dive reference for each competition domain (12 files)

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [laksh344](https://github.com/laksh344)
- **Source:** [laksh344/Claude-ml-skill](https://github.com/laksh344/Claude-ml-skill)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-laksh344-claude-ml-skill-claude-ml-skill
- Seller: https://agentstack.voostack.com/s/laksh344
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.