# Kaggloop Submit > Stage 4 of the kaggloop win-loop — finalize the round by ensembling the kept models, passing the strict data-leakage gate (enforced before any submission), submitting to Kaggle via the API — the standard ensemble AND the round's challenge-track submission (the dual-submission mandate, enforced at stage close) — recording the public leaderboard scores, comparing the actual score to the target, run… - **Type:** Skill - **Install:** `agentstack add skill-qurore-kaggloop-kaggloop-submit` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [qurore](https://agentstack.voostack.com/s/qurore) - **Installs:** 0 - **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [qurore](https://github.com/qurore) - **Source:** https://github.com/qurore/kaggloop/tree/main/.claude/skills/kaggloop-submit ## Install ```sh agentstack add skill-qurore-kaggloop-kaggloop-submit ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # Stage 4 — Submit (gate → ensemble → submit → study the gap → decide) Convert this round's kept models into the strongest valid submission, **pass the leakage gate** (the submission guard will block you otherwise), push to Kaggle, record what the leaderboard says, then **compare to the target and study the gap** to decide the next move. **Both of this round's two submissions must attempt a NEW improvement — never a defensive resubmission of a past best (the two-way-door principle).** Submission **#1 (primary)** is this loop's **highest-confidence new improvement** — the kept, gap-closing bets applied on top of the current best, a genuine measured step forward, *not* a prior iteration's model re-submitted to guard the score. Submission **#2 (challenge)** is the low-confidence, high-variance home-run swing (step 5b). Because a submission is a **reversible door** — every earlier iteration's models and submissions are kept, Kaggle holds two final-selection slots, and a worse LB this round never erases a better earlier one — a bet that fails just means you **roll back and restart from the previous iteration**. The downside is bounded and undoable, so **always be aggressive: neither submission may be a score-protecting rehash.** ## Preconditions - `kept` hypotheses with OOF + test predictions under `experiments/results/`. `python -m kloop.project set --stage submit --status running`. A submission-format reference from the dossier. ## Judged competitions (no CSV — the judge-rubric gate, enforced) — READ FIRST If survey set `scoring_mode` = `judged` / `hybrid` (a human-scored **Writeup**; no leaderboard), the tabular flow below (ensemble → leakage gate → `kaggle submit` CSV) does **not** apply. Instead: 1. **Assemble the final deliverable** — the ≤word-limit **Kaggle Writeup** + its required attachments (media/figures, and for an agent challenge the agent/deck it documents), matching the Submission Requirements exactly. 2. **Judge-rubric gate (enforced — replaces the leakage gate).** Run a final **blind, adversarial** judging pass over the assembled deliverable against `judge_rubric.json`; write `judge/iter_.json` (per-sub-criterion scores + quoted evidence + weighted total). Record the realized score and study the gap on it: ```bash python -m kloop.project set --best-lb --best-submission \ --note "iter judged=/100" python -m kloop.project gap --log # target(rubric) vs judged actual — the compass python -m kloop.journal log --kind gate --decision "judge-rubric gate: judged=/100" \ --rationale "scored vs fixed anchors; weakest crit =; evidence in judge/iter_.json" ``` **Do not finalize** without a primary-source `judge_rubric.json` **and** a fresh `judge/iter_.json` for this iteration — that is the enforced quality gate for judged comps. 2b. **The challenge deliverable (the dual-submission mandate, judged form).** Judged rounds ship a challenge too: the challenge-track bet becomes a **bold, interdisciplinary variant or extension of the deliverable** (a section/demo/artifact importing a mechanism from a foreign field). Judge it blind against the same rubric, write `judge/iter__challenge.json`, and journal `--kind challenge_submission` (or `--kind challenge_deferred` with the hard blocker) — the stage will not close without one of the two. The better-scoring variant becomes the writeup candidate and `--best-lb` carries its total. 3. **The human submits the Writeup on the Kaggle website** (New Writeup → attach assets → pick a Track → **Submit**) before the deadline — the one manual gate (like scout). There is no `kaggle submit` CSV and `guard_submission` does not fire; **never** fabricate a leaderboard number for a judged comp. 4. **Then continue at 6b/6c/7 below** — write the mandatory **iteration journal** (predicted vs judged, the per-criterion gap + its cause with cited sources, next plan), run the **results-driven self-improvement pass** (the judged rubric total recorded via `--best-lb` feeds `kloop.selfimprove check` exactly like a leaderboard score), and make the loop decision on the judged gap: **by default loop back to `hypothesize`** on the weakest-weighted criteria (never stop to ask), finalizing only when the target rubric score is met or a hard external limit is hit (deadline / explicit user stop / user-set `KLOOP_MAX_ITERATIONS` cap). ## Procedure 1. **Assemble OOF/test predictions** from the kept models (ingest any pending Colab results first: `python -m kloop.colab ingest`). 2. **Build the ensemble**, validated on the **same dossier CV** — the blend must beat the best single model on local CV, or ship the single model: ```bash python -m kloop.score blend results//artifacts/oof.npy results//artifacts/oof.npy \ --weights 0.6 0.4 --out experiments/blend_oof.npy --metric --y-true code/y_true.npy ``` For >2 models prefer `kloop.score.greedy_blend` (Caruana-style). Apply the chosen weights to the **test** predictions to make the submission CSV. Sanity-check it against `sample_submission` (columns, row count, id order, ranges, no NaNs). Journal it: ```bash python -m kloop.journal log --kind ensemble --decision "blend h0003*0.6 + h0007*0.4" \ --rationale "blend CV 0.849 > best single 0.846; decorrelated" --evidence "experiments/blend_oof.npy" ``` 3. **Pass the leakage gate (mandatory — the guard enforces it).** Run the automated checks on the *final* ensemble, affirm the checklist, and verify: ```bash python -m kloop.gate check --task --train-ids code/train_ids.npy --test-ids code/test_ids.npy \ --oof experiments/blend_oof.npy --y-true code/y_true.npy --x code/X.npy \ --groups code/groups.npy --folds code/folds.npy python -m kloop.gate affirm --confirm fit_on_train_only,oof_target_encoding,no_future_info,no_banned_external,cv_matches_split,no_test_in_train python -m kloop.gate verify # writes gate.json passed:true; without it, submit is BLOCKED python -m kloop.journal log --kind gate --decision "leakage gate passed" --rationale "no fails; checklist affirmed" ``` 4. **Respect submission budget.** The daily cap lives in project state (`max_daily_submissions` — set during survey via `python -m kloop.kaggle limits --save`); count today's used slots with `python -m kloop.kaggle submissions `. A round ships **two** submissions — the main ensemble (step 5) plus the challenge submission (step 5b) — so it wants two remaining slots; the **main sub always goes first** when the budget is tight. Keep two final-selection slots in mind for the competition's end. 5. **Submit and log it:** ```bash python -m kloop.kaggle submit -f submissions/.csv -m "iter: blend cv=" \ --watch # polls until scored; prints scoring_seconds python -m kloop.project set --best-lb --best-submission submissions/.csv \ --note "iter LB= (cv=)" python -m kloop.journal log --kind submission --decision "submitted .csv" \ --rationale "cv=, expected ~target" --evidence "submissions/leaderboard.jsonl" ``` (Append `{file, cv, lb, message, track: "standard", scoring_seconds, ts}` to `submissions/leaderboard.jsonl`. `scoring_seconds` — how long Kaggle took to score the submission — comes from the `--watch` output (or `python -m kloop.kaggle watch ` after the fact); poll-interval accuracy is fine, and `null` when the transition wasn't observed — e.g. a code-comp rerun scored overnight.) 5b. **The challenge submission (mandatory second submission — enforced at stage close).** Every round also ships the **challenge-track** artifact verified in `/kaggloop-experiment` (the interdisciplinary breakthrough bet — `kloop.ledger list` marks it `CH`). After the main submission is in, re-run the leakage gate on the challenge artifacts (`gate check` on its OOF/preds → `affirm` → `verify` — it is a different model; its gate run must be its own), then submit and journal it: ```bash python -m kloop.kaggle submit -f submissions/_challenge.csv \ -m "iter CHALLENGE: cv=" --watch # prints scoring_seconds too python -m kloop.journal log --kind challenge_submission \ --decision "challenge sub LB= (main LB=)" \ --rationale "" \ --evidence "submissions/leaderboard.jsonl" ``` Append it to `submissions/leaderboard.jsonl` with `track: "challenge"` (+ its `scoring_seconds`, as in step 5). `--best-lb` / `--best-submission` take **whichever of the two submissions scored better** — when the challenge sub wins, the leapfrog worked: promote it to next round's standard baseline. Only a **hard blocker** — zero remaining daily submissions, a gate-failing / structurally broken challenge artifact (rejected in the ledger with the reason), the deadline — may skip it, and that skip must be journaled honestly: ```bash python -m kloop.journal log --kind challenge_deferred --rationale "" \ --decision "challenge submission deferred" ``` `kloop.project set` **refuses to close this stage without one of the two records** (`challenge_submission` or `challenge_deferred`) for this iteration — never game the deferral: "CV was worse than the main sub" is NOT a blocker (upside variance is the point). 6. **Study the gap (the core of the loop).** Compare actual to target and analyze *why*: ```bash python -m kloop.project gap --log # appends target vs actual to progress.jsonl python -m kloop.standing snapshot --note "iter: " # append our score vs the live # medal landscape (top score + gold/silver/bronze cutoff scores, our rank & medal) to # projects//standing.jsonl — one stacked record per iteration. Pass --name when # multiple projects run concurrently (current_project is a shared cache). ``` Read the standing: how far is our realized score from the **bronze/silver/gold lines** and from `top`? Did LB move with CV? Is the gap from underfitting, a CV↔LB mismatch (shake-up / leakage / distribution shift), or a metric/post-processing miss? **Compare the two tracks:** did the challenge submission beat the main one (leapfrog → promote it to next round's baseline), land close (the mechanism has signal → sharpen it next round), or crater (retire it and pick a fresh challenge axis)? **And check the public floor:** is our realized score still below the best public notebook's Public Score (the synced top-5 — `python -m kloop.notebooks list` + `recon.md`)? Below the floor, the gap analysis must explain *why we underperform code anyone can fork*, and closing to that baseline (adapt it — it's local) is next loop's #1 bet before any exotic idea. **Never overfit to the public LB** — trust CV. Journal the analysis (required to close the stage): ```bash python -m kloop.journal log --kind gap_analysis \ --decision "gap remains; likely " --rationale "" ``` 6b. **Write the iteration learning journal (MANDATORY — the meta-learning loop).** Every iteration must leave an explicit, human-readable retrospective MD at `projects//iterations/iter__.md` (zero-padded, one per submit-cycle). The **next** iteration's `/kaggloop-hypothesize` reads the last ≤5 of these first, so write it to be *useful to your future self*: honest, specific, and grounded. Required sections (in order): ```markdown # iter — · · version/sub: · LB: ## What was done # the approach + exact config/knobs changed vs last iter (be concrete) ## Predicted score # the number you expected BEFORE submitting + how you derived it ## Actual score # the real LB (traceable to leaderboard.jsonl / Kaggle); "blank COMPLETE"⇒verify it's not a failure ## Gap # predicted−actual AND target−actual; was the prediction right? ## Gap investigation # WHY the gap — verified against real resources: the synced top-5 # notebooks (projects//notebooks/), discussions, the science # MCP (arxiv/semantic-scholar), the SDK source, a local harness # repro. Cite each. No hand-waving. Include: above or below the # best public notebook's score, and why. ## Challenge track # the 2nd (challenge) submission: the bet, its LB vs the main sub, # verdict (leapfrog → new baseline / signal → sharpen / retire); # or the journaled challenge_deferred hard blocker ## Next iteration — plan & resolve # the concrete plan for what to try/investigate next, and why ``` Fill every section from real evidence (a predicted-vs-actual number with no derivation, or a cause with no cited source, is a failed journal). This file — not memory — is how the loop compounds learning across iterations. 6c. **Pipeline self-improvement (results-driven — the CHECK runs every loop, the EDIT only on real improvement).** The pipeline upgrades itself, but strictly on results-ism: only a *realized* score improvement can trigger edits. After `gap --log` (and the iteration journal), run: ```bash python -m kloop.selfimprove check # pass --name when multiple projects run concurrently ``` - **`improved: false` → touch nothing.** Log the skip and move on: `python -m kloop.selfimprove log --action no_improvement --analysis ""`. If the *previous* loop's entry (`kloop.selfimprove list`) shows pipeline edits and this round regressed, treat them as regression suspects: restore the prior content (`git diff` / `git checkout -- `, or Edit back) and log `--action reverted`. - **`improved: true` (especially `significant: true`) → success retrospective first.** From the ledger, `experiments/results/` and the journal, identify *which bet/lever caused the delta* (cite the evidence) and add a short "What worked & why" note to this iteration's journal (6b). Then ask: **is there a generalizable *process* lesson** — something that would help *any* competition, not just this one? Competition-specific tricks stay in `recon.md` / the journal, never in the shared pipeline. - **If a generalizable lesson exists, edit the pipeline directly** — `.claude/skills/**`, `.claude/hooks/**`, `CLAUDE.md` — via the Edit/Write tools (pre-authorized; no approval prompt). Read `kloop.selfimprove list` first so you never silently re-apply an idea a past loop reverted. Keep diffs small and surgical; preserve each SKILL.md's frontmatter. **Invariants you may never weaken:** the scout human gate, `guard_submission`'s gate-before-submit enforcement, the leakage / judge-rubric gate requirements, journal append-only enforcement, autopilot bounds, "never fabricate scores". - **After editing any hook:** `python -m kloop.selfimprove hookcheck` must pass (syntax + smoke-run of every hook). A broken hook is worse than no improvement — restore immediately if it fails. New hooks need `settings.json` wiring, which is out of self-edit scope: propose that to the human via the journal instead. - **Log the outcome — mandatory every loop, whatever happened:** ```bash python -m kloop.selfimprove log --action improved_and_changed \ --analysis "" --files "" \ --rationale "" --delta --gap-closed-frac python -m kloop.journal log --kind self_improve --decision "" \ --rationale "" ``` (`--action improved_no_change` when the score improved but nothing generalizable emerged.) 7. **Loop decision** (journaled as `loop_decision`): - **Target met** (`python -m kloop.project gap` shows `target_met: true`): finalize. ```bash python -m kloop.project set --stage submit --status done \ … ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [qurore](https://github.com/qurore) - **Source:** [qurore/kaggloop](https://github.com/qurore/kaggloop) - **License:** MIT Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** no - **Filesystem access:** no - **Shell / process execution:** no - **Environment & secrets:** yes - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-qurore-kaggloop-kaggloop-submit - Seller: https://agentstack.voostack.com/s/qurore - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.