Ralph

/ralph — Autonomous Fresh-Context Loop

Ralph is an autonomous multi-model iteration loop that applies the fresh-context pattern to idea generation. Each iteration independently selects a model (weighted by benchmarks), applies a creative divergence lens, calls the LLM, scores and deduplicates the output, and persists everything to disk. The loop self-terminates when output quality saturates or budget limits are hit.

Named after the Ralph Wiggum Loop pattern: instead of managing growing context, discard it entirely and restart fresh each iteration. State lives on disk, not in any model's context window.

---

Table of Contents

• [Quick Start](#quick-start)
• [Usage](#usage)
• [How It Works](#how-it-works)
• [Architecture](#architecture)
• [Configuration](#configuration)
• [Creative Lenses](#creative-lenses)
• [Model Pool](#model-pool)
• [Output](#output)
• [State Files](#state-files)
• [Script Reference](#script-reference)
• [Exit Conditions](#exit-conditions)
• [Failure Handling](#failure-handling)
• [Prerequisites](#prerequisites)
• [Design Decisions](#design-decisions)
• [Examples](#examples)

---

Step 0: Model Configuration Prompt

Before starting the loop, check whether the user's message already specifies model preferences (e.g., "use opus only", "exclude kimi", "low reasoning effort"). If it does, apply those preferences directly and skip this prompt. If it does NOT, present the following using AskUserQuestion:

Model configuration for /ralph:

MODEL POOL (choose one):
  1. auto-weighted — all available models, weighted by domain benchmarks (default)
  2. opus-only    — every iteration uses opus
  3. custom       — specify which models to include/exclude

Available models: run `python3 ~/.claude/skills/llm/scripts/llm_route.py --list-models` to see what's configured

REASONING EFFORT (optional — press Enter for defaults):
  Claude (opus):      thinking budget → [16k tokens (default) / 32k / 64k / 128k]
  ChatGPT (5.4/5.2):  reasoning_effort → [xhigh (default) / high / medium / low]

CONTEXT WINDOW (optional — press Enter for defaults):
  [default / specify tokens / auto (let model_selector decide per iteration)]

Enter choice (e.g. "1", "opus-only", "custom: include=opus,chatgpt-5.4 effort=high"):

Parsing the response:

• If "auto-weighted" or "1" → use default preferred_models from ralph-config.json
• If "opus-only" → set preferred_models to ["opus"] for this session
• If "custom" → parse include/exclude list, update preferred_models and excluded_models for this session
• If reasoning effort specified → pass as override to session_manager.py calls (e.g., --thinking-budget 32000 for opus, --reasoning-effort high for ChatGPT)
• If context window specified → apply as runtime override; if "auto", let model_selector.py choose based on prompt size
• If user presses Enter or says "defaults" → use existing ralph-config.json settings unchanged

---

Quick Start

# Run with defaults — idea-generation preset, academic domain, up to 200 iterations
/ralph

# Cap at 10 iterations (good for testing)
/ralph --iterations 10

# Run directly via shell
bash ~/.claude/skills/ralph/scripts/ralph.sh idea-generation 10

# Analytics mode — SaaS funnel discovery with SQL execution
bash ~/.claude/skills/ralph/scripts/ralph-analytics.sh 15

The loop will:

1. Select a random model (weighted by academic benchmarks)
2. Build a prompt with a random creative lens
3. Call the model and parse the structured idea
4. Score, deduplicate, and store the idea
5. Repeat until saturation, budget limit, or max iterations

On completion, a session report prints to stdout and all ideas are in state//ideas-bank.json.

---

Usage

Start a New Session

# Slash command (via Claude Code)
/ralph
/ralph --iterations 20
/ralph --preset idea-generation --domain academic

# Direct shell invocation
bash ~/.claude/skills/ralph/scripts/ralph.sh  [max_iterations]
bash ~/.claude/skills/ralph/scripts/ralph.sh idea-generation 50

Arguments:

| Argument | Default | Description | |----------|---------|-------------| | preset | idea-generation | Preset name from settings/presets/ | | max_iterations | 200 (from config) | Override the maximum iteration count | | --resume | — | Resume a previous session by ID |

Resume a Previous Session

bash ~/.claude/skills/ralph/scripts/ralph.sh idea-generation --resume ralph-20260225-230341

Resumes from the last completed iteration. All state files are preserved and appended to.

Sync Benchmarks

python3 ~/.claude/skills/ralph/scripts/benchmark_sync.py          # sync if stale (>24h)
python3 ~/.claude/skills/ralph/scripts/benchmark_sync.py --force   # force re-fetch

Refreshes the benchmark rankings CSV from upstream sources. Runs automatically at session start; this command is for manual refresh.

View a Session Report

python3 ~/.claude/skills/ralph/scripts/session_manager.py \
    --report --session  --state-dir ~/.claude/skills/ralph/state/

Prints iteration stats, model usage breakdown, and top-scored ideas.

---

How It Works

Each iteration is a completely independent subprocess pipeline. No state crosses between iterations via memory — only via JSON files on disk.

┌─────────────────────────────────────────────────────────────┐
│                      ralph.sh (bash)                        │
│                                                             │
│  for each iteration:                                        │
│                                                             │
│    [1] model_selector.py   → selects model (stdout)         │
│    [2] prompt_builder.py   → writes prompt + system files   │
│    [3] session_manager.py  → calls LLM, writes result JSON  │
│    [4] idea_evaluator.py   → scores + dedup, updates bank   │
│    [5] memory_indexer.py   → extracts learnings to memory   │
│    [6] circuit_breaker.py  → records success/failure         │
│    [7] exit_evaluator.py   → CONTINUE | SATURATION | LIMIT  │
│                                                             │
│  if exit_evaluator != CONTINUE: break                       │
│                                                             │
│  session_manager.py --report (final summary)                │
└─────────────────────────────────────────────────────────────┘

Scripts communicate exclusively through:

• JSON files on disk (state directory + temp files)
• stdout (single-line outputs: model name, exit decision, eval summary)
• Exit codes (0 = success, 1 = error)

This means every script is independently runnable and testable.

---

Architecture

Directory Structure

~/.claude/skills/ralph/
├── SKILL.md                           # This file
├── scripts/
│   ├── ralph.sh                       # Bash orchestrator — idea generation (7-step)
│   ├── ralph-analytics.sh             # Bash orchestrator — analytics discovery (9-step)
│   ├── session_manager.py             # Session init, LLM calls, logging, reports
│   ├── circuit_breaker.py             # 3-state per-model circuit breaker
│   ├── model_selector.py              # Benchmark-weighted stochastic selection
│   ├── prompt_builder.py              # Prompt assembly + creative lenses
│   ├── exit_evaluator.py              # Saturation detection + budget limits
│   ├── idea_evaluator.py              # Novelty/feasibility scoring + Jaccard dedup
│   ├── memory_indexer.py              # Cross-iteration learning (4-type taxonomy)
│   └── benchmark_sync.py              # Benchmark data freshness check + refresh
├── settings/
│   ├── ralph-config.json              # Main configuration
│   ├── benchmark-profiles.json        # Symlink → debate-agent's domain weights
│   └── presets/
│       └── idea-generation.json       # Default preset
├── references/
│   ├── creative-lenses.yaml           # 20 curated lenses
│   └── architecture.md                # Design rationale document
└── state/                             # Created at runtime, one dir per session
    └── /
        ├── session.json
        ├── iterations.jsonl
        ├── ideas-bank.json
        ├── memory.json
        └── circuit-state.json

Infrastructure Reuse

Ralph does not duplicate LLM infrastructure. It imports from sibling skills:

| Dependency | Source | How | |-----------|--------|-----| | Model calling | /llm skill's llm_route.py | sys.path.insert() import in session_manager.py | | Prompt wrapping | /llm skill's llm_route.call_model() | Auto-applies XML/CTCO/plain per model | | Benchmark data | /llm skill's rankings.csv | 1,071 models, 32 columns, 5 sources | | Model registry | /llm skill's model-registry.json | 11 model configs with routes + reasoning params | | Domain weights | Debate agent's benchmark-profiles.json | Symlinked; 7 domains with per-model weights | | API keys | Auto-discovered by llm_route.get_api_key() | Checks env vars, .env files, provider configs |

---

Configuration

Main Config: settings/ralph-config.json

{
  "loop": {
    "max_iterations": 200,          // Hard cap on iterations
    "max_runtime_hours": 8,         // Wall-clock time limit
    "cooldown_between_iterations_seconds": 5
  },
  "exit_gate": {
    "saturation_window": 10,        // Ideas per comparison window
    "saturation_threshold": 0.15,   // Min improvement to keep going
    "min_unique_ideas": 5,          // Don't check saturation until this many
    "hard_limit_iterations": 200,
    "hard_limit_hours": 8
  },
  "circuit_breaker": {
    "failure_threshold": 3,         // Failures before model is skipped
    "cooldown_seconds": 300,        // Seconds before retry (doubles on re-fail)
    "half_open_max_attempts": 1
  },
  "model_selection": {
    "domain": "academic",           // Domain for benchmark weights
    "exploration_weight": 0.3,      // Recency penalty strength (0=none, 1=max)
    "recency_penalty_window": 5,    // How many recent iterations to penalize
    "excluded_models": ["aristotle", "glm-5"],
    "preferred_models": ["opus", "gemini-3.1-pro", "gpt-5.2", "chatgpt-5.4", "kimi-2.5"]
  },
  "idea_evaluation": {
    "novelty_weight": 0.55,         // Weight for novelty in combined score
    "feasibility_weight": 0.45,     // Weight for feasibility
    "dedup_jaccard_threshold": 0.6, // Similarity threshold for duplicate detection
    "min_key_terms": 5              // Min terms needed for dedup comparison
  },
  "llm_call": {
    "max_tokens": 4096,
    "timeout": 600,                 // 10-minute timeout per LLM call
    "system_prompt_max_length": 8000
  }
}

Presets: settings/presets/.json

Presets define the domain, prompt template, and output parsing for a specific use case. The included idea-generation.json preset targets quantitative finance dissertation ideas.

Preset structure:

| Field | Description | |-------|-------------| | domain | Academic domain (drives benchmark weight lookup) | | subdomain | Specific subfield | | context.dissertation_topic | Research area description | | context.advisor_preferences | What makes a good idea | | context.constraints | Hard constraints (timeline, data, contribution) | | prompt_template.role | System prompt role definition | | prompt_template.task | User prompt with output JSON schema | | prompt_template.creative_lens_instruction | Template for lens injection ({lens} placeholder) | | output_parsing.required_fields | Fields the LLM must return |

To create a new preset: Copy idea-generation.json, change the domain/topic/constraints, and save as settings/presets/.json. Then run with:

bash ralph.sh your-preset 20

---

Creative Lenses

Each iteration applies one of 20 curated creative lenses — divergence-forcing constraints that push the model to approach the problem from an unusual angle. Lenses are selected stochastically with recency avoidance (a lens used in the last 5 iterations is deprioritized; if all are exhausted, the least-recently-used is selected).

| # | Lens | Cognitive Shift | |---|------|----------------| | 1 | Inversion | Deliberately do the opposite of conventional wisdom | | 2 | Cross-Pollination | Import technique from an alien field | | 3 | Failure Mode | Design around an assumption that will break | | 4 | Scale Shift | Study a phenomenon at a radically different scale | | 5 | Data-First | Start from an unusual, underexploited data source | | 6 | Adversarial | Account for intelligent, adaptive counterparties | | 7 | Radical Simplicity | Find the simplest model that captures 80% | | 8 | Regime-Aware | Explicitly handle regime changes | | 9 | Network/Graph | Model the system as a network | | 10 | Information Asymmetry | Focus on who knows what and when | | 11 | Temporal Structure | Challenge standard temporal assumptions | | 12 | Behavioral Finance | Build quantitative models from cognitive biases | | 13 | Market Microstructure | Focus on mechanics of trade execution | | 14 | Synthetic Data | Simulation-based inference as primary methodology | | 15 | Tail Risk | Extreme events from reliability engineering | | 16 | Multi-Asset | Cross asset class boundaries | | 17 | Regulatory Arbitrage | Model unintended consequences of regulation | | 18 | ML Interpretability | Interpretability over prediction accuracy | | 19 | Climate/ESG | Physical risk pricing, not ESG scoring | | 20 | DeFi/Decentralized | Apply classical theory to DeFi protocols |

Lenses are defined in references/creative-lenses.yaml. To add a new lens, append an entry with id, name, and prompt fields.

---

Model Pool

Models are selected stochastically, weighted by academic domain benchmark scores from benchmark-profiles.json:

| Model | Route | Reasoning | Approximate Weight | |-------|-------|-----------|-------------------| | opus | Claude CLI | Inherent extended thinking | 0.90 | | gemini-3.1-pro | Google API | thinkingLevel=HIGH | 1.00 | | gpt-5.2 | Codex CLI | reasoning_effort=xhigh | 0.80 | | chatgpt-5.4 | Codex CLI | reasoning_effort=xhigh | 0.55 | | kimi-2.5 | Kimi CLI | --thinking | 0.65 |

Excluded by default: aristotle (theorem prover, not a general LLM), glm-5 (returns empty responses via OpenRouter).

Selection algorithm:

1. Start with preferred models from config
2. Remove excluded models
3. Remove circuit-OPEN models (HALF_OPEN remain eligible)
4. Look up domain weight for each remaining model
5. Apply recency penalty: for each occurrence in last N iterations, multiply weight by (1 - exploration_weight)
6. Floor all weights at 1e-6 (no model reaches zero probability)
7. Normalize to sum to 1.0
8. Sample via random.choices()

---

Output

Ideas Bank: state//ideas-bank.json

The primary output. Each idea contains:

{
  "idea_id": "idea-001",
  "source_model": "kimi-2.5",
  "iteration": 1,
  "title": "Higher-Order Contagion in Multi-Asset Derivatives Networks...",
  "research_question": "How do higher-order interdependencies...",
  "abstract": "This dissertation develops a hypergraph framework...",
  "methodology": "Hypergraph theory, stochastic volatility models...",
  "key_mechanisms": ["mechanism1", "mechanism2", "mechanism3"],
  "novelty_claim": "First application of hypergraph spectral theory...",
  "key_references": ["Billio et al. (2012) - ...", "..."],
  "feasibility_notes": "Public OCC data provides...",
  "novelty_score": 0.85,
  "feasibility_score": 0.72,
  "combined_score": 0.79,
  "is_duplicate": false,
  "key_terms": ["hypergraph", "contagion", "derivatives", "..."]
}

The stats section provides aggregate metrics:

{
  "stats": {
    "total": 50,
    "unique": 43,
    "duplicates": 7,
    "avg_combined_score": 0.76,
    "top3_ids": ["idea-012", "idea-031", "idea-007"]
  }
}

Session Report

Generated automatically at loop completion and available via --report:

========================

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [ZealousEar](https://github.com/ZealousEar)
- **Source:** [ZealousEar/claude-skills](https://github.com/ZealousEar/claude-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.