# Autoresearch

> Self-improving optimization via Karpathy autoresearch pattern. Generates → evaluates → scores → mutates prompts/descriptions in a loop. Targets — tool-selection, system-prompt, skill, decision-parser. Use when "optimize tools", "autoresearch", "improve skill X", "self-improve prompts", "optimize tool descriptions".

- **Type:** Skill
- **Install:** `agentstack add skill-fortunto2-rust-code-autoresearch`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [fortunto2](https://agentstack.voostack.com/s/fortunto2)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [fortunto2](https://github.com/fortunto2)
- **Source:** https://github.com/fortunto2/rust-code/tree/master/skills/autoresearch

## Install

```sh
agentstack add skill-fortunto2-rust-code-autoresearch
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# /autoresearch — Self-Improving Prompt Optimization

Karpathy autoresearch pattern: **Generate → Evaluate → Score → Keep/Discard → Mutate → Repeat**

## Architecture

Core engine: `crates/sgr-agent/src/autoresearch.rs` (Rust module, feature `genai`).
Dashboard: `skills/autoresearch/dashboard.py` (Python, standalone viewer).
Test cases: `skills/autoresearch/test_cases/*.json` (embedded at compile time via `include_str!`).

## Targets

| Target | What it optimizes | Eval metric |
|--------|------------------|-------------|
| `tool-selection` | Tool descriptions in agent | % correct tool chosen for 66 test tasks |
| `system-prompt` | Agent system prompt (SOUL.md) | Decision quality: right tool + coherent reasoning (4 criteria) |
| `skill` | Any SKILL.md file | Binary criteria pass rate (4 criteria) |
| `decision-parser` | Structured output schema | Parse success + field quality (5 criteria) |

## Usage (Rust API)

```rust
use sgr_agent::autoresearch::{AutoResearch, Config, Target};

let config = Config {
    target: Target::ToolSelection,
    batch_size: 10,
    cycle_secs: 120,
    gen_model: "gemini-2.5-flash".into(),
    eval_model: "claude-sonnet-4-6".into(),
    data_dir: "autoresearch_data/tool-selection".into(),
};
let ar = AutoResearch::new(config);
ar.run(20).await?; // 20 cycles
```

## Quick Start (CLI — when wired into rc-cli)

```bash
# Tool selection optimization (most impactful)
cargo run -- autoresearch tool-selection --cycles 20

# System prompt optimization
cargo run -- autoresearch system-prompt --cycles 10

# Optimize a specific skill
cargo run -- autoresearch skill --name delegate --cycles 15

# Optimize decision parsing
cargo run -- autoresearch decision-parser --cycles 10

# Dashboard (Python, standalone)
python3 skills/autoresearch/dashboard.py --target tool-selection --port 8501
```

## How It Works

Each cycle:
1. **Generate** N outputs with current prompt (via Gemini — same model agent uses)
2. **Evaluate** each output against binary criteria (via Claude Sonnet — different model for objectivity)
3. **Score** = sum of passed criteria across all outputs
4. **Keep** if score > best_score, **discard** otherwise
5. **Mutate** the winning prompt to try improvements
6. **Log** to JSONL for dashboard tracking
7. **Wait** for next cycle (default 2 min)

## Environment

- `GEMINI_API_KEY` — for generation (tests with same model the agent uses)
- `ANTHROPIC_API_KEY` — for evaluation and mutation (different model for objectivity)

## Tips

- **Binary evals** — yes/no criteria work best. Avoid Likert scales.
- **Don't over-constrain** — too many narrow criteria → model games the eval.
- **10-20 cycles** usually enough for significant improvement.
- **Apply results** — best prompt saved to `data//best_prompt.txt`.

## Cost

- Tool selection: ~$0.05/cycle (cheap — one LLM call per test, string match eval)
- System prompt: ~$0.20/cycle (gen + LLM eval per test)
- Skill: ~$0.30/cycle (gen + LLM eval per test, longer outputs)
- Decision parser: ~$0.15/cycle (gen + partial local eval)
- 20 cycles ≈ $1-6 depending on target. Permanent improvement.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [fortunto2](https://github.com/fortunto2)
- **Source:** [fortunto2/rust-code](https://github.com/fortunto2/rust-code)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-fortunto2-rust-code-autoresearch
- Seller: https://agentstack.voostack.com/s/fortunto2
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.