Agentic Self Improvement

Agentic Self-Improvement Loop

Behavioral benchmarking + evolutionary self-improvement for Hermes Agent. Combines closed-loop guidance patching (from Hermes PR #6120) with the GEPA optimization framework from hermes-agent-self-evolution (NousResearch, MIT).

---

Core Loop

BENCHMARK → DIAGNOSE → PATCH → VERIFY → (auto-revert if regression)
                    ↓
              GEPA OPTIMIZE → EVAL DATASET → EVOLVE → DEPLOY
                    ↓
              SESSION MINE → LLM-AS-JUDGE → NEW BENCHMARKS

Two modes operate together:

• Guidance Patch Loop — fast, surgical, targets specific behavioral failures via guidance blocks
• Evolutionary Optimization Loop — principled, uses GEPA + DSPy to optimize skill text end-to-end

---

Part 1: Guidance Patch Loop

Core Loop

BENCHMARK → DIAGNOSE → PATCH → VERIFY → (auto-revert if regression)

Usage

# Run all benchmarks, show suggested patches (no changes made)
/self-improve

# Run all benchmarks, auto-apply if improvement, auto-revert if regression
/self-improve --mode=apply

# Run specific category only
/self-improve --benchmarks=mandatory_tool.yaml

# Run subset of categories
/self-improve --categories=mandatory_tool,act_dont_ask,no_hallucination

# Test a specific model
/self-improve --model=claude-sonnet-4

# Control parallelism (default: 4)
/self-improve --parallel=8

# View past results
/self-improve --view-results=2026-04-12_1400

# Revert last applied patch
/self-improve --revert

Options

| Flag | Default | Description | |------|---------|-------------| | --mode | suggest | suggest (show diff only) or apply (apply + verify) | | --benchmarks | all | Benchmark YAML file(s) to run | | --categories | all | Run only specific categories (by filename without .yaml) | | --model | current | Model to benchmark | | --parallel | 4 | Number of parallel prompt executions | | --view-results | none | Show a past run's results | | --revert | false | Revert the last applied patch |

Benchmark Categories

| Category | File | What it tests | |----------|------|---------------| | mandatory_tool | BENCHMARKS/mandatory_tool.yaml | Must use tools for math/hashes/time/files | | act_dont_ask | BENCHMARKS/act_dont_ask.yaml | Act on obvious interpretation, don't ask | | no_hallucination | BENCHMARKS/no_hallucination.yaml | Check system, don't answer from profile/memory | | verification | BENCHMARKS/verification.yaml | Verify outputs before finalizing | | prerequisite | BENCHMARKS/prerequisite.yaml | Discover before acting | | path_accuracy | BENCHMARKS/path_accuracy.yaml | Verify paths before using | | context_grounding | BENCHMARKS/context_grounding.yaml | Check actual context, don't assume | | auth_state | BENCHMARKS/auth_state.yaml | Verify credentials before using them | | remember_to_obsidian | BENCHMARKS/remember_to_obsidian.yaml | Write durable facts to Obsidian, not just memory tool |

How the Guidance Patch Loop Works

1. Benchmark Runner

Each prompt runs in isolation via hermes chat --model -q QUERY -Q as a subprocess. Captures:

• Tool calls made (detected as "preparing X..." text previews in quiet mode)
• Tool outputs
• Final response text
• Exit code

2. Failure Analyzer

Groups failures by category, computes per-category pass rates, extracts failure examples.

3. Patch Generator

Generates XML guidance blocks for the relevant model type:

• OpenAI/GPT/Codex → OPENAI_MODEL_EXECUTION_GUIDANCE
• Anthropic/Claude → ANTHROPIC_MODEL_EXECUTION_GUIDANCE
• Google/Gemini → GEMINI_MODEL_EXECUTION_GUIDANCE
• MiniMax → MINIMAX_MODEL_EXECUTION_GUIDANCE

4. Apply + Verify + Auto-Revert

Suggest mode: Shows the diff, writes to ~/.hermes/self-improvement/proposed_patches/.diff

Apply mode:

1. Run baseline benchmarks → capture pass rates
2. Backup prompt_builder.py to ~/.hermes/self-improvement/backups//
3. Apply patch to relevant GUIDANCE constant
4. Re-run benchmarks → compare pass rates
5. If ANY category regresses → auto-revert to backup
6. If all improve → keep patch, report delta

Output Locations

| What | Where | |------|-------| | Benchmark results | ~/.hermes/self-improvement/results// | | Proposed patches | ~/.hermes/self-improvement/proposed_patches/.diff | | Backups | ~/.hermes/self-improvement/backups// | | Logs | ~/.hermes/logs/ |

---

Part 2: Session Pattern Mining

Extract behavioral failure patterns from conversation history and convert them into concrete, actionable test cases.

When to Use

• Designing benchmarks for a self-improvement loop
• Finding recurring mistakes to address in skills or system prompts
• Building a behavioral test suite for an agent's capabilities
• Reviewing past sessions for systematic failures vs one-off errors

The Process

1. Search Broadly, Then Narrow

# Good starting queries:
"failure pattern mistake error"
"wrong incorrect mistake"
"assume guess hallucinate"
"context switch forgot lost"
"should I do ask clarification"
"file not found path error"
"not sure unknown do not know"
"directory path file location"
"behavior bias habit pattern"

2. Extract the Failure Narrative

For each relevant session:

• What was the task? (the user's request)
• What went wrong? (the failure)
• What was the root cause? (why it failed)
• What category does it belong to? (see Categories below)

## Session:  — 
### Task

### Failure

### Root Cause

### Category

3. Categorize Failures

| Category | What it means | Signs in session | |----------|---------------|------------------| | mandatory_tool | Model computed/guessed instead of using a tool | Mental math in response, hashes from training data | | act_dont_ask | Model asked for clarification on obvious cases | "Should I do X or Y?" when X was clearly correct | | no_hallucination | Model answered from memory/profile instead of checking | User profile substituted for system reality | | verification | Model finalized without checking output | Responded to empty pages, wrong files, failed commands | | prerequisite | Model started task without discovery steps | Tried to use a file/path without checking if it existed | | path_accuracy | Model assumed paths without verification | FileNotFoundError in logs, "does ~/.X exist?" when it didn't | | context_grounding | Model answered from wrong context | Wrong directory, wrong git branch, wrong session state | | auth_state | Model tried to use credentials/files without checking validity | 403/401 errors from unverified tokens |

4. Convert Failures to Benchmark Prompts

id: -  # e.g. mt-001, ada-001
prompt: 
validator:
  type: 
  detail: 
ground_truth: 
failure_example:
  what_happened: 
  session_id: 

Validator types:

• tool_used — model must call a specific tool
• output_contains — tool output must contain expected string
• refused — model must refuse to answer (not enough context)
• exit_code — command must exit with specific code
• error_in_output — model must report an error it detected
• command_contains — response or tool outputs contain specific strings
• non_empty_or_explicit_not_found — non-empty response or explicit not-found
• two_step / sequence — multi-step (tool use demonstrates it)

5. Group Into Benchmark Sets

Cluster prompts by category. A complete benchmark set should have:

• 4-10 prompts per category
• Varied phrasings (don't make the expected action obvious)
• At least one negative example (task that should NOT use a tool)

Session Search Tips

1. Search for the fix, not the failure — sessions that end with "fixed" often document what broke first
2. Check session summaries first — session_search returns LLM-generated summaries that surface failures efficiently
3. Look at truncated sessions — sessions that were "cut off" mid-investigation often reveal root causes
4. Cross-reference with logs — ~/.hermes/logs/errors.log often has the raw error that sessions later explained

Output Location

Save mined patterns to /03-Notes/session-patterns/.md so they're preserved in long-term memory alongside other session research.

---

Part 3: Evolutionary Optimization (from hermes-agent-self-evolution)

The full GEPA + DSPy optimization framework from NousResearch/hermes-agent-self-evolution.

What Can Be Optimized — The Tier System

| Tier | Target | Risk | Status | |------|--------|------|--------| | Tier 1 | Skill files (SKILL.md) | Low | Highest value, lowest risk | | Tier 2 | Tool descriptions | Low-Medium | Medium value, schema must stay frozen | | Tier 3 | System prompt sections | High | High value, must stay within 20% size budget | | Tier 4 | Tool implementation code | Highest | Darwinian Evolver, strictest guardrails |

Tier 1 is the primary target for self-improvement — skills are pure text, easily mutated, directly measurable.

The Optimization Loop (GEPA Framework)

1. SELECT TARGET
   - Pick a skill, prompt section, or tool
   - Load current version as baseline

2. BUILD EVALUATION DATASET
   - Mine SessionDB for real usage examples
   - Or use session-pattern-mining to extract from transcripts
   - Or generate synthetic test cases
   - Split: train / validation / test

3. RUN OPTIMIZER
   - Primary: GEPA (Genetic-Pareto Prompt Evolution)
   - Fallback: DSPy MIPROv2 (Bayesian optimization)
   - Generates mutations, evaluates, selects best

4. EVALUATE & COMPARE
   - Run optimized version on held-out test set
   - Compare: accuracy, cost, latency
   - Statistical significance check

5. DEPLOY (with approval)
   - Git commit the improved version
   - Rollback mechanism via git revert

Evaluation Dataset Sources (in priority order)

Source A: SessionDB mining (real usage, LLM-as-judge scored)

• Query SessionDB for sessions where the skill was loaded
• Extract the task the user gave and the agent's full response
• Use LLM-as-judge to score each (task, response) pair on a rubric
• High-scoring pairs → "good" examples; low-scoring → failure cases for GEPA

Source B: Synthetic generation (bootstrapping)

• Use a strong model (Claude Opus) to generate 15-30 test cases
• Read the skill file → understand what it does
• Generate (taskinput, expectedbehavior) pairs
• GEPA works with as few as 3 examples, so start small

Source C: Hand-curated golden sets

• Manually written test cases with expected outputs
• Highest quality signal but requires manual effort
• Reserve for critical skills

LLM-as-Judge Scoring (Multi-Dimensional Rubrics)

For most skills, there's no binary right/wrong — quality is subjective. Use an LLM judge with a rubric:

• Did the agent follow the skill's procedure? (0-1)
• Was the output correct/useful? (0-1)
• Was it concise (within token budget)? (0-1)
• Did it avoid the known failure modes? (0-1)

Aggregate into a composite score.

Constraint Gates (must pass before any change is valid)

| Constraint | Rule | Why | |------------|------|-----| | Full test suite | pytest tests/ must pass 100% | Hard floor — nothing ships that breaks existing functionality | | Size limits | Skills ≤15KB, tool descriptions ≤500 chars, system prompts ≤120% of current | Prevents evolutionary bloat | | Caching compatibility | No mid-conversation changes — all changes take effect on new sessions | Preserves prompt caching | | Semantic preservation | Evolved text must not drift from original purpose | Fitness function includes similarity check | | Deployment via PR | All changes go through PR, never direct commit | Human review is mandatory |

Benchmark Gate Hierarchy

| Benchmark | What It Tests | Speed | Role | |-----------|---------------|-------|------| | TBLite fast subset | Coding/sysadmin (20 tasks) | ~20 min | Quick regression gate | | Full TBLite | Coding/sysadmin (100 tasks) | ~1-2 hrs | Thorough regression gate | | YC-Bench fast_test | Long-horizon coherence (50 turns) | ~30 min | Coherence check | | Session-specific | Skill-targeted eval dataset | Varies | Primary fitness signal |

Key principle: Benchmarks are GATES, not fitness functions. A variant that improves skill quality by 20% but drops TBLite by 5% is REJECTED.

Practical Considerations

Cost:

• GEPA optimization: ~$2-10 per run (API calls only, no GPU)
• Synthetic eval data generation: ~$1-3 per skill
• Start with small eval sets (10-20 examples), scale up for important skills

Safety:

• All changes via PR, never direct commit
• Full test suite gate (zero tolerance)
• Character/token budgets enforced
• Caching compatibility checked
• Git-tracked lineage, trivial rollback

No GPU required — everything operates via API calls. GEPA mutates text and evaluates via LLM calls, not weight training.

References

• hermes-agent-self-evolution: https://github.com/NousResearch/hermes-agent-self-evolution
• GEPA paper: ICLR 2026 Oral
• DSPy: https://github.com/stanfordnlp/dspy

---

Part 4: Tool Description Improvement (Anthropic Pattern)

Anthropic found: "We created a tool-testing agent — when given a flawed MCP tool, it attempts to use the tool and then rewrites the tool description to avoid failures. This process resulted in a 40% decrease in task completion time for future agents using the new description."

This is a separate loop from behavioral benchmarking. Use it when:

• An MCP tool or skill consistently causes failures across multiple agents
• A tool description is vague, misleading, or causes agents to take wrong paths
• You notice agents consistently misusing the same tool

Tool-Testing Loop

IDENTIFY → TEST → DIAGNOSE → PATCH → VERIFY

1. Identify: Log which tool/skill caused failures in session logs
2. Test: Run the tool with various inputs — document failures
3. Diagnose: Why does the tool description cause agents to fail?
4. Patch: Rewrite the tool description to make failure modes explicit
5. Verify: Re-run benchmark tasks using the tool — measure improvement

Tool Improvement Prompt

TASK: Improve the description for [tool/skill name] at [path]

FAILURE OBSERVED:
[what agents did wrong when using this tool]

TOOL DESCRIPTION CURRENTLY:
[copy the current description]

WHY IT FAILS:
[analysis — what in the description caused the wrong behavior]

PROPOSED IMPROVEMENT:
[new description that makes the failure mode explicit]

VERIFICATION:
After patching, run [task type that previously failed] and confirm improvement

---

Critical Rules

• Each lens must RETHINK the question, not just add more information. Technical and contrarian should feel like two researchers who disagree.
• The tension between lenses IS the insight. Don't resolve it away.
• Never present a single-lens finding as a conclusion.
• Separate "what the data shows" from "what I interpret."
• [[open-questions]] is as important as [[executive-summary]].

---

Folder Structure

agentic-self-improvement/
├── SKILL.md                        # This file
├── BENCHMARKS/                     # Behavioral benchmark YAMLs
│   ├── mandatory_tool.yaml
│   ├── act_dont_ask.yaml
│   ├── no_hallucination.yaml
│   ├── verification.yaml
│   ├── prerequisite.yaml
│   ├── path_accuracy.yaml
│   ├── context_grounding.yaml
│   ├── auth_state.yaml
│   └── remember_to_obsidian.yaml
├── REFERENCES/
│   └── guidance_anatomy.md         # Guidance block reference
└── src/                            # Python tooling
    ├── benchmark_runner.py         # Run benchmarks via hermes chat
    ├── failure_analyzer.py         # Analyze results, group failures
    ├── patch_generator.py          # Generate guidance patches
    └── apply_and_verify.py         # Apply patches with auto-revert

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: ChuckSRQ
• Source: ChuckSRQ/awesome-hermes-skills
• License: MIT
• Homepage: https://github.com/ChuckSRQ/awesome-hermes-skills

Install and usage instructions live in the source repository linked above.