# Rlhf Feedback Loop > RLHF feedback loop for AI agents. Capture feedback, block mistakes, export DPO data. - **Type:** MCP server - **Install:** `agentstack add mcp-igorganapolsky-rlhf-feedback-loop` - **Verified:** Pending review - **Seller:** [IgorGanapolsky](https://agentstack.voostack.com/s/igorganapolsky) - **Installs:** 0 - **Latest version:** 0.6.2 - **License:** MIT - **Upstream author:** [IgorGanapolsky](https://github.com/IgorGanapolsky) - **Source:** https://github.com/IgorGanapolsky/ThumbGate - **Website:** https://thumbgate.ai?utm_source=github&utm_medium=repo_about&utm_campaign=organic_repo ## Install ```sh agentstack add mcp-igorganapolsky-rlhf-feedback-loop ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # ThumbGate **AI coding agents repeat mistakes — and one wrong tool call can wipe a directory, leak a key, or push broken code.** ThumbGate is the local-first firewall for AI coding agents. It runs in the PreToolUse hook on your machine and blocks dangerous tool calls — `rm -rf`, secret exfiltration, off-scope edits, a bad `git push` — before they execute, across Claude Code, Cursor, Codex, Gemini, Amp, Cline, and OpenCode. No server, no gateway. (Regulated-industry policy templates — legal intake, financial compliance, healthcare — build on the same engine.) The product is a self-improving enforcement layer: thumbs-down feedback, prompt evaluation, and proof from prior runs become prevention rules that permanently stop repeated failures before the next tool call. ``` Agent tries: rm -rf tests/ ThumbGate: ⛔ BLOCKED — "Never delete test directories" Pattern matched: rm.*-rf.*tests Source: your thumbs-down from last Tuesday Tokens spent on this repeat: 0 ``` ```bash npx thumbgate init # auto-detects your agent, wires hooks, 30 seconds ``` Works with **Claude Code, Cursor, Codex, Gemini CLI, Amp, Cline, OpenCode** and any MCP-compatible agent. Free tier: 2 feedback captures/day (10 total) and up to 3 active auto-promoted prevention rules. [Pro: $19/mo or $149/yr](https://thumbgate.ai/checkout/pro?utm_source=github&utm_medium=readme) — unlimited rules, history-aware lessons, feedback sessions, dashboard, DPO export. Enterprise (custom pricing, scoped after intake) adds a shared hosted lesson DB, org dashboard, and shared org-wide enforcement. [](https://github.com/IgorGanapolsky/ThumbGate/actions/workflows/ci.yml) [](https://www.npmjs.com/package/thumbgate) [](LICENSE) --- > *"A better dashboard doesn't make the agents more reliable. The hard part isn't visibility. It's trust."* > > — **Rob May**, CEO & co-founder, Neurometric AI, quoted in [The New Stack](https://thenewstack.io/claude-code-agent-view/) on Anthropic's Claude Code Agent View (May 2026). > > ThumbGate is the open-source layer that makes the trust part real: PreToolUse gates, thumbs-down to rule, audit trail on every interception. --- ## Agentic development cycle fit Agentic development is becoming a loop: **Guide → Generate → Verify → Solve**. ThumbGate gives that loop a hard execution boundary. - **Guide:** standards, prior thumbs-downs, and approval policies become concrete context. - **Generate:** Claude Code, Cursor, Codex, Gemini, Amp, Cline, OpenCode, and MCP agents keep producing plans and tool calls. - **Verify:** risky actions need evidence before execution, not just after PR review. - **Solve:** blocked failures become reusable lessons, shared prevention rules, DPO exports, and audit events. In that stack, ThumbGate is the pre-action gate between generated intent and executed action. --- ## Discoverable slash-commands — the guardrail layer for spec-driven agents Spec-driven agent frameworks like **GSD** (get-shit-done) and **GitHub Spec Kit** are great at *planning and generating* work — they expose dozens of discoverable `/gsd-*` / `/specify` commands in the agent command palette. ThumbGate is the **guardrail layer for spec-driven agents**: it sits *after* the plan, on the boundary between a generated tool call and its execution. It works **alongside GSD / Spec-Kit, not instead of them** — they decide *what* to build; ThumbGate enforces *what the agent must never do while building it*. `npx thumbgate init` installs these commands into your agent's palette (`.claude/commands/`, `.gemini/commands/`, `.antigravitycli/commands/`) so the enforcement layer is as browsable as the planning layer: | Command | What it does | Wraps (existing capability) | |---------|--------------|------------------------------| | `/thumbgate-guard` | Turn the last agent mistake into a hard prevention rule | `capture_feedback` + `thumbgate force-gate` | | `/thumbgate-rules` | List the active prevention rules + lessons guarding this repo | `prevention_rules`, `get_reliability_rules`, `search_lessons` | | `/thumbgate-blocked` | Show what's actually been blocked — gate stats + enforcement matrix | `gate_stats`, `enforcement_matrix` | | `/thumbgate-protect` | Show branch/release governance; grant a scoped, expiring approval | `get_branch_governance`, `approve_protected_action` | | `/thumbgate-doctor` | Health-check the wiring (hooks, MCP, agent-readiness) | `thumbgate doctor` | Each is a thin wrapper over an existing MCP tool or CLI command — **no new enforcement logic, just discoverability**. --- ## 🎬 90-second demo Watch the force-push scenario: agent tries to `git push --force`, one thumbs-down, next session it's blocked — zero tokens spent on the repeat. [**▶ Watch the 90-second demo**](https://thumbgate.ai/#demo?utm_source=github&utm_medium=readme&utm_campaign=demo_video) · [Script](docs/marketing/demo-video-script.md) · [ElevenLabs narration: `npm run demo:voiceover`](scripts/generate-demo-voiceover.js) --- ## First-dollar activation path If someone is not already bought into ThumbGate, do not lead with architecture. Lead with one repeated mistake. 1. **Show the pain:** open the **[ThumbGate GPT](https://thumbgate.ai/go/gpt?utm_source=github&utm_medium=readme&utm_campaign=first_dollar_activation&cta_id=readme_first_dollar_open_gpt&cta_placement=readme_first_dollar)** and paste the bad answer, risky command, deploy, PR action, or agent plan before it runs again. 2. **Capture the lesson:** type `thumbs down:` or `thumbs up:` with one concrete sentence. Native ChatGPT rating buttons are not the ThumbGate capture path; typed feedback is. 3. **Enforce the repeat:** run `npx thumbgate init` where the agent executes so the lesson can become one of your Pre-Action Checks instead of another reminder. 4. **Upgrade only after proof:** Solo Pro is for the dashboard, DPO export, proof-ready evidence, and higher capture limits after one real blocked repeat. Team starts with the Workflow Hardening Sprint around one repeated failure, one owner, and one proof review. The buying question is simple: **what repeated AI mistake would be worth blocking before the next tool call?** --- ## The Problem — the bill nobody talks about Frontier-model calls are not cheap. Sonnet 4.5 is ~$3 / 1M input tokens and ~$15 / 1M output tokens. Opus is 5× that. Every time your agent: - hallucinates a function name and you have to correct it, - retries the same failing tool call until it gives up, - regenerates a 4,000-token plan you already approved last session, - repeats a destructive command you blocked manually yesterday, …you are paying for that round-trip. *Twice if it retries. Three times if you re-prompt.* And the agent has no memory across sessions, so the meter resets every Monday. ``` Session 1: Agent force-pushes to main. You fix it. +4,200 tokens Session 2: Agent force-pushes again. You fix it. +4,200 tokens Session 3: Same mistake. Again. You lose 45m. +5,800 tokens ``` That's ~$0.21 in tokens just to fix the same mistake three times — multiplied by every developer, every repeated-mistake class, every week. The math gets ugly fast. ## The Solution — fix it once, the bill never sees it again ``` Session 1: Agent force-pushes to main. You 👎 it. +4,200 tokens Session 2: ⛔ Check blocks the force-push. Zero round-trip. +0 tokens Session 3+: Never happens again. +0 tokens ``` One thumbs-down. The PreToolUse hook intercepts the call **before** it reaches the model — no input tokens, no output tokens, no retry loop. The dashboard tracks **tokens saved this week** as a live counter so you can see exactly what your prevention rules are worth. Mark a review checkpoint once, and the dashboard narrows the next pass to only the feedback, lessons, and check blocks that landed since your last review. ThumbGate doesn't make your agent smarter. It makes your agent *cheaper to be wrong with.* --- ## 🧠 The Context Brain Every coding agent starts each session amnesiac — it has no memory of the mistakes it made yesterday, the fixes your team already rejected, or the rules this repo enforces. So it repeats them, and you pay for it again. ThumbGate gives your repo a **context brain**: a single, versioned, agent-readable artifact that consolidates everything the agent should know *before it acts* — the lessons it has learned, the guardrails it must not cross, the gates that are enforced, and the project's own instruction files. ```bash npx thumbgate brain --write # → .thumbgate/BRAIN.md ``` Then point your agent at it — add `Read .thumbgate/BRAIN.md first` to your `CLAUDE.md` / `AGENTS.md`, and every Claude Code, Codex, Cursor, or Gemini CLI session boots with your repo's institutional memory already loaded. The output is **deterministic**, so `BRAIN.md` lives in git and only changes when the underlying memory does — review it like any other file. ``` # ThumbGate Context Brain ## What this codebase taught its agents (lessons) - ⛔ Force-pushing to main was rejected — use --force-with-lease on feature branches only ## Guardrails — do NOT repeat these (prevention rules) - Never run DROP on production tables ## Active enforcement (gates) - `DROP.*production` → block ``` Same idea the SEO world is now calling a *"client brain"* — persistent context that AI reads before doing the work — applied to **engineering**: the institutional memory that stops your coding agent from relearning the same lesson on your dime. --- ## Quick Start ```bash npx thumbgate init # auto-detects your agent, wires everything npx thumbgate capture down "Never run DROP on production tables" ``` That single command creates a prevention rule. Next time any AI agent tries to run `DROP` on production: ``` ⛔ Check blocked: "Never run DROP on production tables" Pattern: DROP.*production Verdict: BLOCK ``` --- ## Architecture ThumbGate operates as a 4-layer enforcement stack between your AI agent and your codebase: ### Layer 1: Feedback Capture Your thumbs-up/down reactions are captured via MCP protocol, CLI, or the ChatGPT GPT surface. Each reaction is stored as a structured lesson with context, timestamp, and severity. ### Layer 2: Check Engine The check engine converts lessons into enforceable rules. **The runtime gate decision is deterministic** — literal pattern match → AST match → scoped rule lookup. No LLM call on the enforcement path. Where retrieval is needed (an agent is about to run a destructive command not on the literal block list, but semantically similar to one we've blocked before), ThumbGate uses local CPU-only `bge-small` embeddings via LanceDB's built-in pipeline. No external API call, no inference cost beyond CPU. So **"no LLM in enforcement"** holds: the gate decision uses no LLM; the rule corpus is just searchable via local embeddings. **Thompson Sampling tunes per-rule confidence weights** for soft-gating rules so high-noise rules quiet down and high-signal rules sharpen. It never decides *whether* a rule fires — a hard rule like "block `git push --force` on main" always fires deterministically. Bandit exploration would be terrifying for hard rules; we don't do it. Rules stay in local ThumbGate runtime state. ### Layer 3: Pre-Action Interception Before any agent action executes, ThumbGate's `PreToolUse` hook intercepts the command and evaluates it against all active checks. This happens at the MCP protocol level — the agent physically cannot bypass it. ### Layer 4: Multi-Agent Distribution (the actual moat vs hand-rolled hooks) Claude Code already ships `permissions.deny` and `PreToolUse` hooks. Cursor and Codex have their own. So why ThumbGate over a hand-written hook? Two things hand-written hooks structurally cannot do: 1. **Cross-agent propagation.** A `permissions.deny` pattern lives in one agent's config and stays there. ThumbGate's checks distribute across every connected agent over MCP stdio — thumbs-down once in Cursor, the same pattern blocks on Claude Code, Codex, Gemini CLI, Cline, OpenCode, Amp in the next session, no copy-paste between configs. 2. **Learning loop.** A hand-written hook covers exactly the patterns you wrote. ThumbGate promotes every thumbs-down into a fresh rule, tunes existing rules' confidence weights from outcomes (Thompson Sampling, see Layer 2), and pulls semantically-near patterns into scope via local embeddings. The rule corpus sharpens without an operator hand-writing a regex for every new mistake shape. Hand-rolled hooks are the right tool for a small, static denylist you maintain by hand. ThumbGate is the right tool when you want corrections from any agent to harden every agent automatically. Prompt engineering still matters, but it is only the starting point. ThumbGate adds prompt evaluation on top: proof lanes, benchmarks, and self-heal checks tell you whether your prompt and workflow actually held up under execution instead of leaving you to guess from vibes. Run `npx thumbgate eval --from-feedback --write-report=.thumbgate/prompt-eval-proof.md` to turn real thumbs-up/down feedback into reusable eval cases and a buyer-ready proof report. ### Retrieval & latency: local-first, zero network hops ThumbGate's latency advantage is structural, not a tuned cloud cluster: there is no retrieval service and no model on the enforcement path, so the gate decision never leaves your machine. ```mermaid flowchart LR A["Agent about to runa tool call"] --> B{"Literal / AST matchon an active rule?"} B -- "exact match" --> D["Deterministic gate decision(no model, on-device)"] B -- "no exact match, butsemantically near ablocked pattern" --> C["Local CPU embeddingsbge-small via LanceDB(no external API)"] C --> D D -- "known-bad" --> E["⛔ BLOCK before execution"] D -- "safe" --> F["✓ Allow"] ``` - **Deterministic first.** Most decisions are a literal or AST pattern match against your active rules — sub-millisecond, on-device, no embeddings. - **Local semantic fallback.** When an action isn't on the literal block list but is semantically near one you've blocked before, ThumbGate searches the rule corpus with CPU-only `bge-small` embeddings via LanceDB — still local, still no external API call. - **No LLM on the enforcement path.** The gate never calls a model to decide block/allow. Thompson Sampling only tunes soft-rule confidence weights; hard rules always fire deterministically (see Layer 2). The fastest network round-trip is the one you never make: enforcement is fully local, so it adds negligible latency to the agent loop — no cloud retrieval, no inference hop, no data leaving the machine. ### Managed model benchmark lane When a new managed model drops, do not swap ThumbGate over on vendor claims alone. Rank it against the actual ThumbGate workload first: ```bash npx thumbgate model-candidates --workload=pretool-gating --json npx thumbgate model-candidates --workload=long-trace-review --provider=openai-compatible --gateway=tinker --json ``` The catalog currently includes the April 23, 2026 Tinker additions: - `tinker/qwen3.6-35b-a3b` for pre-action gating, agentic coding, and tool-use - `tinker/qwen3.6-27b` for the cheap fast-path - `tinker/kimi-k2.6-128k` for long-trace review and multi-agent sessions Each recommendation ships with the benchmark commands to run next: feedback-derived prompt eval, `gate-eval`, and `thumbgate bench`. For whole-repo clone claims, add `npx thumbgate bench --programbench-smoke` to generate a ProgramBench-style cleanroom proof report without claiming an official ProgramBench score. That keeps model selection evidence-backed instead of hype-driven. --- ## Install for Your Agent | Agent | Command | |-------|---------| | **Claude Code** | `npx thumbgate init --agent claude-code` | | **Cursor** | `npx thumbgate init --agent cursor` | | **VS Co … ## Source & license This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [IgorGanapolsky](https://github.com/IgorGanapolsky) - **Source:** [IgorGanapolsky/ThumbGate](https://github.com/IgorGanapolsky/ThumbGate) - **License:** MIT - **Homepage:** https://thumbgate.ai?utm_source=github&utm_medium=repo_about&utm_campaign=organic_repo Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Versions - **0.6.2** — security scan: pending review — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/mcp-igorganapolsky-rlhf-feedback-loop - Seller: https://agentstack.voostack.com/s/igorganapolsky - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.