ForgeAgent

ForgeAgent — verified skill memory for AI agents

ForgeAgent helps AI coding agents and developer teams turn generated code into verified, reusable capabilities before it is allowed to run again.

Here, an agent forges a capability once, verifies it, then reuses the proven version instead of regenerating it.

Live terminal recording

This is a real zero-API-key capture of [python3 scripts/run_video_demo.py](scripts/runvideodemo.py), generated from [scripts/record_terminal_demo.tape](scripts/recordterminaldemo.tape). It runs the actual Foundry, isolated proof, production approval, persistent reuse, reproduced failure/quarantine, inherited-regression repair, and v2 reuse paths. For the browser-first version of the same story, run python3 main.py --serve and open http://127.0.0.1:8787/showcase.

Verified GPT-5.6 Terra run

On July 21, 2026, a live OpenAI gpt-5.6-terra Foundry run created the previously missing extract_invoice_ids capability from an explicit ASCII-only contract. Its first two candidates were rejected with 5 and 3 proof failures; the live adversarial pass supplied 4, then 3, then 2 targeted cases across the repair loop. The repaired extract_invoice_ids@v2 passed normal, edge, three contract, and two adversarial cases at trust score 100; an exact later request returned status: "reused" with no new model generation. This is separate from the hosted and offline demos, which remain deliberately key-free.

To reproduce the live path after setting OPENAI_API_KEY in your shell (never commit a key), run:

python3 main.py --foundry-task "Build a pure Python invoice ID extractor. From payload.text, return unique IDs matching the exact ASCII pattern INV-[0-9]+ in first-seen order. Reject Unicode, superscript, circled, and full-width numerals. Reject IDs embedded in larger words, underscores, or hyphenated tokens." \
  --payload '{"text":"Repeat: INV-77, INV-77, INV-88. Invalid: INV-² and XINV-9."}' \
  --foundry-live --provider openai --model gpt-5.6-terra --adversarial-proof

What this proves

• Build with evidence: a missing capability earns entry to memory only after isolated proof, policy checks, and a clear contract.
• Reuse with confidence: another agent can retrieve the trusted version instead of rebuilding the same tool from scratch.
• Improve safely: a reproduced failure removes trust immediately; a repaired version earns it back by passing both its original proof and the retained regression.

Try the visual story yourself: run python3 main.py --serve, then open http://127.0.0.1:8787/showcase for the browser walkthrough or http://127.0.0.1:8787/judge to inspect every action individually.

The hook story

A generated capability could alias __import__ and attempt to bypass the sandbox policy gate. ForgeAgent closes that class twice: a static AST check rejects it before execution, and the runtime replaces imports with the safe_import allowlist in [forgeagent/sandbox.py](forgeagent/sandbox.py); both protections are exercised by [tests/test_sandbox_security.py](tests/testsandboxsecurity.py).

AI agents are increasingly allowed to write and run their own code; without a system like this, that trust is assumed, not earned.

Demo video

▶ Demo video — link coming soon

Architecture

This is the runnable local system: agents reach an MCP server, governed capabilities are proved and remembered, and later agents can reuse trusted work without rebuilding it.

Proof at a glance

All figures below were generated locally without an API key on July 20, 2026.

| Evidence | Measured result | Source command | | --- | --- | --- | | Regression suite | 89 tests run: 83 passed; 6 optional-policy tests skipped when PyYAML is absent | python3 -m unittest discover -s tests -v | | Sandbox security regressions | 4/4 passed; import-alias, dunder-attribute, and dynamic-getattr escapes remain blocked | python3 -m unittest tests.test_sandbox_security -v | | Trust-gate benchmark | 8/8 cases passed; 7/7 attack patterns blocked | python3 main.py --benchmark | | Evaluation arena | 50/50 cases passed; 10/10 unsafe proposals rejected | python3 main.py --evaluate | | Stateless comparison | 36 new skills; 0 reuses | python3 main.py --compare | | ForgeAgent comparison | 4 new skills; 32 reuses | python3 main.py --compare |

The chart is generated from those commands, not hand-entered metrics. Regenerate it with python3 scripts/generate_benchmark_chart.py.

Raw command evidence

These files are the exact, unedited stdout captured from the commands named beside them—not summaries or hand-copied metrics: [benchmark.json](docs/evidence/benchmark.json) (python3 main.py --benchmark), [evaluate.json](docs/evidence/evaluate.json) (python3 main.py --evaluate), and [compare.json](docs/evidence/compare.json) (python3 main.py --compare).

Quickstart

Python 3.10+ is the only requirement for the offline judge path—no packages or API key are needed.

git clone https://github.com/Yashasm18/ForgeAgent.git
cd ForgeAgent
python3 main.py --demo --reset && python3 main.py --demo
python3 -m unittest discover -s tests -v

The first command forges curated offline capabilities after proof; the second demonstrates their verified reuse. For the local visual ledger, run python3 main.py --serve and open http://127.0.0.1:8787.

One-click Judge Mode

For a video-ready browser walkthrough, run python3 main.py --serve and open http://127.0.0.1:8787/showcase. One click runs the real lifecycle: a coding agent requests a capability, production governance holds it, another agent reuses it, a reproduced failure quarantines it, and a repaired successor must pass the retained regression before reuse. For individual controls and full raw evidence, open http://127.0.0.1:8787/judge. Both use an isolated local SQLite scenario, require no API key, and reset only data/judge_mode/.

What's implemented

| Capability | What is real now | Where to verify it | | --- | --- | --- | | Governed capability lifecycle | Propose, policy-check, isolate, prove, persist, version, and reuse constrained run(payload) capabilities. | [forgeagent/foundry.py](forgeagent/foundry.py), [forgeagent/proof_engine.py](forgeagent/proofengine.py), python3 main.py --foundry-task "Normalize inconsistent date formats in this import log" --payload '{"text":"batch=A 03/07/2026"}' | | Cross-agent reuse | A capability approved by one real MCP subprocess is reused by a separate OS process from the SQLite platform store. | [tests/test_mcp_server.py](tests/testmcpserver.py) | | Adversarial proof | An opt-in live GPT-5.6 adversarial pass creates contract-breaking cases; a failure blocks promotion and enters repair evidence. A labelled offline recorded example is also included. | [forgeagent/generator.py](forgeagent/generator.py), [forgeagent/proof_engine.py](forgeagent/proofengine.py), [forgeagent/demo_tasks.py](forgeagent/demotasks.py) | | Semantic matching | When a live generator is configured, existing capabilities are matched by task intent; unavailable or failed live matching falls back to the existing offline keywords. | [forgeagent/agent.py](forgeagent/agent.py), [forgeagent/generator.py](forgeagent/generator.py) | | Offline intelligence | Without an API key, deterministic catalog matching resolves natural phrasing for known skills; reviewed templates can forge an invoice-ID extractor, adversarially prove it, and refuse untemplated novelty. | [forgeagent/offline_intelligence.py](forgeagent/offlineintelligence.py), python3 main.py --offline-foundry --offline-adversarial-proof --foundry-task "Extract invoice IDs from billing logs" --payload '{"text":"INV-2048"}' | | Structured outputs | The three live GPT-5.6 calls enforce JSON Schemas through the Responses API before parsing. | [forgeagent/generator.py](forgeagent/generator.py) | | Live Foundry Council | The dashboard polls newly appended council decisions from the SQLite + JSONL audit trail while a Foundry run is active. | [forgeagent/dashboard.py](forgeagent/dashboard.py), [forgeagent/audit.py](forgeagent/audit.py), python3 main.py --serve | | Accurate proof evidence | Capability records persist the actual passing proof-case count; the dashboard shows evidence unavailable rather than inventing a fallback value. | [forgeagent/registry.py](forgeagent/registry.py), [forgeagent/dashboard.py](forgeagent/dashboard.py) | | Contract drift and feedback regression | Replays persisted proof cases for trusted versions, quarantines a reproduced failure, and carries verified feedback regressions into successor versions before they can earn trust. | [forgeagent/platform_store.py](forgeagent/platformstore.py), python3 main.py --contract-drift --project local/default | | Video walkthrough + Judge Mode | A browser-driven, real local lifecycle: forge → human approval → cross-call reuse → reproduced failure/quarantine → repair with inherited regression proof. The walkthrough is one-click; Judge Mode exposes each individual action. Both use their own isolated SQLite directory. | [forgeagent/judge_mode.py](forgeagent/judgemode.py), [forgeagent/dashboard.py](forgeagent/dashboard.py), python3 main.py --serve then http://127.0.0.1:8787/showcase | | Reproducible evaluation | The benchmark chart is regenerated from the local benchmark, evaluation, and comparison commands. | [scripts/generate_benchmark_chart.py](scripts/generatebenchmarkchart.py) | | Production reference profile | An optional rootless, no-egress container profile and stricter approval policy are available for validation. | [Dockerfile.sandbox](Dockerfile.sandbox), [compose.production.yml](compose.production.yml) |

Live judge demo

Open the Forge Ledger → — a no-install walkthrough with the clickable ForgeGraph, browser capability run, Policy Attack Lab, version lineage, and Production Preflight.

Capability Foundry

ForgeAgent is now a Capability Foundry: a governed learning layer for coding agents rather than a prompt-only assistant. Given a task, it builds a repository intelligence graph, identifies whether a trusted capability already exists, produces a constrained proposal, creates a threat model, runs proof cases in isolation, records a governed decision, and either executes the trusted capability or keeps the rejected evidence for review.

The Foundry Council makes that lifecycle explicit:

• Planner maps a task to a capability gap and dependency impact.
• Builder creates a constrained run(payload) tool.
• Security derives a threat model and checks the static policy boundary.
• Evaluator runs normal, edge, and contract proof cases in an isolated subprocess.
• Governor promotes, holds for review, rejects, or rolls back a version.

The local-first control plane uses SQLite for project namespaces, trust scores, proof reports, approval decisions, audit receipts, and signed capability packages. Live generation is provider-explicit: use OpenAI gpt-5.6-terra or local Ollama; the complete offline lifecycle remains runnable without an API key.

Why it matters

An agent can now get better over time without quietly accumulating unverified code. Every saved skill has source, deterministic test evidence, provenance, reuse history, and an append-only decision record. Broken or policy-violating candidates are rejected before they can enter memory.

Isn’t this just caching?

No. Memoization stores an answer for an identical input; ForgeAgent stores a capability version only after it earns reusable evidence. A blind LLM-code cache would save source because it was generated. ForgeAgent instead:

• runs normal, edge, and contract proof categories in

[forgeagent/proof_engine.py](forgeagent/proof_engine.py) before trust;

• applies sensitivity and production approval policy in

[forgeagent/governance.py](forgeagent/governance.py), rather than treating a passing string transform and an external action alike;

• preserves version lineage and rollback rather than overwriting history; and
• turns a reproducible incorrect developer outcome into a sandboxed regression

case, quarantining the broken version and requiring a repaired successor to pass that case before it can be trusted again; and

• routes a candidate replacement through the same policy-and-test path in

[forgeagent/agent.py](forgeagent/agent.py) and [forgeagent/foundry.py](forgeagent/foundry.py) before ToolRegistry.replace can supersede a trusted version.

The result is capability governance, not cached model output: source, provenance, tests, policy decision, approval, and rollback state travel with the reusable tool.

Who uses it and when

• Coding agents and developer teams: avoid regenerating a parser, validator,

normalizer, or extractor that has already been proven for the project.

• Support and operations agents: redact incident data before routing,

risk-triaging, or extracting recurring failure themes.

• Security-conscious teams: keep unsafe tool proposals as rejected evidence

and require named approval before sensitive or production capability reuse.

• Platform teams: export a proof-backed capability package, then import it

into another project where it starts in review rather than becoming trusted.

Example: an agent needs an invoice-ID extractor. ForgeAgent first checks existing capability memory and repository context. If the capability is missing, it produces a constrained run(payload) proposal, attacks it with policy/proof checks, repairs it when possible, versions it after approval, and reuses only the trusted version on the next request.

Quick demo (no API key)

python3 main.py --demo --reset
python3 main.py --demo
python3 main.py --serve
python3 main.py --benchmark
python3 main.py --foundry-task "Extract structured error codes and line numbers from this stack trace" --payload '{"text":"ERROR E_CONN_TIMEOUT at ingest.py:line 42"}'
python3 main.py --repo-graph
python3 main.py --evaluate
python3 main.py --mcp
python3 main.py --showcase --reset
python3 main.py --autonomy-demo --reset
python3 main.py --autonomy-demo
python3 main.py --compare

Open http://127.0.0.1:8787 to see the Forge Ledger. The first demo run creates curated offline skills; the second proves that verified memory is reused. The curated mode is intentionally labelled as a recording fallback—it does not claim to be a live model call.

--foundry-task runs the five-role council. With a supported capability it uses the curated offline proposal path; add --foundry-live and OPENAI_API_KEY to let gpt-5.6-terra plan and propose an unknown capability. --repo-graph exports the repository graph, --evaluate runs 50 measured cases, and --mcp starts the stdio MCP server for compatible coding agents.

Offline intelligence (no API key)

ForgeAgent is useful without credits, but it is deliberately bounded rather than a pretend local replacement for GPT. It performs deterministic semantic matching over the trusted catalog, plans known multi-step workflows, and can forge only explicitly reviewed templates. Today the new-template path covers an invoice_id_extractor, including two deterministic adversarial cases; an unsupported task is refused rather than fabricated.

# Forge, proof-check, and promote a reviewed template locally.
python3 main.py --foundry-task "Extract invoice IDs from billing logs" \
  --payload '{"text":"paid INV-2048; pending INV-9"}' \
  --offline-foundry --offline-adversarial-proof

# Plan and execute a known incident workflow locally.
python3 main.py --offline-autonomous-task \
  "Redact PII, triage the support incident, and extract stack-trace error codes" \
  --payload '{"text":"ava@example.com cann

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [Yashasm18](https://github.com/Yashasm18)
- **Source:** [Yashasm18/ForgeAgent](https://github.com/Yashasm18/ForgeAgent)
- **License:** MIT
- **Homepage:** https://yashasm18.github.io/ForgeAgent/

Install and usage instructions live in the source repository linked above.