Pentest Ai

pentest-ai

The pentest tool that proves its findings. No oracle, no badge.

[](https://pypi.org/project/ptai/) [](https://pypi.org/project/ptai/) [](https://github.com/0xSteph/pentest-ai/actions/workflows/ci.yml) [](LICENSE) [](https://github.com/0xSteph/pentest-ai/stargazers) [](https://discord.gg/6weeTAubJw)

Website · [Install](#install) · [Why verification](docs/why-verification.md) · [Docs](docs/) · [Benchmarks](docs/benchmarks/juice-shop.md) · Agents · Discord

> ⚠️ Offensive tooling, authorized testing only. By installing you accept the AUP and Terms. Full text in [Responsible use ↓](#responsible-use)

ptai is an AI-driven pentest tool that re-runs every exploit to confirm it. It runs recon, logs in, and chains findings into multi-step attack paths, but it does not ask you to trust the results. The way TruffleHog confirms a leaked secret by logging in with it, ptai confirms a web finding by re-running the exploit: a finding stays a candidate until a machine oracle reproduces it N out of N, and only then does it earn a VERIFIED badge. Third-party scanner output (nuclei, nikto, zap) is held back until an oracle re-proves it. Scanner noise is what trains teams to ignore their tools, so the report carries only what ptai could prove, each VERIFIED finding with a portable proof capsule you can replay yourself.

On a private 20-bug honeypot: 19/20 caught, 100% precision, zero false positives, four bug classes oracle-verified. Runs on your laptop. No cloud, no telemetry.

Prove it yourself

pip install ptai && ptai demo

ptai demo scans a bundled vulnerable app and reports 4 findings, 4 oracle-VERIFIED, replays one live from a proof capsule (replay 3/3), then runs the same routes hardened and reports 0 findings. The only thing that changed between the two runs is the fix, so the findings appear and disappear with the vulnerability, not because the tool went quiet. Two minutes, no API key, no target of your own. Re-prove any capsule yourself with ptai replay.

> Honest numbers. The 19/20 at 100% precision figure is one run against a private 20-bug honeypot (39 findings total), not a field false-positive rate. The oracle gate buys precision without costing recall: it removes false positives, it does not raise catch rate. "Four oracle-verified classes" means four bug classes have a concrete oracle today; more land each release. OAST proves the HTTP-callback blind tier; DNS-only callbacks are a follow-on. The honeypot harness (tests/honeypot/) and a clean-app zero-FP gate (tests/cleanapp/) ship in the repo, so the claims are reproducible rather than screenshots.

What's new: the Trust Spine

Every VERIFIED finding now comes from a named machine oracle, never an LLM assertion, and it is enforced in code: a verdict that cannot name its oracle is rejected. This release adds:

• Eight machine oracles that re-run the exploit to confirm it: reflection, open-redirect, IDOR/BOLA, error-disclosure, MCP-exposure, SQLi (boolean and blind), and out-of-band (OAST) for blind SSRF/XXE over a self-hosted collaborator.
• Prove-or-kill gating: third-party scanner output stays hidden until an oracle re-proves it.
• Portable proof capsules with ptai replay, a live TUI that shows verdicts flip to VERIFIED on screen, and a CI gate (--fail-on verified) that breaks a build only on proven findings.

See it run

One prompt to Claude Code. The MCP server ran ptai's tools against the target, and Claude streamed findings back into the session. Subscription-driven, no API key.

Pointed at a stock OWASP Juice Shop through Claude Code over MCP, ptai found and PoC'd the headline bugs: JWT alg:none accepted on 8+ protected endpoints, SQLi auth bypass on /rest/user/login, UNION-based SQLi on /rest/products/search, path-filter bypass via NUL byte, XXE disclosing /etc/passwd, file-upload polyglot, mass assignment, password reset bypass. Each one has a working PoC. Juice Shop is a heavily-studied target, so read the raw finding volume as breadth; the precision story, only what an oracle could prove, is the demo and honeypot above.

> Recording is the actual output of claude -p against a local OWASP Juice Shop with pentest-ai registered as an MCP server. Cast file in [assets/realdemo.cast](assets/realdemo.cast); the time-paced re-render used for the GIF is in [assets/realdemo-paced.cast](assets/realdemo-paced.cast). Findings are real; inter-line timing was reconstructed for watchability since claude -p buffers and dumps in non-interactive mode. A deterministic synthesized fallback ([assets/demo.tape](assets/demo.tape) + [assets/demo.sh](assets/demo.sh)) is kept for reproducible re-renders.

> Honesty caveat: Juice Shop is the most-written-about deliberately-vulnerable app on the internet, so LLMs and probe authors both have a head start. Against a novel target the catch rate is whatever the curated probe library actually covers: 60 web probes today, growing each release. The LLM coordinates and reasons about results; it doesn't replace the probes. A private honeypot harness in tests/honeypot/ measures coverage against bugs we wrote ourselves and is asserted in CI at 10/10 caught (tests/honeypot/test_mcp_honeypot_e2e.py). Numbers there are lower than Juice Shop, and that's the point. We publish both. See the full [Juice Shop benchmark vs ZAP / Nuclei / HexStrike](docs/benchmarks/juice-shop.md).

Install

pip install ptai

Path 1: Drive it from Claude Code (no API key)

If you already pay for Claude Pro / Max / Team, your subscription IS the LLM. Wire ptai in as an MCP server:

claude mcp add pentest-ai -- ptai mcp

Restart Claude Code, then ask:

> "Run an authenticated pentest against staging.acme.com. Login is at /login, password is in $APP_PASS."

> What hits the network: ptai's tools and probes execute locally against your target. Your prompts and the tool output that Claude Code reads go through Anthropic's API, same as any Claude Code session. If you need an air-gapped path, see Path 3 (Ollama / on-prem LLM).

Claude Code drives ptai via these MCP tools (49 of them as of 0.14.0):

• list_tools / run_tool: list and invoke any of 200+ wrapped security tools
• plan_tools / ensure_tools_installed: get the canonical tool list for an engagement, batched install
• list_probes / run_probe: 60 SPA-aware probes for OWASP Top 10 bug classes
• http_request: raw HTTP under a hard scope guard for novel chains
• start_engagement / get_findings / get_attack_chains: the engagement record
• plus test_web_app, test_active_directory, test_cloud, test_api_security, and the rest

Path 2: Other MCP clients (Cursor, VS Code Copilot, Codex, Claude Desktop)

ptai setup --mcp

Auto-detects every MCP-compatible client you have installed and writes their config files. Restart the client and the same 49 tools are there.

Path 3: Standalone CLI when you DON'T have an MCP client

If you're using Claude Code, Cursor, Codex, or Claude Desktop, use Path 1 or 2 above and skip this section. No API key needed there.

Path 3 is for CI/CD pipelines, scheduled cron jobs, air-gapped terminals, and users without an MCP client. The standalone CLI has no LLM of its own, so you bring one via env var:

export ANTHROPIC_API_KEY=sk-ant-...           # Claude (best results)
# or
export OPENAI_API_KEY=sk-...                  # OpenAI
# or, fully local, no cloud
export PENTEST_AI_LLM_PROVIDER=ollama         # Ollama (default localhost:11434)
# or, any of 300+ models via LiteLLM (OpenRouter, Azure, DeepSeek, Groq, Mistral, ...)
pip install litellm

ptai start https://your-target.com

Hitting an OpenAI-compatible endpoint (DeepSeek cloud, Groq, Together AI, vLLM, etc.)? Set OPENAI_BASE_URL + PENTEST_AI_MODEL and use the openai provider. Full recipes for every provider - including custom model names, troubleshooting, and the LiteLLM-300+ list - live in [docs/llm-providers.md](./docs/llm-providers.md).

Spending cap (Path 3 only)

The standalone agent loop drives its own LLM, so runaway loops cost real money. ptai caps spend per engagement at $10 USD by default. A normal Sonnet 4.6 web-app sweep with prompt caching finishes well under that; an Opus 4.7 deep run can blow past it.

Change it via env var (no CLI flag - env var is the only knob):

export PTAI_PRICE_LIMIT=25        # raise to $25
export PTAI_PRICE_LIMIT=0         # unlimited (logs a warning)
unset PTAI_PRICE_LIMIT            # back to the $10 default

If the cap fires mid-engagement, the engagement is marked aborted_cost_limit and its checkpoint is preserved. Raise the cap and resume from where it stopped:

export PTAI_PRICE_LIMIT=25
ptai resume 

Paths 1 and 2 (MCP) don't use this cap - your AI client (Claude Code, Cursor, etc.) handles its own LLM billing.

Installing security tools

ptai wraps 200+ external tools. Three ways to get them on the box:

# 1. Zero-config (recommended). At engagement start, the planner predicts
#    which tools the LLM will need and asks ONCE to install the missing
#    ones. Decline once and the answer persists in
#    ~/.pentest-ai/install-preferences.json.
ptai start https://target.example.com

# 2. Batch install upfront. Skips the engagement-time prompt entirely.
ptai setup --tier core            # ~6 essentials, ~30s
ptai setup --tier recommended     # + fuzzers, crawlers, password tools, ~5m
ptai setup --tier full            # everything, ~30m

# 3. Install specific tools by name.
ptai setup --per-tool wpscan,dalfox,paramspider
ptai setup --wizard               # interactive picker

In non-interactive contexts (PTAI_NON_INTERACTIVE=1 or no TTY) ptai uses what's on PATH and logs (rather than prompts) for anything missing.

Other paths: REST API, MCP composition, HITL teleoperation, cloud workspace, public benchmarks

HTTP REST API (for dashboards and integrations)

pip install ptai[api]
ptai serve --port 8888

Endpoints: /health, /version, /agents, /tools, /engagements (list, detail, findings, chains, detection rules, SARIF export). Write endpoints (POST /engagements, POST /engagements/{id}/abort) require Authorization: Bearer $PENTEST_AI_API_TOKEN. Live event stream at WS /engagements/{id}/stream.

Load other MCP servers as tool sources

Compose with hexstrike or any other MCP-compatible security server. Edit ~/.pentest-ai/mcp_servers.json:

{
  "servers": [
    {"name": "hexstrike", "command": "python3 hexstrike_mcp.py", "transport": "stdio"}
  ]
}

Take over mid-run (HITL teleoperation)

While an engagement is running, press Ctrl+C twice within 600ms to pause the orchestrator and drop into a REPL: step, inspect findings, inject , skip, resume, abort. Current LLMs aren't fully autonomous. The operator owns the call when it matters.

Public benchmarks

Reproducible solve-rate measurements live in [benchmarks/](benchmarks/):

./benchmarks/scripts/run_all.sh   # writes JSON per run + RESULTS.md

Spec, harness, results all in git. The full Juice Shop comparison vs ZAP / Nuclei / HexStrike is at [docs/benchmarks/juice-shop.md](docs/benchmarks/juice-shop.md). No "98.7% detection rate" claims you can't audit.

Cloud workspace (Pro / Team / Enterprise)

The CLI is free forever and stores everything locally. If you want engagement history, branded client-ready PDF reports, and team collaboration, link the CLI to an app.pentestai.xyz workspace:

# Sign up, then Dashboard -> API Keys -> Generate -> copy ptai_...
ptai auth login        # paste the key (hidden prompt)
ptai auth status       # confirm link
# or use an env var for CI:
export PENTESTAI_API_KEY=ptai_...

ptai start auto-syncs findings to your cloud workspace when authed. No cloud = no calls; integration is silently off unless you log in.

No LLM at all (interactive launcher)

ptai menu

Numeric category navigation, search (/term), tag filtering (t web), keyword-based recommendation. Real engagements still go through ptai start with full scope confirmation.

Why it's different

| | | |---|---| | 🤖 LLM-coordinated, not LLM-dependent | Seventeen agents cover recon, web, API, AD, cloud, mobile, wireless, browser, credentials, privesc, vuln scan, chaining, PoC, detection, report, social engineering, and LLM red team. The LLM runs the phase loop and reasons about results; bug detection is in the curated deterministic probe library. Set no API key and the same probes still run. The LLM coordinates; it doesn't scan. | | 🔓 No API key on the MCP path | Claude Code / Cursor / Codex users drive ptai through MCP using their existing subscription. 200+ tool wrappers and 60 probes are LLM-callable without an Anthropic key. The standalone CLI (ptai start --agent-mode) is where the API key matters; that's the Codex-without-MCP, CI, and air-gapped paths. | | 🔐 It logs in | Most scanners die at the login page. This one holds a session, refreshes credentials when they expire, and every downstream tool inherits the cookie. Auth profiles store references (env vars, op://, Vault paths, AWS Secrets Manager ARNs), never the value. | | 🧪 Every finding is proven | A non-destructive proof of concept runs against the target. No more triaging 40 maybes from a noisy scanner. | | ⚡ CI-native | GitHub Action, severity gates, SARIF output, PR comments. Drop it into your workflow file and it runs on the next PR. | | 💾 Runs on your laptop | MIT licensed, no cloud calls. Runs offline with Ollama. Findings stay on your disk. |

How it works

┌─────────────────────────────────────────────────────────────┐
│                    ptai start                       │
└─────────────────────────────────────────────────────────────┘
                             │
          ┌──────────────────┼──────────────────┐
          ▼                  ▼                  ▼
      ┌────────┐        ┌────────┐        ┌─────────┐
      │ recon  │   ->    │  auth  │   ->    │   web   │
      └────────┘        └────────┘        └─────────┘
                                               │
          ┌────────────────────────────────────┤
          ▼                                    ▼
      ┌────────┐                          ┌─────────┐
      │   ad   │   ┌──────────────────┐   │ cloud   │
      └────────┘   │  Findings DB     │   └─────────┘
          │        │  (sqlite + evidence)│       │
          └───────▶│  scope-guarded     │◀──────┘
                   │  deduplicated      │
                   └──────────────────┘
                             │
                ┌────────────┼────────────┐
                ▼            ▼            ▼
           ┌──────┐    ┌─────────┐  ┌──────────┐
           │chain │    │validate │  │ detect   │
           └──────┘    └─────────┘  └──────────┘
                             │
                             ▼
                       ┌──────────┐
                       │  report  │   md · html · pdf · SARIF · JUnit
                       └──────────┘

Each agent runs with an LLM when you've set a key, or as a deterministic tool loop when you haven't. Either way the phase order is the same.

Agents

| Agent | Phase | Does | |---|---|---| | recon | 1 | Port scan, DNS and subdomain enum, service fingerprinting | | web | 2 | Authenticated OWASP Testing Guide v4 pass | | api_security | 2 | OpenAPI/GraphQL/REST surface analysis, OWASP API Top 10 | | browser | 2 | Playwright-driven DOM analysis, XHR capture, security-header grading | | ad | 3 | AD enum, Kerberoasting, BloodHound pathfinding, delegation abuse

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: 0xSteph
• Source: 0xSteph/pentest-ai
• License: MIT
• Homepage: https://pentestai.xyz

Install and usage instructions live in the source repository linked above.