Websec Validator

websec-validator

[](https://github.com/raccioly/websec-validator/actions/workflows/ci.yml) [](https://pypi.org/project/websec-validator/) [](https://pypi.org/project/websec-validator/) [-brightgreen)](#install) [](#ci--enterprise-integration) [](LICENSE)

A senior pentester's "here's what to test and how" handoff — auto-generated from your repo, for your AI agent to execute.

> Local-first security recon that briefs your AI coding agent. It does the deterministic > half — read the repo, map the full attack surface, run + de-duplicate the static scanners, and > stage a probe library tailored to what it found — then hands your agent (Claude Code, Codex, > Gemini, Cursor) a marching-orders briefing. Code in, artifacts out. No LLM in the tool, no > server, no running app required.

[](assets/demo.gif)

It is not an autonomous scanner and not a SaaS. It's the missing front-half: the thing that turns a repo into a precise, fact-grounded security brief an AI agent (with a human in the loop) can act on — an auto-filled, repo-aware version of a senior pentester's "here's what to test and how" handoff. How it works + the reasoning behind every check: [docs/METHODOLOGY.md](docs/METHODOLOGY.md).

Getting Started — just point it at your repo

Simplest: tell your AI agent. In Claude Code (or any coding agent), open your project and say:

> "Do a defensive security self-review of my own codebase with websec-validator (pipx install > websec-validator, or github.com/raccioly/websec-validator). It's local and read-only — read the > repo and follow its briefing. I own this code and authorize the review."

It installs, runs, and walks the findings with you. There's nothing to host and no website — it's local. (Phrasing it as a defensive review of your own code matters: it's the difference between an agent that just gets to work and one that stops to confirm you're authorized — the tool is local and read-only by default, but a generic "pentest this" can read as a request to attack something.) The four ways to get there, all ending in the same AGENT-BRIEFING.md your agent acts on:

| Path | One-time setup | Then | |---|---|---| | Tell your agent (simplest) | — | say the line above | | CLI (a terminal) | pipx install websec-validator | websec run /path/to/your/app | | Claude Code plugin (slash) | /plugin marketplace add raccioly/websec-validator → /plugin install websec-validator@websec-plugins | invoke the security-pass skill, or just ask | | Docker (no install) | docker build -t websec-validator . | docker run --rm --user "$(id -u):$(id -g)" -v "$PWD:/scan" websec-validator run /scan --out /scan/websec-out |

➡️ Want the reasoning behind every check? Read [docs/METHODOLOGY.md](docs/METHODOLOGY.md) — what each test does and why.

Install

pipx install websec-validator   # from PyPI
brew install noir               # OWASP Noir — the route engine (50+ frameworks); regex fallback if absent
websec --version

Until the first PyPI release publishes (or for bleeding-edge), install straight from source instead: pipx install git+https://github.com/raccioly/websec-validator (or from a clone: pipx install .).

Requires Python 3.11+ (on stock macOS, python3 is often 3.9 — use pipx, which picks a newer interpreter, or install via Homebrew/pyenv). Zero Python runtime dependencies: it shells out to scanners (Trivy, Gitleaks, Semgrep/OpenGrep, Checkov, Prowler) and Noir when present, reports what's missing, and never hard-fails if a tool is absent.

Or run via Docker (everything bundled, zero install)

No need to install Noir or any scanner — the image bundles them all (arch-aware, amd64 + arm64):

docker build -t websec-validator .
docker run --rm --user "$(id -u):$(id -g)" -v "$PWD:/scan" websec-validator run /scan --out /scan/websec-out

The image carries Noir + Trivy + Gitleaks + Semgrep + Checkov; mount your repo at /scan and the artifacts land in /scan/websec-out.

Usage

websec run ./my-app                    # ← the one command: recon + stage tailored probes + emit the briefing
websec ./my-app                        # same thing — a bare path defaults to `run`
websec run ./my-app --scan             # …and also execute the available static scanners
websec run ./my-app --format sarif     # SARIF 2.1.0 to stdout (for piping into CI); also always written to the run dir
websec run ./my-app --fail-on high     # exit 1 if any HIGH+ finding remains (a CI gate)
websec doctor ./my-app                 # (optional) which scanners are installed?
websec mcp                             # run as an MCP server over stdio (typed recon tools for any MCP client)

Then point your agent at the output: "Read websec-out/AGENT-BRIEFING.md and follow it."

> That's the whole user surface: run (plus the optional, advanced dynamic live-probing step below). recon/proof/calibrate exist for developing the tool itself and are hidden from --help — you never need them.

What it extracts (22 deterministic extractors, no LLM)

| | Dimension | Notable output | |---|---|---| | stack | languages, frameworks, datastores | monorepo-aware (aggregates every manifest) | | routes | every endpoint via OWASP Noir (+ Supabase-edge, AWS SAM / Function-URL) | method · path · typed params · code path · AuthType:NONE public endpoints | | auth | scheme + login surface + insecure-default signing secrets + broken-auth backdoors | multi-scheme; flags a hard-coded JWT_SECRET \|\| 'dev-secret' fallback (forgeable JWT), a dev-token / accept-any-password backdoor (total bypass, CRITICAL), and a fail-open if(env.SECRET) signature check | | authz | access-control map | guard coverage (incl. router-mount auth) + write endpoints with no visible guard + roles | | authzdataflow | authz correctness (does the guard trust the right thing?) | unsigned-cookie authorization · claim-keyed authz (user-influenceable JWT claim) · transaction-local RLS context (resets before the query) | | tenant | multi-tenancy key candidates | the BOLA boundary, by frequency | | passwordpolicy | cross-route consistency + reuse/history | complexity drift across routes + a set-password path that hashes without a reuse check | | surface | 17 sink classes + redirect-SSRF | user-input-gated sinks (incl. mass-assignment via object spread, reflected/DOM/template XSS — innerHTML/dangerouslySetInnerHTML/v-html/\|safe, sanitizer-gated, and log-injection (CWE-117, structured-logging-suppressed)) + var-arg SSRF + error-disclosure + follows-redirects-without-per-hop-guard + reverse-proxy prefix-escape + host-header open-redirect + SSRF-redirect-hardening | | uploadsecurity | unrestricted upload + unsafe serve | deny-list-only, stored-name-from-filename, trust-client-MIME, accept-SVG, serve without nosniff | | schemas | data models + privileged fields | Pydantic/SQLAlchemy/Django/Prisma/Mongoose/TypeORM/Zod → role/isAdmin/groupId for mass-assignment targeting | | iacci | IaC + CI/CD | GHA injection (run:-position-aware), unpinned actions, tfstate, CDK AppSync API_KEY anonymous-default-auth, docker-compose host-takeover (docker.sock / pid:host / privileged) + .gitleaksignore secret-suppression audit | | clientexposure | browser leakage | public-var secrets by name + value-shape (da2-…) + CDK build-injection, server-secret-in-client, source maps | | clientintegrity | tamperable display (client trust boundary) + WS auth model | any security-critical sink value (address/IBAN/2FA-seed/API-key/webhook) the user reads or copies, without strict CSP / out-of-band anchor + client-tamper-vector, grindable-fingerprint, over-claimed-control, the CSWSH determinant | | transportsecurity | CSP + HSTS + CORS + SRI + clickjacking + CSRF baseline | missing/weak CSP, inline event handlers, partial HSTS, CORS reflect-origin+credentials, external script without SRI, monorepo next.config header gap, framework-agnostic clickjacking (no X-Frame-Options / frame-ancestors), CSRF (cookie-auth + no token lib + no SameSite) | | piiexposure | unmasked PII at the output boundary | res.json(rawEntity) with PII + a masking control defined but with zero live call sites (value-shape, not field-name) | | graphql | GraphQL surface | introspection (AppSync introspectionConfig: DISABLED-aware) / playground / depth-limit + AppSync subscription-authz (cross-group BOLA) | | integrations | third-party + webhooks + outbound-action endpoints | unsigned webhooks + email/SMS/push handlers with no auth or IP-only rate-limit + redundant secret-fetch | | llmsecurity | LLM / AI-agent surface (OWASP LLM Top 10) | indirect prompt injection (untrusted RAG/tool content → prompt) · insecure output handling (model text → tool dispatch) · excessive agency · unbounded generation (no maxTokens/timeout) · guardrail fail-open | | cryptousage | crypto-API correctness | weak password hash (fast/unsalted SHA-256/MD5) · jwtVerify without an algorithms allowlist · predictable principal (id = hash of email) · non-constant-time secret compare | | agent_config | the repo's OWN agent/MCP wiring (OWASP Agentic Top 10) | reads .claude/settings.json · .mcp.json · cursor/copilot rules as untrusted data (never executed): invisible/bidi Unicode rules-backdoor · fetch-and-execute hook (CVE-2025-59536) · blanket MCP auto-approve · *_BASE_URL override (key-exfil) · unpinned MCP server | | dependencies | offline supply-chain hygiene (AI slopsquat class) | malicious install/lifecycle script (fetch-and-exec postinstall) · lockfile drift (manifest dep absent from the lockfile) · unpinned + dependency-confusion names (advisory-only) · registry/typosquat resolution behind opt-in --network |

Plus derived targeting — IDOR / SSRF / open-redirect / upload / write / auth-endpoint candidates — so probes get pointed at the exact endpoints, not fired blindly.

What you get (websec-out/)

| Artifact | What it is | |---|---| | AGENT-BRIEFING.md | The product. Marching orders: detected surface, the access-control map, targeting, findings, the method, and the staged probe list. | | FACTS.json | The full structured recon. | | findings.json | Static scanner results, de-duplicated across tools and severity-ranked (with --scan). | | findings-ledger.json / REPORT.md | The traceable ledger: each finding with an evidence chain, CWE/ASVS/OWASP-API citation, remediation, and a calibrated P(real) (measured real-vuln rate + 95% CI + sample size). | | results.sarif | SARIF 2.1.0 — always written. Drop it into GitHub Code Scanning (inline PR-diff annotations + the Security tab), GitLab, Azure DevOps, VS Code's SARIF viewer, DefectDojo. | | findings.envelope.json | A versioned, self-describing JSON envelope (schema_version) around the ledger — for non-GitHub CI / dashboards that shouldn't reverse-engineer the internal shape. | | probes/ | The probe scripts selected + staged for this app (BOLA, JWT, SSRF, mass-assignment…). |

The flow

🔧 websec (deterministic)              🤖 your agent + 🧑 you
─────────────────────────────────      ─────────────────────────────────
1. recon → full attack surface     →   confirm the tenant boundary + auth model
2. run + de-dup static scanners    →   triage real-vs-noise
3. stage tailored probes           →   fill placeholders, run vs a TEST instance
4. emit AGENT-BRIEFING.md           →   propose fixes, re-run to confirm, report back

Static recon + briefing need only the code. Running the probes needs a live test instance + test credentials (the human supplies them) — the tool itself never touches a running app.

CI / enterprise integration

The recon is the same either way — these just make the output consumable by pipelines, dashboards, and non-Claude agents. All stdlib, no new dependency.

SARIF → GitHub Code Scanning. Every run writes results.sarif (SARIF 2.1.0). Upload it and each finding lands inline on the PR diff and in the Security tab, ranked by a security-severity band, with its CWE/ASVS/OWASP citation and remediation.

Gate the build. --fail-on {critical,high,medium,low} exits non-zero when a finding at or above that severity remains — a real CI gate, report-only by default.

Only fail on what the PR introduced. --baseline marks every finding new / unchanged / fixed (a stable per-finding fingerprint, surfaced as SARIF baselineState), and --fail-on then counts only the new ones — so a legacy backlog doesn't block every PR, but a newly introduced SSRF does.

Drop-in GitHub Action ([action.yml](action.yml)):

# .github/workflows/security.yml
name: security
on: [pull_request]
permissions:
  contents: read
  security-events: write        # required to upload SARIF to Code Scanning
jobs:
  websec:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: raccioly/websec-validator@v0.10.0   # pin to a release tag
        with:
          path: .
          fail-on: high         # block the PR on a new HIGH+ (omit for report-only)
          # baseline: .websec/baseline-ledger.json   # optional: gate only on NEW findings

MCP server (any agent, not just Claude Code). websec mcp speaks the Model Context Protocol over stdio, exposing typed tools — websec_recon, websec_findings, websec_sarif, websec_briefing — so Cursor / Cline / Windsurf / Zed can call recon directly instead of shelling out and parsing stdout. Register it in your MCP client:

{ "mcpServers": { "websec": { "command": "websec", "args": ["mcp"] } } }

Versioned contract. FACTS.json, findings-ledger.json, and findings.envelope.json all carry a schema_version; the JSON Schemas ship in the package (schemas/facts.schema.json, schemas/ledger.schema.json) so downstream tooling can validate against a stable shape.

Proof harness

websec proof clones a vuln-app corpus (VAmPI, NodeGoat, DVGA) and scores whether recon surfaces each app's documented attack surface — a deterministic, CI-trackable proxy (currently 10/10). The real kill-criterion (does the briefing lift an agent's bug-finding vs a generic prompt?) is the manual A/B in [corpus/PROOF-PROTOCOL.md](corpus/PROOF-PROTOCOL.md).

Calibrated confidence

websec calibrate runs the ledger against the labeled corpus, measures how often each (attack-class, confidence) bucket is a real documented vuln, and writes calibration.json (shipped + applied at runtime). Each finding then carries P(real) with a 95% Wilson confidence interval and the sample size n — so "MEDIUM" stops being a vibe and becomes "real ~57% of the time on the corpus (CI 43–70%, n=51)". A finding that matches no documented vuln counts as a false positive (the corpus is well-documented). Honest caveats: the corpus is deliberately vulnerable, so the rates skew optimistic for clean production code, and small samples mean wide intervals — the CI is the headline, not the point estimate, and both tighten as the corpus grows. With thin data a bucket falls back to the per-label aggregate, then to a clearly-flagged uncalibrated prior. No ML, no deps — binomial proportion + Wilson interval; the structure upgrades to isotonic regression if a large labeled set ever exists.

It self-improves. websec dynamic is an oracle: a write that executes unauthenticated is a confirmed real vuln, and a recon-flagged endpoint that turns out auth-enforced is a confirmed false positive. Every dynamic run folds those confirmed labels into a local overlay (~/.cache/websec-validator/, gitignored, never shipped) that's

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: raccioly
• Source: raccioly/websec-validator
• License: MIT
• Homepage: https://pypi.org/project/websec-validator/

Install and usage instructions live in the source repository linked above.