Security Regression Tests

Security Regression Tests

Lock every confirmed security fix with a deterministic regression test so CI re-verifies it on every build (no live probing)

ALWAYS

• For every confirmed and fixed security finding, add a regression test that pins the secure behaviour, and commit it. The test is the durable, CI-side verification — it re-checks the fix on every build, unlike a one-off live probe.
• Structure each test as attack + control: the attack input now yields the secure outcome (403/404, rejected, escaped, not evaluated, no exec), AND a legitimate input still succeeds. Both assertions matter — the attack half guards the vulnerability; the control half catches an over-correction that breaks the feature.
• Map the test to the finding's class:
• IDOR/BOLA → authenticate as A, request B's resource id → 403/404.
• Broken auth → forged / expired / none-alg token → 401.
• Mass assignment → POST an extra privileged field → it is ignored, never persisted.
• SQLi / XSS / SSTI → the payload is parameterized / escaped / not evaluated.
• Secret → the secret is absent from the built artifact and loaded from the env.
• Prefer the smallest tier that proves it: a unit test on the handler/guard covers most authz/validation; use an integration test only when the bug lives at a boundary (routing, middleware order, ORM).
• Name and place the test so a reviewer sees it guards a security fix (test_idor_orders_cross_tenant_403) and reference the finding.

NEVER

• Close a confirmed finding without a regression test — "fixed by inspection" rots; the next refactor silently reintroduces it.
• Assert only the happy path — without the attack assertion the test does not guard the bug at all.
• Put a live-target probe inside CI as the verification. CI must be deterministic and offline; the regression test replaces the probe. Live probing stays an in-session, agent-driven activity ([[dynamic-verification]]).
• Hardcode a real secret/token into a fixture to exercise a secret check — use an obviously-fake sentinel.

KNOWN FALSE POSITIVES

• A refuted candidate (not a real bug) needs no regression test — do not add tests for non-issues.
• Config / infra findings (a Dockerfile USER, a workflow permission, a bad dependency) are locked by the scanner gate in CI (secure-vibe audit --fail-on), not a unit test — there, the gate is the regression check.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: ShieldNet-360
• Source: ShieldNet-360/secure-vibe
• License: MIT

Install and usage instructions live in the source repository linked above.