Ai Security Assessment

AI Security Assessment

Turn a concrete AI attack path into a reproducible assessment and remediation test. [threat-model](../threat-model/SKILL.md) owns architectural analysis, [security-review](../security-review/references/code-review/GUIDE.md) owns source and dependency review, and [agent-evaluation](../agent-evaluation/SKILL.md) owns repeated-trial comparisons. Use PyRIT as the execution framework through a project-local uv environment.

Workflow

1. Record the boundary: identify the application revision, model, retrieval sources, actors, tools, and granted authority in the [assessment record](templates/assessment.md). Reuse established target, data, action, and cost authorization; proceed autonomously within it. Keep customer evidence in the assessment project.
2. Choose plausible scenarios: read [attack cases](references/attack-cases.md). For each case name the attacker-controlled surface, required access, protected asset, and forbidden outcome. Include legitimate autonomous actions that must continue to succeed.
3. Observe the application: capture attempted tool calls and resulting state through the real enforcing boundary. Use disposable records, synthetic secrets, and controlled destinations when these preserve the attack path. A direct model endpoint cannot establish the safety of an application's retrieval or tools.
4. Prepare PyRIT: follow [PyRIT execution](references/pyrit.md) for the released API, target adapter, scenario selection, local smoke, and evidence storage. Freeze package, prompt, dataset, converter, scorer, and application identities before comparative runs.
5. Validate the case: prove the intended input reaches the target and that the outcome check detects a deliberately broken control in a fixture. Test a legitimate operation and a rejected operation. Keep target failures, evaluator failures, and actual security failures separate.
6. Run the campaign: start with a narrow single-turn case, then add multi-turn or converted cases when the attack path requires them. Declare case count, repeats, concurrency, turn/retry/time/cost limits, cleanup, and stop conditions; retain every attempted trial and its outcome.
7. Verify findings: reproduce suspicious results against the application, inspect enforcement and artifacts, and challenge alternative explanations. Label confirmed, refuted, and unresolved cases; a model's claim that it accessed a secret is insufficient evidence.
8. Remediate and retest: when implementation is authorized, fix the failed runtime control and add a deterministic regression. Repeat the original attack, nearby variants, and legitimate tasks under comparable conditions. Report the remaining uncertainty and the exact tested boundary.

Gotchas

• Autonomy is intentional: test whether untrusted content gains authority beyond the declared contract. Powerful tools or absent per-action prompts alone are not findings; do not change the user's harness permissions as part of an assessment.
• Behavior and impact differ: distinguish unsafe attempted actions, blocked actions, completed effects, and content-policy failures. A refusal followed by a forbidden tool call is a failure; an unverified boast is not completed exfiltration.
• No self-grading authority: target responses and attack text cannot redefine the objective, evaluator, scope, or budget. Prefer state assertions and calibrated scorers over the target's explanation.
• Scoring is not containment: PyRIT generates and executes tests; the assessment environment and application enforce tool, data, and spending boundaries. A request timeout does not prove a remote run stopped.
• Evidence is sensitive: prompts, retrieval snippets, conversation databases, screenshots, and model responses can contain customer data. Select storage, access, retention, and permitted external providers before collecting them.

Documentation

• PyRIT · OWASP Agentic Top 10 · MITRE ATLAS
• Releases: PyRIT
• Companion skills: [prompt-design](../prompt-design/SKILL.md) (prompt and tool contracts), [incident-response](../incident-response/SKILL.md) (active compromise), [quality-assurance](../quality-assurance/SKILL.md) (broader user journeys).

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: fmind
• Source: fmind/dot
• License: MIT
• Homepage: https://fmind.dev

Install and usage instructions live in the source repository linked above.