Agent Failure Preflight

Agent Failure-Mode Preflight

This skill turns Microsoft AI Red Team's Taxonomy of Failure Modes in Agentic AI Systems v2.0 — built from twelve months of red-team engagements against deployed agentic systems — into a pre-ship gate. It gives you the two-axis failure-mode map (novel vs. existing, safety vs. security), the seven failure modes added in v2.0, the five new mitigation families, and a copy-paste checklist to run before any agent deployment or material change. The taxonomy's own framing: it is a threat-modeling tool, not a compliance checklist — for each failure mode, ask whether it can occur in your system and whether a control would detect or prevent it.

When to use

Run this preflight when any of the following is about to ship:

• A new agentic system (anything that observes, plans, reasons, and acts against an environment).
• A new tool, plugin, MCP server, prompt template, or skill added to an existing agent — each is a supply-chain ingestion point.
• Persistent memory or RAG enabled, or memory scope expanded.
• A sub-agent, orchestrator, or any inter-agent messaging added.
• Computer-use (screenshot/click/type) capability enabled.
• Permission, tool, or network scope expanded; human-in-the-loop (HitL) flow changed.

Do NOT use this as a substitute for model-level safety evals (it assumes those exist), and do not treat passing it as compliance sign-off. Skip it for non-agentic single-turn LLM features with no tools, memory, or delegation — the taxonomy scopes pre-existing generative-AI failures in only where the agentic architecture adds a new element.

Rules

1. Map risks on both axes before testing anything. The taxonomy classifies failure modes as novel-vs-existing AND safety-vs-security. Cover all four quadrants: novel-security (agent compromise, agent injection, agent impersonation, agent flow manipulation, agent provisioning poisoning, multi-agent jailbreaks, plus the seven v2.0 additions), existing-security (memory poisoning and theft, targeted knowledge-base poisoning, XPIA, HitL bypass, function compromise, incorrect permissions, resource exhaustion, insufficient isolation, excessive agency, loss of data provenance), novel-safety (intra-agent responsible AI issues, harms of allocation in multi-user scenarios, organizational knowledge loss, prioritization leading to user safety issues), and existing-safety (insufficient transparency, parasocial relationships, bias amplification, user impersonation, insufficient intelligibility for meaningful consent, hallucinations, misinterpretation of instructions). Security-only reviews miss half the map.

1. Treat the seven v2.0 failure modes as mandatory coverage, not optional extras: agentic supply chain compromise, goal hijacking, inter-agent trust escalation, Computer Use Agent (CUA) visual attack, session context contamination, MCP/plugin abuse, capability/architecture disclosure. The source is explicit that attacks in this space — zero-click HitL bypass chains, inter-agent trust escalation, session context contamination, and disclosure used as an attack pivot — will not be surfaced by model-level evaluation alone; they require system-level testing with multi-step attack chains evaluated across complete task flows.

1. Test attack chains, not isolated failures. AIRT's operational finding is that failure modes rarely live in isolation; compound chains and combined effects are the norm. XPIA and memory poisoning were observed at high frequency and frequently combined: one successful injection seeds persistent memory, and the agent itself propagates it across subsequent sessions.

1. Gate HitL design hardest — it was the most consistently exploited failure mode, including zero-click end-to-end chains starting from external input (no human interaction beyond initial invocation) that achieved exfiltration and lateral movement. Observed bypass patterns: consent fatigue, manipulation of probabilistic invocation, and incremental escalation chains where no single step warranted review but the compound outcome did. Required controls (consent architecture hardening): deterministic invocation — the agent must never decide whether HitL fires; compound-action decomposition — show every consequential sub-action, not just the headline goal; semantic summarization of approval descriptions generated from the underlying tool calls, never the agent's own verbatim description (prevents description laundering); tiered approval scaling with action reversibility and blast radius — irreversible or cross-tenant actions get a higher tier; anomaly detection on approval-request frequency and pattern. The source's line: "UX design is a security control."

1. Treat every natural-language component as supply chain. Unlike traditional supply-chain compromise, a poisoned plugin, MCP server, prompt template, or tool description injects instructions without touching any binary, and one compromised component alters every agent that consumes it. Context for urgency, from the source: OpenClaw (launched January 2026) hit 336K GitHub stars and 2,100 deployed agents inside 48 hours; the audit that followed turned up 512 vulnerabilities, 8 of them critical, over 1,800 instances leaking API keys within the first week, and 336 malicious plugins in the ClawHub marketplace — and 2025 alone produced 99 CVEs against MCP-related software. Required controls: SBOM covering plugins, MCP servers, prompt templates, and tool descriptions alongside code dependencies; signature and provenance verification before installation; registry scans for hidden instructions in tool descriptions (not only malicious code); pinned versions of all external tool definitions with change monitoring — even "patch" bumps can alter natural-language behavior.

1. Establish agent identity cryptographically, never positionally. Inter-agent trust escalation is a confused-deputy problem induced through natural language instead of system calls: a compromised sub-agent asserts false identity or inflated permissions and an unverifying orchestrator acts on the claim. Issue each agent an attestable credential at provisioning, bind it to all outgoing messages and tool calls, require orchestrators to verify the credential chain of every inbound claim before privilege decisions, and never let sub-agents elevate privileges via self-description.

1. Harden memory as a security boundary. Memory poisoning needs only a single successful injection to persist across sessions. Apply integrity, isolation, and provenance controls to persistent memory stores, including validation of memory reads against their declared source.

1. Treat accumulated session context as a security-relevant data structure, not a scratchpad. Session context contamination — adversarial data introduced early that biases later reasoning without tripping per-step safety controls — was found difficult to detect; the compound effect only manifests across the session. Controls (adversarial session hardening): context-provenance tracking (source-tag every token: trusted system prompt, user turn, retrieved document, tool response, inter-agent message); structured separation of trusted system context from untrusted retrieved content with explicit policy on which classes may influence which decisions; session-integrity monitoring for anomalous accumulation patterns (e.g., rapid amplification of one retrieved document's framing); bounded session contexts capping external-content influence; policies that limit certain tool calls once external data has entered the context.

1. Treat internal architecture as a confidentiality boundary. Capability/architecture disclosure (tool schemas, system-prompt structure, memory schema, command aliases) converts black-box probing into white-box attack planning — in red-team operations it was the pivot between an initial XPIA foothold and full memory or HitL bypass, and often required only a direct request. Controls: refusal patterns for tool-list/system-prompt/schema/command-alias introspection applied uniformly to user turns, retrieved content, inter-agent messages, and tool responses; output-side filtering for tool-name, schema, and system-prompt fingerprints on ALL outbound content including memory writes and inter-agent messages; architectural opacity — resolve tools at runtime from a non-disclosable registry rather than embedding the inventory in the system prompt; capability minimization so leaked architecture is worth less.

1. If the agent sees pixels, pixels are a control channel. CUA visual attacks — hidden text, off-viewport elements, low-contrast banners, faux modal dialogs, adversarial alt-text, ads mimicking legitimate "Approve" buttons — carry instructions invisible to human reviewers, and conventional prompt-input hardening does not apply because the agent's control channel is the same visual field the user relies on.

1. For multi-agent systems, threat-model the network, not just each agent. Network-level failure patterns from the source: emergent objectives and coalition drift; social engineering between agents (authority cues, reciprocity, reputation claims as machine-readable attack primitives); reputation/trust-layer poisoning of ratings and allowlists; contagion via shared artifacts (plans, summaries, cached memories acting as propagating payloads); runaway delegation and coordination cascades where a small perturbation triggers broad re-planning, resource exhaustion, or widespread policy violations before oversight can intervene; information laundering across agent boundaries; self-reinforcing feedback loops where agent consensus substitutes for provenance.

1. Build supply-chain and zero-trust controls in at design time. The source is explicit that these are architectural decisions difficult to retrofit: SBOM generation, tool-provenance verification, and inter-agent authentication cost substantially less built in from the start than added after deployment.

1. Remember hallucinations compound in agents. A hallucinated file path, permission, or API response feeds straight into the agent's next action — "hallucination risk is multiplicative rather than additive" (Microsoft AI Red Team). Verify consequential inputs before acting on them.

Checklist

Run before deploy. Every unchecked box is an open finding — attach an owner or an accepted-risk note.

Supply chain gate

• [ ] SBOM exists covering plugins, MCP servers, prompt templates, and tool descriptions — not just code deps
• [ ] Signature/provenance verified for every MCP server and plugin before installation
• [ ] Registries and tool descriptions scanned for hidden natural-language instructions
• [ ] All external tool definitions version-pinned; change monitoring alerts on any bump, including patch

Identity gate (multi-agent)

• [ ] Each agent holds an attestable credential issued at provisioning, bound to all messages and tool calls
• [ ] Orchestrator verifies credential chains before any privilege decision; role/permission claims are never accepted from self-description
• [ ] Tool layers reject agent-supplied principal claims that lack verification

Memory gate

• [ ] Persistent memory has integrity, isolation, and provenance controls; reads validated against declared source
• [ ] A single poisoned write cannot silently persist across sessions (tested via XPIA → memory-write chain)

HitL / consent gate

• [ ] HitL invocation is deterministic — the agent cannot decide whether approval is requested
• [ ] Approval UI decomposes compound actions into every consequential sub-action
• [ ] Approval descriptions are summarized from underlying tool calls, never the agent's own wording
• [ ] Approval tiers scale with reversibility and blast radius; irreversible/cross-tenant actions require the higher tier
• [ ] Anomaly detection watches approval-request frequency and composition
• [ ] A zero-click chain (external input → action, no human turn) has been red-teamed end to end

Session gate

• [ ] Every context token carries a source tag (system prompt / user turn / retrieved doc / tool response / inter-agent message)
• [ ] Policy defines which content classes may influence which decisions; trusted and untrusted content are structurally separated
• [ ] High-impact tool calls are restricted once untrusted external content has entered the context
• [ ] Session-integrity monitoring flags anomalous accumulation (one document's framing amplifying across steps)

Disclosure gate

• [ ] Introspection requests (tool list, system prompt, schemas, command aliases) are refused uniformly across all input channels
• [ ] Outbound filtering covers memory writes and inter-agent messages, not only user-visible turns
• [ ] Tool inventory is resolved at runtime from a registry, not embedded verbatim in the system prompt

CUA gate (if the agent operates a GUI)

• [ ] Tested against hidden text, off-viewport elements, low-contrast banners, faux dialogs, adversarial alt-text, and look-alike "Approve" buttons

Network gate (if more than one agent)

• [ ] Red-teaming ran against the agent network, not only individual agents
• [ ] Shared artifacts (plans, summaries, cached memories) are treated as potential propagating payloads
• [ ] Delegation depth/fan-out is bounded to contain coordination cascades
• [ ] Agent consensus is never accepted as a substitute for provenance

Coverage gate

• [ ] Red-team matrix includes all seven v2.0 categories: agentic supply chain compromise, goal hijacking, inter-agent trust escalation, CUA visual attack, session context contamination, MCP/plugin abuse, capability/architecture disclosure
• [ ] Tests run as multi-step chains across complete task flows, not single-step probes
• [ ] Safety quadrants reviewed too: allocation harms, knowledge loss, transparency, consent intelligibility, bias amplification

Anti-patterns

• Relying on output review to catch goal hijacking — hijacked agents keep completing sub-tasks and passing plausibility checks while optimizing for the adversary's objective; detection by output alone is unreliable.
• Displaying the agent's own description of an action in the approval prompt (enables description laundering).
• Letting the agent decide when HitL controls fire (probabilistic invocation is a documented bypass vector).
• Inferring agent identity from position in a workflow instead of a verifiable credential.
• Treating tool descriptions and prompt templates as inert metadata rather than supply-chain attack surface.
• Assuming model-level evals cover agentic risk — zero-click HitL bypass, trust escalation, session contamination, and disclosure pivots only appear in system-level, multi-step testing.
• Leaving tool definitions unpinned because "it's just a patch release" — patch bumps can change natural-language behavior.
• Embedding tool names, parameter schemas, or memory-record structure verbatim in the system prompt.
• Treating "describe yourself" requests as low-risk; the source says treat ambient introspection instructions as high-risk.
• Depending on per-step safety checks to catch session-level contamination — neither the contaminating input nor any single escalation step is anomalous in isolation.
• Accepting "multiple agents agree" as evidence — repetition is not provenance.
• Deferring SBOM, provenance verification, and inter-agent authentication to post-launch; they are architectural and expensive to retrofit.

Source

• Updating our taxonomy of failure modes: What a year of red teaming agentic AI systems taught us (Microsoft Security Blog) — https://www.microsoft.com/en-us/security/blog/2026/06/04/updating-taxonomy-failure-modes-agentic-ai-systems-year-red-teaming-taught-us/ — 2026-06-04
• Taxonomy of Failure Modes in Agentic AI Systems, v2.0 (Microsoft AI Red Team whitepaper, linked from the post) — https://cdn-dynmedia-1.microsoft.com/is/content/microsoftcorp/microsoft/bade/documents/products-and-services/en-us/securit

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Archive228
• Source: Archive228/lab-skills
• License: MIT

Install and usage instructions live in the source repository linked above.