Chaos Test

Chaos Test — Resilience Gap Analysis

Scan the codebase for failure modes that would surface under real-world stress: network partitions, dependency outages, slow consumers, and burst traffic.

Usage

/chaos-test                  # full scan of staged changes and key integration points
/chaos-test            # target a specific file or directory
/chaos-test --service  # focus on a named external service dependency

Behavior

Step 1 — Identify integration boundaries

# Find outbound HTTP calls, queue producers/consumers, DB calls, cache hits
grep -rn "fetch\|axios\|http\.\|requests\.\|redis\|pg\.\|mysql\|amqp\|kafka" \
  --include="*.ts" --include="*.py" --include="*.go" --include="*.js" \
  -l . | head -30

Step 2 — Check each boundary for resilience controls

For every integration point found, verify presence of:

| Control | What to grep for | Risk if absent | |---|---|---| | Timeout | timeout, AbortController, signal | Hangs under slow upstream | | Retry | retry, backoff, attempt | Single transient failure = hard failure | | Circuit breaker | opossum, cockatiel, breaker, circuitbreaker | Cascading failure under sustained outage | | Fallback / degraded mode | fallback, default, catch returning a stub | No graceful degradation | | Rate-limit handling | 429, RateLimitError, retry-after | Thundering herd kills upstream | | Bulkhead / concurrency cap | semaphore, pLimit, asyncio.Semaphore | Single slow dep starves thread pool |

Step 3 — Score each boundary

For each integration point, assign a risk tier:

• Critical — no timeout AND no retry AND no fallback
• High — missing two of the three controls above
• Medium — missing one control
• Low — all controls present; note any configuration concerns (timeout too high, retry without jitter, etc.)

Step 4 — Report findings

Output a prioritized list:

CHAOS ANALYSIS — 

Critical (fix before next deploy)
  src/payments/stripe.ts:42   POST /charges — no timeout, no retry, no fallback
  src/auth/token.ts:88        Redis GET — no timeout; outage causes auth failure

High
  src/search/elastic.ts:17    ES query — no circuit breaker; slow queries block response

Medium
  src/email/sendgrid.ts:31    Retry present but no jitter (thundering herd risk)

Low
  src/cache/redis.ts:55       All controls present; timeout at 30s is high — consider 5s

Suggested experiment targets (fault to inject → expected behavior → gap if wrong):
  1. Kill Redis → auth should degrade to DB lookup → currently: hard 500
  2. Throttle Stripe to 2s → checkout should timeout at 1s → currently: no timeout set

Step 5 — Suggest quick wins

For each Critical/High finding, propose the minimal fix:

stripe.ts:42 — Add AbortSignal timeout (5s) and exponential-backoff retry (3 attempts, 100ms base, jitter):
  const res = await fetch(url, { signal: AbortSignal.timeout(5000) })

Edge Cases

• No external dependencies: Report "No integration boundaries detected" and exit.
• Monorepo: Scan from $ARGUMENTS path; skip test files unless --include-tests passed.
• gRPC / GraphQL: Detect grpc, graphql-request, and apollo-client as integration points.
• Already uses resilience library: Note which library is in use and check it is configured, not just imported.

Token Optimization

Expected range: 600–2,000 tokens (grep-driven discovery); 100–300 tokens (no integrations found, early exit)

Patterns used: Grep-before-Read (scan for integration points before reading any file), git diff scope (staged changes first), progressive disclosure (summary → details on request)

Early exit: If git diff --staged --name-only returns no files and no $ARGUMENTS path is given, report "No staged changes to analyze" and exit.

Caching: Caches file list from boundary scan in .claude/cache/chaos-test/boundaries.json (invalidated when package.json, go.mod, or requirements.txt changes).

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: manastalukdar
• Source: manastalukdar/ai-devstudio
• License: MIT

Install and usage instructions live in the source repository linked above.