# Async Patterns

> Designs asynchronous workflows - message queues, event-driven architecture, pub/sub, background jobs, webhooks, and eventual consistency. Use when building systems that process work asynchronously, communicate via events, or need to decouple producers from consumers.

- **Type:** Skill
- **Install:** `agentstack add skill-pvnarp-agent-skills-async-patterns`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [pvnarp](https://agentstack.voostack.com/s/pvnarp)
- **Installs:** 0
- **Category:** [Cloud & Infrastructure](https://agentstack.voostack.com/c/cloud-infrastructure)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [pvnarp](https://github.com/pvnarp)
- **Source:** https://github.com/pvnarp/agent-skills/tree/main/skills/async-patterns

## Install

```sh
agentstack add skill-pvnarp-agent-skills-async-patterns
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Async Patterns

Synchronous = caller waits for result. Asynchronous = caller sends work and moves on. Use async when: the work is slow, the caller doesn't need the result immediately, or you need to decouple systems.

## When to Go Async

| Situation | Sync or Async? |
|-----------|---------------|
| User requests data, needs it now | Sync |
| Send confirmation email after signup | Async |
| Process payment (user waiting for result) | Sync (with timeout) |
| Generate PDF report | Async (return job ID, poll for result) |
| Notify 5 services about an event | Async (pub/sub) |
| Resize uploaded image | Async |
| Real-time API response | Sync |
| Analytics event recording | Async (fire-and-forget) |

## Core Patterns

### 1. Work Queue (Task Queue)

```
Producer → [Queue] → Consumer(s)
```

One message, one consumer. Load distributed across workers.

**Use for:** Background jobs, email sending, image processing, PDF generation.

```
Producer enqueues: { type: "send_email", to: "user@x.com", template: "welcome" }
Consumer picks up job, processes, acknowledges.
If consumer crashes → message redelivered to another consumer.
```

**Key decisions:**
- At-least-once delivery (default, safe) vs at-most-once (lossy, fast)
- Retry policy: how many times? Exponential backoff?
- Dead letter queue: where do permanently failed messages go?
- Concurrency: how many consumers? Rate limiting?

### 2. Pub/Sub (Fan-Out)

```
Publisher → [Topic] → Subscriber A
                    → Subscriber B
                    → Subscriber C
```

One message, multiple consumers. Each gets a copy.

**Use for:** Event notification, audit logging, cross-service sync.

```
Order service publishes: { event: "order.placed", order_id: "123", amount: 5000 }
  → Inventory service: reserves items
  → Email service: sends confirmation
  → Analytics service: records event
```

**Key decisions:**
- Subscribers are independent - one failing doesn't block others
- Message ordering: guaranteed per partition/key? Or unordered?
- Subscriber lag: what happens when a consumer falls behind?

### 3. Request-Reply (Async RPC)

```
Requester → [Request Queue] → Worker
         ← [Reply Queue]   ←
```

**Use for:** Long-running operations where the caller eventually needs the result.

```
1. Client POSTs /reports → returns { job_id: "abc", status: "pending" }
2. Worker processes the report
3. Client polls GET /reports/abc → { status: "completed", url: "..." }
   (or receives a webhook callback)
```

### 4. Event Sourcing

```
Commands → Event Store → [event1, event2, event3, ...] → Projections (read models)
```

Store events (facts that happened), not current state. Rebuild state by replaying events.

**Use for:** Audit-critical systems, financial transactions, collaborative editing.

**Caution:** High complexity. Only use when you genuinely need full event history and the ability to rebuild state at any point in time.

## Message Design

### Message Structure
```json
{
  "id": "msg_abc123",
  "type": "order.placed",
  "version": 1,
  "timestamp": "2024-03-15T10:23:45Z",
  "source": "order-service",
  "correlation_id": "req_xyz789",
  "data": {
    "order_id": "ord_123",
    "user_id": "usr_456",
    "total_cents": 5000
  }
}
```

**Rules:**
- Every message has a unique `id` (for deduplication)
- Every message has a `type` and `version` (for evolution)
- Include `correlation_id` for tracing through async flows
- Include `timestamp` for ordering and debugging
- `data` contains the payload - keep it self-contained

### Idempotency

Messages WILL be delivered more than once. Your consumer MUST handle duplicates.

```
Consumer receives message with id: "msg_abc123"
  → Check: have I already processed "msg_abc123"?
    YES → skip (already done)
    NO  → process, then record "msg_abc123" as processed
```

Store processed message IDs in a database with a TTL. Or design operations to be naturally idempotent (SET is idempotent, INCREMENT is not).

## Failure Handling

### Dead Letter Queue (DLQ)
Messages that fail after all retries go to a DLQ instead of being lost.

```
Queue → Consumer → fails → retry 1 → retry 2 → retry 3 → DLQ
```

**DLQ rules:**
- Monitor DLQ size (alert if > 0)
- Include original error in DLQ message metadata
- Build tooling to replay DLQ messages after fixing the bug
- Set retention (don't keep dead letters forever)

### Poison Messages
A message that crashes the consumer every time it's processed.

```
Detection: same message redelivered N times
Action: move to DLQ, alert, continue processing other messages
Prevention: validate message schema before processing
```

### Consumer Lag
When consumers can't keep up with producers.

```
Detection: queue depth growing over time
Actions:
  1. Scale consumers horizontally
  2. Batch process (consume N messages at once)
  3. If acceptable: shed load (drop low-priority messages)
  4. Investigate: is the consumer doing unnecessary work?
```

## Eventual Consistency

In async systems, data across services is NOT immediately consistent. It's eventually consistent.

```
User places order (Order service: order = PLACED)
  → 50ms later → Inventory service: items reserved
  → 200ms later → Email service: confirmation sent
  → 2s later → Analytics: event recorded
```

**Handling inconsistency:**
- Show the user their local state immediately (optimistic UI)
- Design reads to tolerate stale data ("last updated 5s ago")
- Use compensating actions for failures (if payment fails after order created → cancel order)

> Reference: `reference/queue-patterns.md` - Queue comparison tables, delivery guarantees, backpressure strategies, idempotency patterns.

## Technology Choices

| Tool | Type | Best For |
|------|------|----------|
| Redis (Streams/Lists) | Queue | Simple queues, low latency, volatile |
| RabbitMQ | Queue + Pub/Sub | Flexible routing, reliable delivery |
| Apache Kafka | Event log | High throughput, event sourcing, replay |
| AWS SQS | Queue | Managed, simple, no ops |
| AWS SNS + SQS | Pub/Sub | Managed fan-out |
| NATS | Pub/Sub | Lightweight, fast, cloud-native |
| BullMQ / Celery / Sidekiq | Job queue | Application-level background jobs |
| Database (polling) | Queue | Already have a DB, low volume, don't want new infra |

**Decision guide:**
- < 1000 msgs/sec, simple jobs? → Database polling or Redis
- Need reliable delivery, routing? → RabbitMQ
- High throughput, event replay, multiple consumers? → Kafka
- Don't want to manage infrastructure? → SQS/SNS
- Already have Redis? → BullMQ/Redis Streams

## Anti-Patterns

| Anti-Pattern | Problem | Fix |
|-------------|---------|-----|
| Async for everything | Unnecessary complexity, hard to debug | Start sync, go async only when you have a reason |
| No idempotency | Duplicate processing, double charges | Store processed message IDs, design idempotent operations |
| No DLQ | Failed messages disappear forever | Always configure a dead letter queue |
| Synchronous mindset | Blocking on async result, negating the benefit | Return job ID, let client poll or receive callback |
| Giant messages | Queue becomes a bottleneck, consumers slow | Put data in storage, put reference in message |
| No monitoring | Queue backs up silently | Alert on queue depth, consumer lag, DLQ size |
| Ordering assumptions | Messages arrive out of order, logic breaks | Design for out-of-order, or use partitioned ordering |

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [pvnarp](https://github.com/pvnarp)
- **Source:** [pvnarp/agent-skills](https://github.com/pvnarp/agent-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-pvnarp-agent-skills-async-patterns
- Seller: https://agentstack.voostack.com/s/pvnarp
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.