# Asyncopenai Concurrency Httpx Pool

> |

- **Type:** Skill
- **Install:** `agentstack add skill-kennethkhoocy-applied-micro-skills-asyncopenai-concurrency-httpx-pool`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [kennethkhoocy](https://agentstack.voostack.com/s/kennethkhoocy)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [kennethkhoocy](https://github.com/kennethkhoocy)
- **Source:** https://github.com/kennethkhoocy/applied-micro-skills/tree/main/plugins/applied-micro/skills/asyncopenai-concurrency-httpx-pool

## Install

```sh
agentstack add skill-kennethkhoocy-applied-micro-skills-asyncopenai-concurrency-httpx-pool
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# AsyncOpenAI Concurrency: the Hidden httpx Pool Cap

## Problem

Async batch scorers typically gate concurrency with `asyncio.Semaphore(N)`.
Raising N above ~100 silently does nothing: the OpenAI SDK's default httpx
transport caps the connection pool at `max_connections=100`, so excess tasks
queue inside httpx instead of reaching the provider. The semaphore looks like
the throttle but is not the binding constraint — there is no error, just a
throughput ceiling.

## Context / Trigger Conditions

- `asyncio.Semaphore(N)` with N > 100 around `client.chat.completions.create`
  shows the same throughput as N = 100
- Client constructed as `AsyncOpenAI(api_key=..., base_url=...)` with no
  `http_client` argument (the default transport)
- Provider is known to allow high concurrency (DeepSeek v4-flash: ~2500)
- Symptom check: requests-in-flight measured at the server never exceeds ~100

## Solution

Size the httpx pool to the semaphore when constructing the client:

```python
import httpx
from openai import AsyncOpenAI

CONCURRENCY = 2000
client = AsyncOpenAI(
    api_key=..., base_url="https://api.deepseek.com",
    http_client=httpx.AsyncClient(limits=httpx.Limits(
        max_connections=CONCURRENCY,
        max_keepalive_connections=CONCURRENCY)))
sem = asyncio.Semaphore(CONCURRENCY)
```

Both edits are required; either alone caps the other. Keep the per-request
retry loop — at high concurrency transient failures are more likely, and the
retry envelope is what turns them into non-events.

## Verification

Throughput scales with N. Verified 2026-07-17 on DeepSeek v4-flash
(deepseek-chat, JSON-mode unit scoring, ~1.5k-token prompts): at semaphore 50
a cold 13.8k-request chunk took ~70 min; at semaphore 2000 + matched pool, a
14.4k-request chunk took ~11 min (~40 req/s sustained, ~250/s burst on a
1.7k-request tail chunk), 0 failed requests, 0 schema-invalid responses.
Effective speedup ~7x rather than 40x — server-side queuing absorbs the rest —
but with zero reliability cost.

## Example

Specialist Directors US T1 campaign: final 8 chunks (100,243 calls) completed
in ~55 minutes for $8.08 after the fix, versus a projected ~9 hours at the old
setting. The edit is two lines in the scorer; the semaphore constant alone
would have been a silent no-op.

## Notes

- DeepSeek publishes no hard rate limit and handled 2000 in-flight cleanly;
  the practical ceiling reported is ~2500. Other providers enforce RPM/TPM
  caps — check before sizing.
- Windows: default asyncio proactor loop handled 2000 sockets without
  tuning; no ulimit-style adjustment needed.
- Companion ops lesson from the same campaign: when a run's *plan* changes
  scale (two-night legs -> one-shot), re-audit the launch glue's hardcoded
  limits — a wrapper `MAX-HOURS=10` safety cap sized for the old plan
  hard-killed a healthy runner at 97/108 chunks. Caps and budgets in
  supervisor scripts must be revisited whenever expected duration changes.
- Concurrency is a pure throughput knob: per-request outputs are unchanged
  (temperature 0, independent requests), so raising it mid-campaign does not
  create a scoring seam — unlike model/prompt changes, which do (see
  [llm-campaign-drift-gate]).

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [kennethkhoocy](https://github.com/kennethkhoocy)
- **Source:** [kennethkhoocy/applied-micro-skills](https://github.com/kennethkhoocy/applied-micro-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-kennethkhoocy-applied-micro-skills-asyncopenai-concurrency-httpx-pool
- Seller: https://agentstack.voostack.com/s/kennethkhoocy
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.