# Vobiz Audio Streams

> Stream live call audio bidirectionally over WebSocket for AI voice agents, real-time transcription, and custom audio pipelines.

- **Type:** Skill
- **Install:** `agentstack add skill-vobiz-ai-agent-skills-vobiz-audio-streams`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [vobiz-ai](https://agentstack.voostack.com/s/vobiz-ai)
- **Installs:** 0
- **Category:** [Content & Media](https://agentstack.voostack.com/c/content-and-media)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [vobiz-ai](https://github.com/vobiz-ai)
- **Source:** https://github.com/vobiz-ai/Agent-Skills/tree/main/skills/vobiz-audio-streams

## Install

```sh
agentstack add skill-vobiz-ai-agent-skills-vobiz-audio-streams
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Vobiz Audio Streams skill

Use this for **AI voice agents, real-time transcription, custom STT/TTS pipelines, or any workflow that needs raw audio frames** rather than just recordings.

## How it works

1. During a call, your VobizXML answer handler returns a `wss://your-server/ws` verb.
2. Vobiz opens a WebSocket to your server and sends a single `start` event with `callId`, `streamId`, and `mediaFormat`.
3. Audio frames flow Vobiz → your server as `media` events (~50 per second per track, 20 ms each).
4. For bidirectional streams, your server sends `playAudio` events back, plus `checkpoint`, `clearAudio`, and `stop` control messages.
5. When the call ends, the WebSocket **closes** - there is no inbound `stop` event.

## Two ways to start a stream

| Approach | When to use |
|---|---|
| **`` XML verb** | The call is being set up by your answer handler. Best for AI agents that own the call from the first ring. Bidirectional playback requires this path. |
| **REST `POST .../Stream/`** | The call is already live and you want to attach (or fork) audio mid-call - e.g. start transcription after a transfer. Returns a `stream_id` you can later list/get/stop. |

REST streams are forks: `bidirectional=true` over REST still requires `audio_track=inbound`. For full duplex voice agents, prefer the XML verb.

## REST endpoints

Base: `https://api.vobiz.ai/api/v1`. Auth headers: `X-Auth-ID`, `X-Auth-Token`.

| Op | Method + Path | Notes |
|---|---|---|
| Start stream | `POST /Account/{auth_id}/Call/{call_uuid}/Stream/` | Body needs `service_url`; returns `stream_id` (202). |
| List streams | `GET /Account/{auth_id}/Call/{call_uuid}/Stream/` | Active + stopped, paginated (`limit`/`offset`). |
| Get stream | `GET /Account/{auth_id}/Call/{call_uuid}/Stream/{stream_id}/` | Full Stream object; 404 if unknown. |
| Stop one stream | `DELETE /Account/{auth_id}/Call/{call_uuid}/Stream/{stream_id}/` | 204; leaves other forks running. |
| Stop all streams | `DELETE /Account/{auth_id}/Call/{call_uuid}/Stream/` | 204; tears down every fork on the call. |

## Audio format

- `contentType` (XML) / `content_type` (REST): `audio/x-l16;rate=8000` (default), `audio/x-l16;rate=16000`, `audio/x-l16;rate=24000`, or `audio/x-mulaw;rate=8000`.
- L16 is 16-bit linear PCM, mono. **On the wire `media.payload` is network byte order (big-endian)** - swap to little-endian before writing a WAV. `playAudio` you send should be little-endian L16.
- µ-law is fixed at 8 kHz, mono. Recommended for carrier compatibility and lowest bandwidth.
- 20 ms frame = 160 bytes µ-law @ 8 kHz, 320 bytes L16 @ 8 kHz, 640 bytes L16 @ 16 kHz.
- Frames are base64-encoded inside `media.payload`. Never send WAV headers - raw samples only.
- The format Vobiz reports in `start.mediaFormat` is the format you must match in every `playAudio`.

## WebSocket events

**Vobiz → your server**

- `start` - one handshake frame. IDs live inside the nested `start` object: `data.start.streamId`, `data.start.callId`, `data.start.mediaFormat`.
- `media` - base64 audio frame; `media.track`, `media.timestamp`, `media.chunk`, `media.payload`.
- `playedStream` - ack that audio before your `checkpoint` finished playing. Shape is just `event` + `name` (no `streamId` on the WS event).
- `clearedAudio` - ack that `clearAudio` flushed the playback queue.

**Your server → Vobiz** (require `bidirectional="true"`)

- `playAudio` - queue a ~20 ms chunk: `{ event, media: { contentType, sampleRate, payload } }`.
- `checkpoint` - mark end of an utterance: `{ event, streamId, name }`; ack arrives as `playedStream`.
- `clearAudio` - barge-in flush: `{ event, streamId }`; ack arrives as `clearedAudio`.
- `stop` - end the stream from your side: `{ event, streamId }`. Vobiz then runs the next XML element, or hangs up.

## HTTP status callbacks (separate channel)

If you set `statusCallbackUrl`, Vobiz POSTs form-encoded lifecycle events - distinct from WebSocket events:

- `Event=StartStream` - WebSocket established (carries `ServiceURL`).
- `Event=PlayedStream` - checkpoint reached (carries `Name`); only fires if playback completed.
- `Event=StopStream` - stream ended. **Only reliably fires for server-initiated stops** - not for caller-hangup or mid-call kill.

The authoritative "call is over" signal across all paths is the `Event=Hangup` POST to your call's `hangup_url`.

## When to search docs

- "Start mid-call over REST" → `audio-streams/start-audio-stream`; Stream fields → `audio-streams/stream-object`
- "Stop one fork vs all forks" → `audio-streams/stop-audio-stream`, `audio-streams/stop-all-audio-streams`
- "`` attributes / status callbacks / maxRetries" → `xml/stream`
- "Handshake, codecs, end-of-stream detection" → `xml/stream/initiate`
- "Full event protocol both directions" → `xml/stream/stream-events`
- "Playing audio mid-stream" → `xml/stream/play-audio`
- "Clearing queued TTS on interruption / barge-in" → `xml/stream/clear-audio`
- "Checkpoint / playback acks" → `xml/stream/checkpoint-event`
- "Ending the stream from your side" → `xml/stream/stop-event`
- "Pipe audio to Deepgram / OpenAI Realtime / Pipecat / LiveKit" → `integrations/openai-realtime`, `integrations/websockets`, `integrations/pipecat`, `integrations/livekit`

## Pitfalls

- **No inbound `stop` event.** When the call ends, the WebSocket simply **closes** after the last `media` frame. Do not wait for `{ "event": "stop" }` from Vobiz - you will wait forever. Treat WS `close` as your end-of-stream signal and flush buffers there.
- **`StopStream` callback is conditional.** It fires reliably only when *you* send an outbound `stop`. It is *not* observed for caller-hangup or mid-call kills. Use the `hangup_url` `Event=Hangup` webhook as the authoritative end-of-call signal.
- **IDs are nested.** Read `data.start.streamId` / `data.start.callId`, not `data.streamId`, on the `start` event.
- **`playedStream` is conditional.** No ack arrives if a `clearAudio` or disconnect interrupted the queued audio. Always have a timeout fallback - never block conversation logic on a checkpoint ack.
- **`bidirectional` excludes `both`/`outbound`.** With `bidirectional="true"` you must use `audioTrack="inbound"` (or omit it). `audioTrack="both"` + `bidirectional="true"` makes the call hang up with "End Of XML Instructions".
- **`keepCallAlive` needs `bidirectional="true"`.** Without `keepCallAlive`, a `` with no following elements ends the XML and hangs up instantly. Use it instead of a ``.
- **Endianness bites.** `media.payload` L16 is big-endian inbound; `playAudio` L16 must be little-endian. Mismatched endianness = static/garbled audio.
- **Format must match `start.mediaFormat`.** Sending `playAudio` at a different `contentType`/`sampleRate` than negotiated produces garbled playback.
- **Never detect end-of-call from `media.payload`.** All-`0xFF` (base64 all-`/`) bytes are just silence and appear mid-call. Use the WS `close` event.
- **Chunk size matters for barge-in.** Send ~20 ms chunks so a `clearAudio` flush is responsive. Large chunks add latency that `clearAudio` cannot recover.
- **`wss://` in production.** `ws://` is for local testing only; carriers and Vobiz require TLS in production. The URL must be publicly reachable.
- **`stop` is terminal.** After you send `stop`, the WS closes immediately - any queued `playAudio`/`checkpoint`/`clearAudio` sends are dropped. Send a farewell + `checkpoint`, wait for `playedStream`, then `stop`.
- **Reconnects.** `maxRetries` (XML, 0-10, default 0) controls how many times Vobiz retries a failed/dropped WebSocket. Make your `start` handler idempotent - a reconnect replays a fresh `start` with a new `streamId`.

## Complete bidirectional voice-agent recipe

A minimal full-duplex loop: greet on `start`, transcribe inbound `media`, barge-in with `clearAudio`, respond with `playAudio` + `checkpoint`, end with `stop`.

```javascript Node.js bidirectional agent skeleton
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });

wss.on('connection', (ws) => {
  let streamId = null;
  let mediaFormat = null;   // { encoding, sampleRate } from start
  let botSpeaking = false;
  let checkpointTimer = null;

  ws.on('message', async (raw) => {
    const data = JSON.parse(raw);

    switch (data.event) {
      case 'start':
        streamId = data.start.streamId;          // IDs are NESTED
        mediaFormat = data.start.mediaFormat;    // match this in playAudio
        await speak(ws, await tts('Hi, how can I help?'), 'greeting');
        break;

      case 'media': {
        const pcm = Buffer.from(data.media.payload, 'base64'); // big-endian L16
        // Barge-in: caller talks while bot is talking.
        if (botSpeaking && isSpeech(pcm)) {
          ws.send(JSON.stringify({ event: 'clearAudio', streamId }));
          botSpeaking = false;
        }
        feedTranscriber(pcm); // your STT; on final transcript -> respond()
        break;
      }

      case 'clearedAudio':
        // Playback queue flushed; safe to enqueue the new response.
        break;

      case 'playedStream':
        // Utterance finished. Clear any fallback timer.
        if (checkpointTimer) { clearTimeout(checkpointTimer); checkpointTimer = null; }
        botSpeaking = false;
        if (data.name === 'farewell') {
          ws.send(JSON.stringify({ event: 'stop', streamId }));
        }
        break;
    }
  });

  ws.on('close', () => {
    // Canonical end-of-stream. Flush recordings/transcripts here.
    teardown(streamId);
  });

  async function respond(text, name) {
    await speak(ws, await tts(text), name);
  }

  async function speak(ws, pcmBuffer, name) {
    botSpeaking = true;
    const frame = 320; // 20 ms of L16 @ 8 kHz; adjust to mediaFormat.sampleRate
    for (let i = 0; i  { botSpeaking = false; }, 15000);
  }
});
```

Pair it with this answer handler:

```xml VobizXML answer_url response

    
wss://your-server.com/ws
    

```

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [vobiz-ai](https://github.com/vobiz-ai)
- **Source:** [vobiz-ai/Agent-Skills](https://github.com/vobiz-ai/Agent-Skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-vobiz-ai-agent-skills-vobiz-audio-streams
- Seller: https://agentstack.voostack.com/s/vobiz-ai
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.