# HAL9000

> The world's first all-seeing, all-hearing, (almost) all-doing AI agent.

- **Type:** MCP server
- **Install:** `agentstack add mcp-shandar-hal9000`
- **Verified:** Pending review
- **Seller:** [shandar](https://agentstack.voostack.com/s/shandar)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [shandar](https://github.com/shandar)
- **Source:** https://github.com/shandar/HAL9000
- **Website:** https://affordance.design/hal9000/

## Install

```sh
agentstack add mcp-shandar-hal9000
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

HAL 9000

  "I am completely operational, and all my circuits are functioning perfectly."

  Local, multimodal AI agent — sees, hears, thinks, speaks, and acts on your machine.
  Cross-platform (macOS, Windows, Linux) · Free mode (zero API keys) · Claude Code co-work hub.

  Product Page ·
  Free Mode ·
  Quick Start ·
  Co-Work ·
  Changelog

---

A local, multimodal AI agent that **sees** you via webcam, **hears** your voice, **thinks** via LLM, **speaks** with a cloned voice, **acts** on your machine, and **integrates** with Claude Code via MCP. Runs entirely on your machine with a browser-based control panel.

**Works on macOS, Windows, and Linux.** One codebase, auto-detects OS at runtime.

---

## What HAL Can Do

| Capability | How |
|------------|-----|
| **See** | Webcam feed with browser HUD — scanlines, corner brackets, REC indicator |
| **Hear** | Browser mic recording (Web Audio API, live waveform, silence detection) + Whisper STT (API or local faster-whisper) |
| **Think** | Multi-provider LLM (GPT-4o, Claude, Gemini, **Ollama**) with function calling |
| **Speak** | 3 voice providers — Edge TTS (free/fast), ElevenLabs (paid/best), XTTS (local/cloned) |
| **Act** | 43 cross-platform tools — shell, apps, files, web search, memory, clipboard, app automation, Claude Code delegation, background tasks, artifacts, multi-agent orchestration |
| **Chat** | Terminal-style chat with streaming responses, 35 slash commands (categorized menu, keyboard nav), mic button — type or speak to HAL |
| **Disambiguate** | Smart choice sheet UI — HAL presents numbered options, user clicks to select |
| **Integrate** | MCP server exposes 21 tools to Claude Code/Desktop for bidirectional AI collaboration |
| **Remember** | Typed persistent memory — facts, decisions, preferences, session summaries |
| **Know** | Knowledge upload (drag-drop or button) — PDFs, docs, code, images. BM25 keyword search, deep-read or skim modes. Also loads local files + remote llms.txt URLs at boot |
| **Co-Work** | Background task runner, artifact workspace, multi-agent orchestration, cross-agent context handoff |

---

## Architecture

```
┌──────────────────────────────────────────────────────────┐
│                     HAL9000 ENGINE                        │
│                                                          │
│  Vision ──┐                                              │
│            ├──→ Brain (LLM + function calling)           │
│  Browser ──┘       │              │                       │
│  Mic + Chat        ▼              ▼                       │
│                 Voice          Tools (43)                 │
│           (Edge/11Labs/XTTS) (OS agent layer)            │
│                   │                                      │
│                   ▼                                      │
│          Browser Audio + Waveform                        │
│                                                          │
│  Knowledge ─── Memory (typed) ─── Session Tracking       │
│  TaskRunner ── Orchestrator ── Artifact Store             │
└──────────────────────────────────────────────────────────┘
         │                              │
    Flask Server                  MCP Server
    localhost:9000              (Claude Code integration)
```

---

## Free Mode

Run HAL with **zero API keys** and **zero cost** — fully local operation.

```bash
# 1. Install Ollama (local LLM)
brew install ollama        # macOS
# or: curl -fsSL https://ollama.com/install.sh | sh   # Linux
# or: download from ollama.com                         # Windows

# 2. Pull a model
ollama pull llama3.1

# 3. Set up HAL
git clone https://github.com/shandar/HAL9000.git
cd HAL9000 && python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 4. One line in .env
echo "FREE_MODE=true" > .env

# 5. Run
python server.py
```

| Layer | Free Provider | Paid Alternative |
|-------|--------------|------------------|
| **Brain** | Ollama (Llama 3.1, Mistral, Phi-3) | GPT-4o, Claude, Gemini |
| **STT** | faster-whisper (local Whisper) | OpenAI Whisper API |
| **TTS** | Edge TTS (default, always free) | ElevenLabs, XTTS |

`FREE_MODE=true` overrides `AI_PROVIDER`, `STT_PROVIDER`, and `TTS_PROVIDER` in one toggle. You can also mix — e.g., `FREE_MODE=true` with `STT_PROVIDER=whisper_api` for local brain + cloud STT.

---

## Cross-Platform Support

HAL auto-detects your OS and uses the right system commands:

| Feature | macOS | Windows | Linux |
|---------|-------|---------|-------|
| Volume | AppleScript | nircmd / PowerShell | pactl / amixer |
| Brightness | ioreg | WMI | brightnessctl |
| Notifications | osascript | Toast API | notify-send |
| Clipboard | pbcopy/pbpaste | Get/Set-Clipboard | xclip / wl-clipboard |
| Screenshot | screencapture | PIL.ImageGrab | scrot / grim |
| Battery | pmset | psutil / WMI | psutil / sysfs |
| WiFi | networksetup | netsh wlan | nmcli |
| Apps | open -a + .app scan | start + Start Menu scan | gtk-launch + .desktop scan |
| Terminal | AppleScript Terminal | Windows Terminal / cmd | gnome-terminal / konsole |
| Embedded Terminal | ✅ xterm.js + PTY | ❌ External only | ✅ xterm.js + PTY |

No `#ifdef`, no separate builds — one `pip install`, one `python server.py`.

---

## System Requirements

### Minimum (Free Mode — Ollama + faster-whisper)

| Component | Requirement |
|-----------|-------------|
| **OS** | macOS 12+, Windows 10+, or Ubuntu 20.04+ (any modern Linux) |
| **CPU** | 4 cores (Intel i5 / Apple M1 / AMD Ryzen 5 or better) |
| **RAM** | 8 GB (Ollama loads models into memory — llama3.1 8B needs ~5 GB) |
| **Disk** | 6 GB free (3 GB for Ollama model + 1 GB for faster-whisper model + HAL) |
| **Python** | 3.10 or higher |
| **Browser** | Any modern browser (Chrome, Firefox, Safari, Edge) |
| **Microphone** | Required for voice input (built-in or USB) |
| **Webcam** | Optional — required only for vision features |
| **Network** | Not required (fully offline operation) |

### Recommended (Paid Providers — GPT-4o, Claude, ElevenLabs)

| Component | Requirement |
|-----------|-------------|
| **RAM** | 4 GB (no local models loaded) |
| **Disk** | 500 MB free |
| **Network** | Required (API calls to OpenAI/Anthropic/Google) |
| **API Keys** | At least `OPENAI_API_KEY` for GPT-4o + Whisper STT |

### Performance Notes

| Mode | Brain Latency | STT Latency | TTS Latency | RAM Usage |
|------|--------------|-------------|-------------|-----------|
| **Free (Ollama llama3.1)** | ~2-5s (CPU), ~1-2s (Apple Silicon) | ~1-3s (faster-whisper base) | ~0.7s (Edge TTS) | ~5-6 GB |
| **Free (Ollama phi3)** | ~1-2s (CPU) | ~1-3s | ~0.7s | ~3 GB |
| **Paid (GPT-4o)** | ~1-2s (API) | ~0.5s (Whisper API) | ~0.7s (Edge) | ~200 MB |
| **Paid (Claude)** | ~1-3s (API) | ~0.5s | ~1.2s (ElevenLabs) | ~200 MB |

> **Apple Silicon users**: Ollama runs significantly faster on M1/M2/M3/M4 chips with Metal acceleration. An M1 MacBook Air can run llama3.1 8B comfortably.
>
> **GPU users (Linux/Windows)**: Ollama supports NVIDIA CUDA. With a 6 GB+ VRAM GPU, expect 2-3x faster inference than CPU.

---

## Quick Start

```bash
git clone https://github.com/shandar/HAL9000.git
cd HAL9000
python -m venv .venv
source .venv/bin/activate         # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env              # fill in your API keys (or set FREE_MODE=true)
python server.py                  # start the web control panel
```

Open **http://localhost:9000** → click **Activate**.

> **Free mode?** Just `echo "FREE_MODE=true" > .env` — no API keys needed. See [Free Mode](#free-mode).

### Claude Code Integration

```bash
# Register HAL as an MCP server for Claude Code
claude mcp add hal-9000 -- python /path/to/HAL9000/hal_mcp_server.py
```

Now Claude Code can see through your webcam, speak aloud, control your Mac, access HAL's memory, and hand off session context.

---

## Co-Work Features

HAL operates as a **co-work hub** — coordinating work across HAL, Claude Code CLI, and Claude Desktop.

### Typed Memory & Context Handoff
- Memories are typed: `fact`, `decision`, `preference`, `task`, `session_summary`
- Sessions auto-summarize on shutdown — HAL remembers what happened
- Claude Code can call `hal_get_context` to load relevant context at session start
- Manual "wrap up" via `hal_save_session` or the `save_session` tool

### Background Task Runner
- Submit long-running coding tasks via `background_task` tool
- Tasks run asynchronously via `claude --print` with real-time progress
- Configurable concurrency (default 2), 600s timeout, cancellation
- Task queue panel in the UI shows status with live updates

### Shared Workspace (Artifacts)
- HAL creates visual artifacts (code, markdown, HTML, Mermaid diagrams) via `create_artifact`
- Artifacts appear in a tabbed workspace panel alongside the chat
- 3-column layout when artifacts are active
- Copy button, close button, sandboxed HTML rendering

### Multi-Agent Orchestration
- Spawn multiple named Claude Code agents on parallel tasks via `orchestrate`
- Conflict detection when agents modify the same files
- Agent dashboard with status indicators, file lists, cancel controls
- Results summarized and stored in session memory

---

## API Keys

> **With `FREE_MODE=true`, no API keys are needed at all.** See [Free Mode](#free-mode).

| Key | Where | Required |
|-----|-------|----------|
| `OPENAI_API_KEY` | platform.openai.com | Only if using GPT-4o brain or Whisper API STT |
| `ANTHROPIC_API_KEY` | console.anthropic.com | Only if `AI_PROVIDER=anthropic` |
| `GEMINI_API_KEY` | aistudio.google.com | Only if `AI_PROVIDER=gemini` |
| `ELEVENLABS_API_KEY` | elevenlabs.io | Only if `TTS_PROVIDER=elevenlabs` |
| `ELEVENLABS_VOICE_ID` | ElevenLabs voice library | Only if `TTS_PROVIDER=elevenlabs` |

**No API key needed for**: Edge TTS (default voice), Ollama (local LLM), faster-whisper (local STT).

---

## Voice Providers

HAL supports 3 TTS providers, switchable at runtime from the dashboard:

| Provider | Cost | Speed | Quality | Config |
|----------|------|-------|---------|--------|
| **Edge TTS** (default) | Free | ~0.7s | Good — deep male voice | `TTS_PROVIDER=edge` |
| **ElevenLabs** | Paid | ~1.2s | Best — natural prosody | `TTS_PROVIDER=elevenlabs` |
| **XTTS** (local) | Free | ~4.5s | Good — voice cloning | `TTS_PROVIDER=local` (requires Python 3.11) |

Switch from the dashboard UI or set `TTS_PROVIDER` in `.env`.

---

## Web Dashboard

Access at **http://localhost:9000** after starting the server.

| Panel | Description |
|-------|-------------|
| **HAL panel** | HAL 9000 eye with real-time waveform visualization overlaid on red block during speech |
| **HAL image controls** | 3D-style Vision/Voice/Claude buttons positioned on the HAL image strip |
| **Webcam HUD** | Live MJPEG feed with scanlines, corner brackets, REC indicator — collapses when vision is off |
| **Voice selector** | Segmented switch to swap between Edge/ElevenLabs/XTTS at runtime |
| **Chat window** | Terminal-style chat with prompt prefixes, streaming responses, formatted lists, mic button — Enter to send, Shift+Enter for newline |
| **Slash commands** | 35 categorized commands with keyboard navigation — type `/` to open menu |
| **Choice sheet** | Slide-up modal for disambiguation — auto-detects when HAL presents numbered options |
| **Task queue** | Collapsible panel showing background tasks and agents with live status |
| **Workspace** | Tabbed artifact panel — code, diagrams, HTML — stacks above chat when artifacts are created |
| **Embedded terminal** | Full interactive xterm.js terminal (PTY-backed) — run shell, Claude Code, review artifacts in-app (macOS/Linux) |
| **Resizable layout** | Drag handles between columns to resize HAL, workspace, and chat panels |
| **Power button** | Circular SVG power icon in top toolbar — activates/deactivates HAL |
| **Status bar** | Connection status, timestamp, version |
| **Boot greeting** | Time-aware creative HAL-style greeting with 20 randomized boot lines |

The UI uses a sci-fi industrial aesthetic — brushed metal bezels, LED indicator lights, recessed panels.

**PWA support**: Add to Home Screen on mobile for a native app experience.

---

## Project Structure

```
HAL9000/
├── server.py              # Flask web server + API endpoints (localhost only)
├── hal9000.py             # HAL engine — lifecycle, main loop, browser audio
├── hal_mcp_server.py      # MCP server for Claude Code/Desktop integration (21 tools)
├── config.py              # Settings + env loading with safe parsing
├── requirements.txt
├── .env.example
├── .mcp.json              # Project MCP config for Claude Code
│
├── core/
│   ├── brain.py           # Multi-provider LLM + function calling (thread-safe)
│   ├── vision.py          # Webcam capture + MJPEG stream
│   ├── hearing.py         # Mic recording + VAD + Whisper STT
│   ├── voice.py           # Multi-provider TTS (Edge/ElevenLabs/XTTS)
│   ├── memory_store.py    # Typed memory store with auto-migration
│   ├── task_runner.py     # Async background task queue for Claude Code
│   ├── orchestrator.py    # Multi-agent coordinator with conflict detection
│   ├── terminal_server.py # Embedded WebSocket terminal (xterm.js PTY bridge, port 9001)
│   ├── platform/           # Cross-platform OS abstraction (auto-detected)
│   │   ├── __init__.py     # Auto-detect: Darwin → mac, Windows → windows, Linux → linux
│   │   ├── base.py         # Abstract PlatformAPI interface (15 methods)
│   │   ├── mac.py          # macOS: AppleScript, osascript, pbcopy, screencapture
│   │   ├── windows.py      # Windows: PowerShell, WMI, Toast, PIL.ImageGrab
│   │   └── linux.py        # Linux: pactl, xclip, notify-send, brightnessctl
│   │
│   ├── tools/              # Tool registry + 43 tools across 8 domain modules
│   │   ├── __init__.py     # Registry, execute(), format converters, security
│   │   ├── shell.py        # run_shell (whitelisted commands)
│   │   ├── apps.py         # open/quit/list apps, open URLs, app_action (cross-platform)
│   │   ├── files.py        # list/read/write/search/info
│   │   ├── system.py       # volume, brightness, notifications, clipboard, screenshot (cross-platform)
│   │   ├── web.py          # web_search, fetch_url
│   │   ├── memory.py       # remember, recall, forget, list_memories, save_session
│   │   ├── delegation.py   # claude_code, background_task, orchestrate, agents (cross-platform)
│   │   └── artifacts.py    # create_artifact, update_artifact
│   └── knowledge.py       # Knowledge loader (files + URLs) + upload ingestion + BM25 search
│
├── knowledge/             # Drop files here or upload via UI — HAL indexes at boot + runtime
│   ├── sources.txt        # Remote URLs to fetch (llms.txt, etc.)
│   └── *.txt              # Local knowledge files
│
├── memory/                # Persistent typed memory (created at runtime)
│   └── facts.json         # Typed entries: {id, type, content, timestamp, source, session_id, metadata}
│
├── assets/
│   ├── HAL.png            # Dashboard hero image
│   ├── HAL-eye.png        # App icon / PWA icon source
│   ├── manifest.json      # PWA manifest
│   ├── sw.js              # Service worker for PWA
│   └── voice/             # XTTS reference clips (optional)
│
└── templates/
    └── index.html         # Web dashboard + chat UI + waveform + task panel + workspace
```

---

## Security

HAL has been security-hardened:

| Measure | Detail |
|---------|--------|
| **Command whitelist** | `run_shell` only allows 77 approved commands (ls, git, npm, etc.) |
| **Blocked commands** | `sudo`, `shutdown`, `diskutil`, etc. are explicitly blocked |
| **AppleScript escaping** | All user strings escaped before osascript interpolation |
| **App action blocklist** | `app_action` blocks `do shell script`, `system events`, etc. |
| **Localhost binding** | Flask binds to `127.0.0.1` by default (override with `HAL_HOST`) |
| **Code exec guard** | `/api/run` only accepts requests

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [shandar](https://github.com/shandar)
- **Source:** [shandar/HAL9000](https://github.com/shandar/HAL9000)
- **License:** MIT
- **Homepage:** https://affordance.design/hal9000/

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** yes
- **Environment & secrets:** yes
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: flagged — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-shandar-hal9000
- Seller: https://agentstack.voostack.com/s/shandar
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.