# Renfield

> Self-hosted AI assistant with voice satellites, 100+ tool integrations, RAG knowledge base, and full offline capability. Built with FastAPI + React + Ollama.

- **Type:** MCP server
- **Install:** `agentstack add mcp-ebongard-renfield`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [ebongard](https://agentstack.voostack.com/s/ebongard)
- **Installs:** 0
- **Category:** [Cloud & Infrastructure](https://agentstack.voostack.com/c/cloud-infrastructure)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [ebongard](https://github.com/ebongard)
- **Source:** https://github.com/ebongard/renfield

## Install

```sh
agentstack add mcp-ebongard-renfield
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

**English** | [Deutsch](README.de.md)

# Renfield

**Self-hosted AI assistant with voice satellites, 100+ tool integrations, and full offline capability.**

[](https://python.org)
[](https://fastapi.tiangolo.com)
[](https://react.dev)
[](https://ollama.ai)
[](LICENSE)
[](tests/)

---

## Why Renfield?

- **Voice-first with physical satellites** — Raspberry Pi Zero 2 W + ReSpeaker HAT in every room (~$30/satellite). Wake word, STT, TTS — fully local.
- **10 MCP servers, 100+ tools, Agent Loop** — Weather, search, news, calendar, smart home, media, DLNA, documents, email, workflows. ReAct agent chains tools autonomously for complex queries.
- **Fully offline, self-hosted, GDPR-friendly** — Runs entirely on your hardware. No cloud dependencies. Your data never leaves your network.

## Screenshots

Chat Interface — Conversation history, streaming responses, voice input

Knowledge Base (RAG) — Document upload, hybrid search, knowledge bases

Integrations — 10 MCP servers, 100+ tools

Room Management — 17 rooms synced with Home Assistant, DLNA renderer assignment

DLNA Output Device — Assign DLNA renderers to rooms for album playback

Satellite Monitor — Live status of voice satellites

Memory — Long-term recall of preferences and facts

## Features

### Core
- **Chat interface** with streaming responses, conversation history, inline album art, collapsible agent steps
- **ReAct Agent System** with specialized roles, multi-step tool chaining, and media transport shortcuts
- **Knowledge Graph** — entity-relation triples with post-extraction validation, string-similarity dedup, and bulk cleanup API
- **Conversational memory** — long-term recall of preferences, facts, and instructions with contradiction detection
- **Intent feedback learning** — learns from corrections via semantic matching (3-scope: intent, tool, complexity)
- **Voice I/O** — Whisper STT + Piper TTS + SpeechBrain speaker recognition
- **Presence detection** — BLE scanning, voice recognition, and web auth track who's in which room
- **Proactive notifications** — webhook-based alerts from Home Assistant/n8n + generic MCP notification polling
- **Paperless audit** — automated LLM-based metadata validation with review queue, OCR quality scoring, and auto-fix
- **Hook/Plugin system** — async extension API for custom integrations without modifying core

### Integrations (10 MCP Servers)

| Server | Description | Transport |
|--------|-------------|-----------|
| Weather | OpenWeatherMap forecasts | stdio |
| Search | SearXNG metasearch | stdio |
| News | NewsAPI headlines | stdio |
| Calendar | Exchange, Google, CalDAV (multi-account) | stdio |
| Jellyfin | Media server queries | stdio |
| DLNA | Media renderer control with gapless queue | streamable_http |
| n8n | Workflow automation | stdio |
| Home Assistant | Smart home control | streamable_http |
| Paperless | Document management | stdio |
| Email | IMAP/SMTP | stdio |

### Knowledge Base (RAG)
- **Hybrid search** — dense embeddings (pgvector) + BM25 full-text (German FTS, OR matching), combined via RRF
- **Formats** — PDF, DOCX, PPTX, XLSX, HTML, Markdown, TXT (with EasyOCR fallback for garbled PDFs)
- **Knowledge bases** — thematic collections with sharing and access control
- **Agent tool** — `knowledge_search` for combined RAG + Paperless search within the agent loop
- **Knowledge Graph** — entity-relation triples with post-extraction validation, string-similarity dedup, bulk cleanup API, admin dashboard

### Multi-Room Voice Satellites

| Component | Cost |
|-----------|------|
| Raspberry Pi Zero 2 W | ~$18 |
| ReSpeaker 2-Mics Pi HAT | ~$12 |
| MicroSD, power supply, speaker | ~$28 |
| **Total per room** | **~$58** |

- Local wake word detection (OpenWakeWord)
- Audio output routing to best device per room (Renfield, HA, DLNA)
- IP-based room context detection

### Presence Detection

Multi-source room-level presence tracking:

| Source | Trigger | Latency |
|--------|---------|---------|
| BLE Scanning | Satellite detects phone/watch via Bluetooth | ~30s (hysteresis) |
| Voice Presence | Speaker recognition identifies user | Instant |
| Web Auth | Authenticated user on room-assigned device | Instant |

- **Privacy-aware TTS** — notifications respect room occupancy (public / personal / confidential)
- **Automation hooks** — `enter_room`, `leave_room`, `first_arrived`, `last_left` events fire webhooks for n8n / Home Assistant
- **Presence dashboard** — real-time room occupancy in the admin UI

### Security & Access Control
- Role-permission based access control (RPBAC) with JWT
- Rate limiting, circuit breakers, trusted proxy support
- Docker secrets management for production

### Platform
- Progressive Web App (desktop, tablet, mobile)
- Dark mode (light, dark, system)
- Multilingual (German, English)
- Prometheus metrics (opt-in)

## Quick Start

```bash
git clone https://github.com/ebongard/renfield.git && cd renfield
cp .env.example .env                          # configure your settings
docker compose up -d                          # start the stack
docker exec -it renfield-ollama ollama pull qwen3:8b  # download LLM
```

Open **http://localhost:3000** and start chatting.

> For GPU acceleration, use `docker compose -f docker-compose.prod.yml up -d` with NVIDIA Container Toolkit.

## Architecture

## Configuration

All settings via `.env`, loaded by Pydantic Settings. See [docs/ENVIRONMENT_VARIABLES.md](docs/ENVIRONMENT_VARIABLES.md) for the full reference.

### LLM (Multi-Model)

Separate models for different tasks:

```env
OLLAMA_URL=http://ollama:11434
OLLAMA_CHAT_MODEL=qwen3:14b       # chat responses
OLLAMA_INTENT_MODEL=qwen3:8b      # intent recognition
OLLAMA_RAG_MODEL=qwen3:14b        # RAG answers
OLLAMA_EMBED_MODEL=nomic-embed-text  # embeddings (768 dim)
```

**Pluggable inference — not everything runs on Ollama.** The LLM tier speaks an OpenAI-compatible protocol, so each task can route to a different backend (`LLM_OPENAI_FOR_*`). Ollama is the simple default; the reference deployment splits inference across dedicated GPU servers:

| Tier | Server | Model |
|------|--------|-------|
| Chat · Agent · RAG · Intent · KG · Memory | **llama.cpp** | Qwen3.6-35B-A3B (MoE) |
| Embeddings | **llama.cpp** | Qwen3-Embedding-4B |
| Vision | **Ollama** | qwen3-vl |

**Voice runs on its own server.** Speech-to-text (Whisper), text-to-speech (Piper), and speaker recognition (ECAPA-TDNN) run in a dedicated, GPU-resident **`voice-server`** microservice over a streaming WebSocket — set `VOICE_SERVER_URL` to use it (they run in-process for local dev otherwise).

### Key Settings

```env
AGENT_ENABLED=false               # ReAct agent loop (opt-in)
MEMORY_ENABLED=false              # long-term memory (opt-in)
AUTH_ENABLED=false                 # RPBAC auth (opt-in)
MCP_ENABLED=true                  # master switch for integrations
PRESENCE_ENABLED=false            # room presence detection (opt-in)
KNOWLEDGE_GRAPH_ENABLED=false     # entity-relation extraction (opt-in)
NOTIFICATION_POLLER_ENABLED=false # proactive MCP notifications (opt-in)
METRICS_ENABLED=false             # Prometheus /metrics (opt-in)
```

## Development

```bash
make lint                    # lint all code (ruff + eslint)
make test                    # all tests
make test-backend            # backend tests (3,400+)
make test-frontend-react     # React tests (Vitest + RTL)
make test-coverage           # tests with coverage report
```

See [CONTRIBUTING.md](CONTRIBUTING.md) for the full development guide.

## Docker Compose Variants

| File | Use Case | GPU |
|------|----------|-----|
| `docker-compose.yml` | Standard | No |
| `docker-compose.dev.yml` | Development (Mac) | No |
| `docker-compose.prod.yml` | Production | NVIDIA |
| `docker-compose.prod-cpu.yml` | Production | No |

## Documentation

| Document | Content |
|----------|---------|
| [CONTRIBUTING.md](CONTRIBUTING.md) | How to contribute |
| [docs/FEATURES.md](docs/FEATURES.md) | Detailed feature documentation |
| [docs/ENVIRONMENT_VARIABLES.md](docs/ENVIRONMENT_VARIABLES.md) | Full configuration reference |
| [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) | Deployment guide |
| [docs/SECRETS_MANAGEMENT.md](docs/SECRETS_MANAGEMENT.md) | Docker secrets for production |
| [docs/SECURITY.md](docs/SECURITY.md) | Security headers, CSP, dependency security |
| [docs/ACCESS_CONTROL.md](docs/ACCESS_CONTROL.md) | Role-based access control & MCP permissions |
| [docs/SPEAKER_RECOGNITION.md](docs/SPEAKER_RECOGNITION.md) | Speaker recognition |
| [docs/OUTPUT_ROUTING.md](docs/OUTPUT_ROUTING.md) | Audio output routing |
| [docs/PROACTIVE_NOTIFICATIONS.md](docs/PROACTIVE_NOTIFICATIONS.md) | Webhook notifications & proactive alerts |
| [docs/MULTILANGUAGE.md](docs/MULTILANGUAGE.md) | Internationalization (i18n) |
| [docs/WAKEWORD_CONFIGURATION.md](docs/WAKEWORD_CONFIGURATION.md) | Wake word detection setup |
| [docs/EXTERNAL_OLLAMA.md](docs/EXTERNAL_OLLAMA.md) | External Ollama instances |
| [docs/SATELLITE_MONITORING.md](docs/SATELLITE_MONITORING.md) | Satellite monitoring |
| [docs/SATELLITE_OTA_UPDATES.md](docs/SATELLITE_OTA_UPDATES.md) | Satellite over-the-air updates |
| [docs/AUDIO_CAPTURE_4MIC.md](docs/AUDIO_CAPTURE_4MIC.md) | ReSpeaker 4-Mic array support |
| [CLAUDE.md](CLAUDE.md) | Developer reference (architecture, patterns) |

## Disclaimer

Renfield is an independent open-source project. It is not affiliated with, endorsed by, or connected to any third party, organization, company, or brand with the same or similar name.

## Acknowledgments

- [Ollama](https://ollama.ai/) — Local LLM inference
- [Whisper](https://github.com/openai/whisper) — Speech-to-text
- [Piper](https://github.com/rhasspy/piper) — Text-to-speech
- [SpeechBrain](https://speechbrain.github.io/) — Speaker recognition (ECAPA-TDNN)
- [IBM Docling](https://github.com/DS4SD/docling) — Document processing for RAG
- [pgvector](https://github.com/pgvector/pgvector) — Vector similarity search
- [Home Assistant](https://www.home-assistant.io/) — Smart home platform
- [n8n](https://n8n.io/) — Workflow automation
- [SearXNG](https://docs.searxng.org/) — Metasearch engine
- [OpenWakeWord](https://github.com/dscripka/openWakeWord) — Wake word detection

## License

MIT License — see [LICENSE](LICENSE) file.

## Contributing

Contributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) and check the [good first issues](https://github.com/ebongard/renfield/labels/good-first-issue).

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [ebongard](https://github.com/ebongard)
- **Source:** [ebongard/renfield](https://github.com/ebongard/renfield)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** yes
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-ebongard-renfield
- Seller: https://agentstack.voostack.com/s/ebongard
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.