Geny

Geny — Geny Execute, Not You

I'll handle it — you just sit back.

A multi-agent VTuber + autonomous worker platform. Pair a chatty Live2D / Spine VTuber with a tool-running Sub-Worker, watch them collaborate inside a 3D city, and swap any of five LLM backends with one config click.

한국어 README · Architecture · LLM Providers · Sessions & Delegation · Environments · Error Codes

# 60-second start — no GPU, no API key required
git clone https://github.com/CocoRoF/Geny.git && cd Geny && ./geny up
# → open http://localhost:3000

Details, profiles and manual setup → Installation

The Geny ecosystem

These projects are built to work together. Geny is the product at the top of the stack; everything below is a building block you can also use on its own. ➡️ marks where you are.

| Project | What it is | Role in the stack | |---|---|---| | ➡️ Geny | Multi-agent VTuber + autonomous-worker platform | The product — uses every project below | | geny-executor | 21-stage, manifest-driven agent pipeline · PyPI · Apache-2.0 | The engine everything runs on | | GAPT | Self-hosted AI DevOps platform — sandbox · edit · build · deploy | Where agents safely touch real repos | | geny-avatar | 2D live-avatar editor with AI texture generation | Where Geny's faces are made | | edit2docs | AI-agent-native DOCX / XLSX / PPTX engine · PyPI · Apache-2.0 | How agents generate, edit and preview office documents | | AN-Web | AI-native headless browser engine · PyPI | How agents read and search the web |

How they fit together

                  Geny — the product (uses everything below)
                    │
      ┌─────────────┼──────────────┐
 agent engine    avatars      sandbox + deploy
      │             │              │
      ▼             ▼              ▼
 geny-executor  geny-avatar      GAPT
  (the engine)  (avatar editor)  (AI DevOps platform)
      │
      ├── edit2docs  (documents: generate · edit · native preview)
      └── AN-Web     (web: browse · read · search)

---

> 📸 Image needed — hero screenshot: the 3D city playground with agents walking, the VTuber chat panel, and the Live2D avatar.

---

Desktop connector download

A VTuber connector that floats at the bottom of your desktop — leave the server running, and a live avatar lives at the bottom of your screen.

> 📸 Image needed — the desktop connector: the floating avatar at the bottom of a real desktop, tray menu open.

1) Installer — ➡️ Get it from the latest Release

| OS | File | Install / first launch | |---|---|---| | Windows | Geny-Setup-*.exe | Double-click → on the SmartScreen warning, More info → Run anyway (currently unsigned) | | macOS | Geny-*.dmg | Open and drag to Applications → first launch: right-click → Open (Gatekeeper) | | Linux | Geny-*.AppImage / *.deb | AppImage: chmod +x Geny-*.AppImage && ./Geny-*.AppImage · deb: sudo dpkg -i Geny-*.deb |

> No installer listed yet? The release is still building — this is a public repo, so GitHub Actions auto-builds the macOS / Windows / Linux installers.

2) After launch — just three fields and you're in

1. Geny server address — prefilled with https://geny-x.hrletsgo.me (or your own host if you self-host).
2. admin ID
3. admin password

→ Log in and the token is saved to your OS keychain; the avatar appears at the bottom of the screen, ready to use. (Drag the avatar to move it; the tray icon opens settings / updates / quit.)

Auto-update

From v0.3.0 the connector updates itself via GitHub Releases — it detects a new release while running, downloads it, and applies it on restart (you can also check manually via tray → Check for updates). Windows · Linux (AppImage) work unsigned; macOS auto-update turns on once the app is code-signed. Install manually once, and every version after that is automatic.

Build from source

git clone https://github.com/CocoRoF/Geny.git
cd Geny/desktop
npm install
npm run dev          # dev run
npm run dist:win     # / dist:mac / dist:linux — build the installer yourself

More: [desktop/README.md](desktop/README.md) · design: [dev_docs/vtuber-desktop/PLAN.md](dev_docs/vtuber-desktop/PLAN.md)

---

What Geny is

| Concept | What it does | |---|---| | VTuber session | The conversational face. Live2D / Spine avatar, persona prompt, TTS, emotion tags. Talks to the user; delegates real work to a paired Sub-Worker. | | Sub-Worker session | The execution layer. Tool-using agent — file ops, shell, web fetch, MCP-bridged host tools. Reports results back through structured [SUB_WORKER_RESULT] messages. | | Environment | A serialisable [EnvironmentManifest](docs/environments.md) that pins every pipeline stage + provider + tool set. One artifact, deterministic reproduction. | | 3D City Playground | Three.js / React Three Fiber dashboard — agents appear as characters walking a procedural city. | | Five LLM backends | anthropic / openai / google / vllm (self-host) / claude_code_cli — switch per env, no code change. | | Stable error codes | Every executor failure surfaces with a stable exec.. code. The frontend renders localised, actionable Korean / English prompts. |

The backend is built on geny-executor — a 21-stage manifest-driven agent pipeline (no LangChain, no LangGraph). The frontend is Next.js 16 with R3F-powered 3D, Pixi.js for whiteboard / 2D overlays, and a Korean/English i18n layer.

---

Architecture (high level)

┌────────────────────────── Geny ──────────────────────────────────┐
│                                                                  │
│  Frontend (Next.js 16 + R3F + Pixi)                              │
│   ├── 3D City Playground (Three.js / R3F)                        │
│   ├── VTuber chat panel + Live2D / Spine avatar                  │
│   ├── Environment editor (21-stage manifest UI)                  │
│   ├── LLM Backends settings (5 provider cards)                   │
│   ├── Memory / Knowledge / Whiteboard tabs                       │
│   └── Logs tab (i18n'd error codes, tool traces, stage events)   │
│                                                                  │
│  Backend (FastAPI)                                               │
│   ├── controller/  ← FastAPI routes (sessions, env, vtuber, …)   │
│   ├── service/                                                   │
│   │   ├── executor/    ← geny-executor hookup                    │
│   │   ├── gapt/        ← GAPT client + workspace provider        │
│   │   ├── environment/ ← manifest store + templates              │
│   │   ├── llm_patches/ ← Korean error envelopes + CLI tool tap   │
│   │   ├── memory/      ← session memory v2 + vector retrieval    │
│   │   ├── permission/  ← per-tool ACL evaluator                  │
│   │   ├── vtuber/      ← Live2D / Spine library + thinking-trig  │
│   │   └── chat/        ← chat-room store + delegation routing    │
│   ├── tools/  ← auto-loaded Python tools (send DM, memory, …)    │
│   ├── mcp/    ← auto-loaded MCP server configs                   │
│   ├── scripts/geny_mcp_bridge.py ← per-session MCP wrap for CLI  │
│   └── prompts/  ← role markdown (vtuber.md, worker.md, …)        │
│                                                                  │
│  geny-executor ≥2.47  (PyPI dep)                                 │
│   ├── 21-stage agent pipeline                                    │
│   ├── 5 LLM client implementations                               │
│   ├── ContainerCLIRunner + SandboxHandle (run CLI in a sandbox) │
│   └── ExecutorErrorCode taxonomy                                 │
│                                                                  │
│  gapt/  ← GAPT submodule (sandbox/devops platform)               │
│   ├── server/  FastAPI control plane (projects/workspaces/…)     │
│   ├── per-workspace containers (git · fs · terminal · preview)  │
│   └── deploy pipeline (compose / ssh / webhook targets)          │
└──────────────────────────────────────────────────────────────────┘

Full architecture writeup → [docs/architecture.md](docs/architecture.md). Geny ⇄ GAPT integration → [docs/analysis/gapt-integration-plan.md](docs/analysis/gapt-integration-plan.md).

---

Key features

VTuber ↔ Sub-Worker pairing

Every VTuber session is auto-paired with a Sub-Worker. The VTuber handles conversation and personality; the Sub-Worker does real work. Delegation flows through a single MCP-bridged tool (mcp__geny__send_direct_message_internal) — see [docs/sessions.md](docs/sessions.md).

Five LLM backends, one selector

Settings → LLM Backends gives each of the 5 providers (Anthropic / OpenAI / Google / vLLM / Claude Code CLI) its own card with health probe + auth flow. Stage 6 of any environment picks one via dropdown — see [docs/providers.md](docs/providers.md).

> 📸 Image needed — Settings → LLM Backends: the five provider cards with their health status.

Manifest-driven environments

Pipelines are defined as EnvironmentManifest JSON artifacts — 21 stages, one strategy per slot, version-controlled. The Environment editor in the UI lets you customise any preset (worker / VTuber / Sub-Worker) without touching code — see [docs/environments.md](docs/environments.md).

> 📸 Image needed — the Environment editor: the 21-stage manifest UI with strategy dropdowns.

Per-session MCP wrap (Claude Code CLI)

When a session pins claude_code_cli as its Stage 6 backend, Geny attaches a per-session MCP bridge so the spawned CLI's LLM sees Geny's tool registry as mcp__geny__ — file ops, web fetch, memory, blog publisher, sub-worker delegation — all callable natively inside the CLI's agentic loop.

Stable error codes + i18n

Every executor exception carries a stable exec.. code. The session log renders the matching Korean (or English) message + recommended next step instead of the raw English server error — see [docs/error_codes.md](docs/error_codes.md).

3D city playground

Active sessions appear as walking characters in a procedural Kenney-asset city. A* pathfinding, bone-animated avatars, time-of-day cycle. R3F + Drei + Three.js.

> 📸 Image needed — the 3D city playground: agents walking the procedural city under time-of-day lighting.

Live2D + Spine + AI-baked avatars

Geny ships with a separate puppet-editor service (geny-avatar) wired in as a git submodule. Upload a Spine or Cubism puppet, decompose layers, paint masks, regenerate textures with AI, and bake the model straight into Geny's VTuber library.

TTS / STT / voice notes

edge-tts for output, Whisper for input, OmniVoice integration for multi-speaker scenes. The voice-notes feature lets users dictate into the whiteboard.

Knowledge whiteboard + memory v2

Session memory is routed through geny-executor's Stage 2 (Context) + Stage 18 (Memory) — progressive disclosure, vault map, vector retrieval. The Knowledge whiteboard exposes a collaborative Pixi.js canvas for diagram-style sessions.

Multi-pod ready

Redis-backed session metadata sharding lets multiple backend pods serve one user — useful for cloud deployments.

Sandboxed projects & deploy (GAPT)

Project / workspace / sandbox / deploy is delegated to GAPT (geny-adapted-project-toolkit), wired in as a git submodule at [gapt/](gapt/) (tracks main; git pull auto-rolls it via [.githooks/post-merge](.githooks/post-merge)). GAPT runs each workspace in its own isolated container (git · file ops · terminal · dev-server preview · compose/ssh deploy targets), Postgres-backed and Caddy-routed. Geny keeps its own agent runtime (persona · voice · emotion · memory) and points sessions at a GAPT workspace via the executor's ContainerCLIRunner — so the agent edits code inside the sandbox while Geny's moat stays host-side. Agents can also drive GAPT directly through its 41-tool MCP. Design: [docs/analysis/gapt-integration-plan.md](docs/analysis/gapt-integration-plan.md) · Deploy + test: [docs/operations/gapt-test-guide.md](docs/operations/gapt-test-guide.md).

---

Project structure

geny/
├── README.md / README_ko.md          # this hub
├── img/                              # logos and screenshots
├── docs/                             # topic-page docs (architecture, sessions, …)
├── backend/                          # FastAPI + geny-executor host
│   ├── main.py                       # app entry + executor wiring
│   ├── pyproject.toml                # pins geny-executor >= 2.47.0
│   ├── controller/                   # FastAPI routes
│   │   ├── agent_controller.py       # session + stream + invoke
│   │   ├── llm_backends_controller.py# 5-provider health + auth
│   │   ├── mcp_bridge_controller.py  # per-session MCP RPC
│   │   ├── vtuber_*.py               # VTuber library + chat + thinking
│   │   ├── memory_*.py               # memory + knowledge + opsidian
│   │   ├── chat_controller.py        # chat-room CRUD
│   │   ├── environment_controller.py # manifest editor backend
│   │   └── …                         # cron, whiteboard, voice-notes, ...
│   ├── service/
│   │   ├── executor/                 # AgentSessionManager + AgentSession
│   │   ├── environment/              # manifest store + templates
│   │   ├── llm_patches.py            # Korean error envelopes + CLI tool tap
│   │   ├── memory/                   # session memory v2
│   │   ├── permission/               # per-tool ACL
│   │   ├── vtuber/                   # Live2D / Spine library + triggers
│   │   ├── chat/                     # chat-room store + delegation routing
│   │   ├── config/                   # ConfigManager + settings cards
│   │   ├── logging/                  # SessionLogger (now with error_code)
│   │   └── …
│   ├── tools/                        # auto-loaded Python tools
│   │   ├── built_in/                 # ships: messaging, memory, knowledge
│   │   └── custom/                   # web_search, browser, whiteboard, blog
│   ├── mcp/                          # auto-loaded MCP server configs
│   ├── scripts/geny_mcp_bridge.py    # stdio bridge for CLI MCP wrap
│   └── prompts/                      # role markdown (vtuber.md, worker.md, …)
├── frontend/                         # Next.js 16 + R3F + Pixi
│   └── src/
│       ├── components/               # tabs, modals, panels, env_management/…
│       ├── lib/                      # api.ts, i18n/, modelCatalog.ts, …
│       ├── store/                    # Zustand stores
│       └── types/                    # shared TypeScript types
├── vendor/geny-avatar/               # puppet-editor submodule
├── gapt/                             # GAPT git submodule (tracks main)
│   ├── server/                       # FastAPI control plane (projects/workspaces/…)
│   ├── compose/                      # GAPT compose stacks (postgres/redis/caddy/…)
│   ├── docker/workspace/             # per-workspace sandbox image
│   └── mcp/                          # 41-tool agent MCP for GAPT
└── docker-compose.{yml,dev,prod}.yml # compose stacks

For the developer-facing internal architecture maps see [backend/docs/](backend/docs/) and [docs/architecture.md](docs/architecture.md).

---

Tech stack

| Layer | Technology | |---|---| | Frontend | Next.js 16, React 19, TypeScript, Tailwind CSS v4, Zustand 5, Pixi.js | | 3D engine | Three.js, React Three Fiber, Drei | | Avatars | Live2D Cubism, Spine 4, [geny-avatar](http

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: CocoRoF
• Source: CocoRoF/Geny
• License: Apache-2.0
• Homepage: https://geny-x.hrletsgo.me

Install and usage instructions live in the source repository linked above.