HyprPilot

HyprPilot

Programmatic control of Hyprland for AI agents and humans. Typed Rust IPC client, long-running daemon with undo and snapshots, CLI, MCP server, input synthesis, and screen capture + OCR.

> Status: v1.0.0 — first stable release. Typed IPC client, daemon with > undo/snapshots, CLI, stdio MCP server, input synthesis (typing, key chords, > and mouse via wlr-virtual-pointer-v1), and screen capture (grim) + OCR > (tesseract) in a Vision capability group (excluded from the default > profile because screenshots can capture private content).

Install

On Arch and Arch-based distros, install from the AUR:

yay -S hyprpilot-mcp      # or: paru -S hyprpilot-mcp

This installs the hyprpilot, hyprpilot-daemon, and hyprpilot-mcp binaries. The Vision and input backends (grim, slurp, tesseract, wtype, ydotool) are optional dependencies — install the ones you need.

To build from source, see [Quick start](#quick-start-cli) below.

Crates

• hyprpilot-core — async Hyprland IPC client (control socket + event

socket), typed queries and dispatchers, error types. No daemon awareness.

• hyprpilot-daemon — long-running process exposing JSON-RPC over a Unix

socket. Records reversible operations in an undo stack.

• hyprpilot-cli — hyprpilot binary. Subcommands for queries,

window/workspace ops, daemon control, and undo.

• hyprpilot-mcp — hyprpilot-mcp stdio MCP server. Exposes Hyprland

control to MCP hosts (Claude Desktop, Claude Code) as typed tools with capability profiles and dry-run default.

• hyprpilot-input — input synthesis (typing, key chords, mouse) via

wtype, ydotool, and Hyprland's sendshortcut.

• hyprpilot-vision — screen capture via grim (wlr-screencopy) and

OCR via tesseract. Daemon-agnostic: image bytes are bulky and stateless, so the CLI and MCP server call into this crate directly.

Quick start (CLI)

cargo build --release

# Direct queries (no daemon needed)
./target/release/hyprpilot query active-window
./target/release/hyprpilot query clients

# Window ops (direct to Hyprland)
./target/release/hyprpilot win cycle
./target/release/hyprpilot ws switch 2

# Daemon-backed undo
./target/release/hyprpilot-daemon &
./target/release/hyprpilot --use-daemon ws send +1
./target/release/hyprpilot undo
./target/release/hyprpilot daemon stop

Quick start (MCP for agents)

1. Run the daemon (persists across MCP sessions, owns undo state):

``sh ./target/release/hyprpilot-daemon & ``

1. Register the MCP server with your host. Example Claude Desktop config

(~/.config/Claude/claude_desktop_config.json): ``json { "mcpServers": { "hyprland": { "command": "/absolute/path/to/target/release/hyprpilot-mcp", "args": ["--profile", "default"] } } } ``

1. Restart the host. It will discover ~24 tools by default (read,

window, workspace, undo groups). Destructive and process tools are hidden unless you switch to a more permissive profile.

Capability profiles

Profiles live at ~/.config/hyprpilot/profiles/.toml. Built-in profiles available without a file:

• default — read + window + workspace + undo. Hides kill, close, exec.
• unrestricted — every tool. Use sparingly.

Profile file format:

# ~/.config/hyprpilot/profiles/strict.toml
allow = ["read", "window", "workspace", "undo"]
allow_tools = ["kill_active"]   # add specific tools outside the groups
deny_tools  = ["focus_window"]  # remove specific tools from the groups

Tool groups: read, window, workspace, destructive, process, undo.

Dry-run

Mutating tools accept a dry_run: bool argument, default true. With dry_run=true the server returns a preview ("would …") and does not touch the daemon. Agents must pass dry_run=false to actually mutate.

This is layered on top of capability profiles; both must agree before a mutation reaches the daemon.

Snapshots

Capture a known-good layout and restore to it later. Snapshots are JSON files under $XDG_STATE_HOME/hyprpilot/snapshots/.json.

./target/release/hyprpilot snapshot save before-meeting
./target/release/hyprpilot snapshot list
./target/release/hyprpilot snapshot diff before-meeting   # preview restore
./target/release/hyprpilot snapshot restore before-meeting
# Auto-saves a `_pre-restore-` snapshot so the restore itself is
# reversible.
./target/release/hyprpilot snapshot delete before-meeting

Restore is best-effort. It matches live windows to snapshot entries by address, then PID, then (initial_class, initial_title). For each match, the diff is computed across these dimensions:

• workspace — always.
• floating state — always.
• floating-window geometry — exact (x, y) and (w, h), via

movewindowpixel and resizewindowpixel. Tiled windows are layout-driven, so geometry diffs are suppressed for them.

• fullscreen mode — via fullscreenstate, no focus disturbance.
• pinned state — for floating windows only (Hyprland refuses to

pin tiled windows).

• active focus — the snapshot's focused window is refocused last

so it overrides any focus side-effects from earlier actions.

Re-spawn of missing windows (snapshot entries whose process is gone) is future work. Windows present live but absent from the snapshot are left alone; restore is never destructive.

Persistent undo

The daemon's undo stack is persisted to $XDG_STATE_HOME/hyprpilot/undo.json on every push/pop. Surviving a daemon restart means hyprpilot undo still works after the daemon's process dies. Malformed files are surfaced at startup; the stack starts empty rather than aborting.

Input synthesis (v0.5)

Type text, press key chords, move and click the mouse. Built on external tools plus Hyprland's native sendshortcut for per-window key delivery.

Backend routing per operation:

| Operation | Backend | Why | |---|---|---| | input type | wtype (virtual-keyboard) | Fast batched text into the focused window; bind-triggering is irrelevant for plain text. | | input keys | ydotool (uinput→libinput), falls back to wtype | ydotool events reach Hyprland's global bind matcher, so super+T fires its bind. wtype's virtual-keyboard events are filtered out of the bind matcher (focused client only). | | input mouse-move / mouse-click | ydotool | Pointer synthesis. | | input shortcut | Hyprland sendshortcut | Per-window, no focus change, no external backend. |

> input keys needs ydotoold running to trigger binds. hyprpilot > looks for its socket at $YDOTOOL_SOCKET, then > $XDG_RUNTIME_DIR/.ydotool_socket, then /tmp/.ydotool_socket. If > ydotoold is unreachable, input keys degrades to wtype — chords still > reach the focused window but won't trigger compositor binds. Run > ydotoold as a systemd --user service so it's always available.

This is the most dangerous tool surface in HyprPilot. An agent that can type can execute arbitrary shell commands in any focused terminal. Two independent gates apply before any input call reaches the wire:

1. Daemon env-var. The daemon refuses every Input* request

unless started with HYPRPILOT_DANGEROUS_INPUT_OK=1: ``sh HYPRPILOT_DANGEROUS_INPUT_OK=1 hyprpilot-daemon & ``

1. Capability profile. The input group is NOT in the built-in

default profile. To grant it, either use the built-in unrestricted profile or write a custom one — see docs/profile-with-input.example.toml: ``sh hyprpilot-mcp --profile unrestricted # or, with a custom profile: hyprpilot-mcp --profile my-input-profile ``

Both gates must align before an agent can synthesize input.

# CLI examples (require both gates above)
hyprpilot input type "hello world"
hyprpilot input keys ctrl+shift+t
hyprpilot input shortcut ctrl+t class:firefox
hyprpilot input mouse-move 100 200 --absolute
hyprpilot input mouse-click left

input_shortcut uses Hyprland's sendshortcut dispatcher and is strictly safer than input_keys — it targets a specific window by selector instead of going wherever focus happens to be.

Backends are detected at daemon startup; if wtype or ydotool is missing, the corresponding tools return input_backend_missing with the missing backend name.

Declarative rules

The daemon subscribes to Hyprland's event socket and matches each event against rules declared in $XDG_CONFIG_HOME/hyprpilot/rules.toml. Each rule pairs an event kind with field-equality predicates and a list of actions to fire on match.

[[rule]]
name = "slack to scratchpad"
on = "openwindow"
when = { class = "Slack" }
do = ["move_to_workspace_silent special:scratch"]

Semantics:

• First match wins. Rules evaluated top-to-bottom; first whose

on matches and whose when predicates all hold fires.

• Equality predicates only in v0.4 (no regex, ranges, negation).
• Reentrance guard. While the engine is executing actions, incoming

events skip rule matching, with a ~100 ms tail after actions finish. Prevents rule → action → echo → rule loops.

• Per-action failure isolation. A failing dispatcher logs a warning

and the rule's remaining actions still run.

Inspect and validate via CLI or MCP:

hyprpilot rules path        # where the daemon reads from
hyprpilot rules list        # current on-disk rules
hyprpilot rules validate    # parse + compile, before restarting

See docs/rules.example.toml for the full action grammar reference.

Editing rules.toml while the daemon is running has no effect — the engine uses the version it loaded at startup. Restart the daemon to reload.

Screen capture + OCR (v0.6)

The hyprpilot-vision crate shells out to grim (wlr-screencopy) for capture and tesseract for OCR. Both binaries are detected at construction time; missing → typed error.

# CLI — image bytes go to --output or stdout.
hyprpilot capture full   --monitor eDP-1  --output /tmp/screen.png
hyprpilot capture region 0 0 1920 1080    --output /tmp/region.jpeg --format jpeg

# OCR a region.
hyprpilot ocr region 100 200 800 400 --lang eng --psm 6

MCP tools (group: vision, opt-in only, NOT in the default profile):

| Tool | Result | |-----------------------|---------------------------------------------------------------------| | screenshot | Content::Image block (base64 PNG/JPEG); whole compositor | | screenshot_region | Content::Image; rectangular region | | screenshot_monitor | Content::Image; single named monitor | | ocr_screen | Content::Text; extracted text from the whole screen | | ocr_region | Content::Text; extracted text from a region |

Vision is excluded from Profile::default_safe() because screenshots can capture private content. To grant it, list vision in a custom profile's allow (see docs/profile.example.toml) or use --profile unrestricted for trusted local development.

Both backends are graceful about misuse:

• Missing binary → backend grim is not installed (or not on $PATH).
• Zero-dimension region → invalid region: width and height must be > 0.
• Tesseract finding no text in a blank image → empty string (success), not an error.

Design

• The compositor's IPC is the source of truth. We do not cache.
• Mutating operations are typed at the API boundary. No stringly-typed

dispatch in user code.

• The daemon is optional for the CLI; it's required for the MCP server

(undo state must outlive the MCP host's spawn lifecycle).

• Hyprland version is detected on connect; commands incompatible with

the live version fail fast with a typed error.

Milestones

• v0.1 (done): core + daemon + CLI, ~15 dispatchers, in-memory undo.
• v0.2 (done): MCP server, capability profiles, dry-run.
• v0.3 (done): snapshots (capture / list / diff / restore / delete),

persistent undo across daemon restarts.

• v0.4 (done):
• restore completeness: geometry, fullscreen, pin, active focus

in snapshot_restore.

• rules engine: daemon-side reactions to socket2 events with TOML

rule files, first-match semantics, reentrance-guarded execution.

• v0.5 (done):
• input synthesis: typing, key chords, mouse via wtype,

ydotool, and Hyprland's sendshortcut. Gated by env var + capability profile.

• v0.6 (in progress):
• screen capture: grim-backed PNG / JPEG capture (full,

per-monitor, region) via new hyprpilot-vision crate.

• OCR: tesseract-backed text extraction.
• MCP Content::Image with base64 PNG/JPEG plus 5 new tools

in ToolGroup::Vision (opt-in only).

• v0.7: composite tools (click_text, find_text_position); rich

preview for snapshot-restore that embeds the diff actions; libei backend scaffold (feature-flagged stub).

• v0.8: input keys routed through ydotool (uinput→libinput) so

Hyprland global binds fire — super+T now triggers its bind. XDG-aware ydotoold socket discovery. The libei migration was shelved (docs/libei-design.md): Hyprland has no RemoteDesktop portal, and the only community bridge falls back to virtual-keyboard, which Hyprland filters from its bind matcher — so libei would not have achieved the goal. ydotool already does.

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: sumerchoudhary78
• Source: sumerchoudhary78/HyprPilot
• License: Apache-2.0
• Homepage: https://sumerchoudhary78.github.io/HyprPilot/

Install and usage instructions live in the source repository linked above.