# Clawdcursor

> clawdcursor compiles whatever's on screen into one UI map — accessibility tree and OCR fused into stable, addressable elements, with a screenshot only when needed — then drives apps through reusable scripts, verifying every action and routing it through a single safety gate.

- **Type:** MCP server
- **Install:** `agentstack add mcp-amrdab-clawdcursor`
- **Verified:** Pending review
- **Seller:** [AmrDab](https://agentstack.voostack.com/s/amrdab)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [AmrDab](https://github.com/AmrDab)
- **Source:** https://github.com/AmrDab/clawdcursor
- **Website:** https://clawdcursor.com

## Install

```sh
agentstack add mcp-amrdab-clawdcursor
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

Clawd Cursor

  Safe desktop control for any AI agent. Compiles the screen into a UI map and acts on elements by stable id (screenshot/vision only as a last resort),
  verifies its own actions, and gates everything through one safety checkpoint. Local · cross-OS · any model.

  
  
  
  
  
  
  

  Quickstart ·
  Why it's different ·
  The engine ·
  How it works ·
  Tools ·
  Platforms ·
  Changelog

---

## What it is

Clawd Cursor is a **local MCP server** that gives any tool-calling agent — Claude Code, Cursor, Windsurf, OpenClaw, the Claude Agent SDK, or your own loop — safe control of the **real desktop**. It clicks, types, reads the screen, opens apps, and drives any GUI the way a human would: native apps, the browser, even a canvas.

Most "let an agent use the computer" tools take a screenshot and feed it to a vision model — slow, expensive, and brittle. Clawd Cursor **compiles the screen into one UI map**: it fuses the accessibility tree and OCR into a confidence-scored set of elements, each tagged with a stable `el_NN` id, and acts on elements by id — not pixel coordinates. Coordinates appear only in the last-resort screenshot/vision tier (live pixels off the current frame), for canvas-only apps or tasks that genuinely need spatial reasoning. The result is cheaper, faster, private, and — uniquely — it **checks that each action actually did what it claimed**.

> **If a human can do it on a screen, your agent can too.** No API, no integration, no problem — only the right sequence of reads, clicks, keys, and waits. Use it as the **last-mile fallback**: native API exists? Use it. CLI? Use it. Clawd Cursor is for the click, the legacy app, the GUI with no public surface.

---

## Why it's different

The desktop-agent space is crowded. The closest **install-and-go** peers are [Windows-MCP](https://github.com/CursorTouch/Windows-MCP) and [Terminator](https://t8r.tech/) (desktop MCP servers); browser-only tools (browser-use, Playwright MCP) are adjacent; and [OmniParser](https://github.com/microsoft/OmniParser) / [UI-TARS](https://github.com/bytedance/UI-TARS-desktop) are vision-centric *parsing approaches* you'd build an agent around, not products you install. Here's the honest comparison across those approaches — what Clawd Cursor does that the popular options don't:

|                                          | **Clawd Cursor**        | browser-use | Playwright MCP | OmniParser / UI-TARS | computer-use |
|------------------------------------------|:-----------------------:|:-----------:|:--------------:|:--------------------:|:------------:|
| Any desktop app, not just the web        | ✅                      | web only    | web only       | ✅                    | ✅           |
| Cross-OS (Windows + macOS + Linux)       | ✅                      | —     | —        | varies               | sandbox      |
| Perception **without a vision model**    | ✅ compiled a11y + OCR map | DOM      | a11y tree      | ❌ vision-centric     | ❌ vision    |
| **Verifies its own actions** (deviation) | ✅                      | —     | —        | —              | —      |
| Single safety chokepoint (allow/confirm/block) | ✅                | —     | —        | —              | —      |
| Any model / vendor                       | ✅                      | ✅          | not an agent   | model-specific       | Claude only  |
| MCP-native (one config, any host)        | ✅                      | library     | test framework | —              | tool-use API |
| Local-only, no cloud required            | ✅                      | ✅          | ✅             | needs a model        | screens → cloud |

Three things here are genuinely rare:

1. **Cheapest-tier-first perception, fully local.** Accessibility tree (free) → OCR (cheap) → screenshot (expensive — the *only* tier that puts pixels in the model's context; "screenshot" and "vision" are the same step). The agent climbs only when it must, so token cost tracks task difficulty — and with a local model, nothing leaves the machine. Vision-centric agents (OmniParser, UI-TARS) need a screenshot in the model for *every* observation.
2. **It verifies.** Pass `expect` on a consequential action and Clawd Cursor re-checks the live screen (with a short settle window for async UIs) and reports a **DEVIATION** instead of a hollow "success." A completed task can't be marked done on evidence that was already true before it acted.
3. **One safety gate.** Every call — from an editor over stdio, an external agent over HTTP, or the built-in loop — routes through a single `safety.evaluate()` chokepoint (allow / confirm / block) before it touches the desktop. The agent cannot bypass it.

Plus: an on-screen **"desktop control in progress" banner** with a blinking red dot whenever an agent is driving — **double-click it to stop.** A human at the machine always knows, and always has a kill switch.

---

## Install

clawdcursor is an MCP server published to npm — install it into **any** MCP-capable
agent (Claude Code, Claude Desktop, Cursor, Windsurf, Zed, OpenAI Codex, or your own
loop) the same way you install any other MCP server.

### 1 — Install the engine + grant consent (once)

```bash
npm i -g clawdcursor
clawdcursor consent --accept    # one-time desktop-control consent (required)
clawdcursor grant               # macOS only — approve Accessibility + Screen Recording
```

> **Zero-install** also works — swap `clawdcursor` for `npx -y clawdcursor` in any
> snippet below and npx fetches it on demand. A **global install is recommended**
> anyway: it's pinnable and inspectable on disk (safer for a tool with full desktop
> control than auto-fetching `latest` every run), and it's the path on which the macOS
> native helper builds at install time. Requires **Node.js 20+**.

> **Per-OS prerequisites.** **Windows** installs clean — `sharp` and `@nut-tree-fork/nut-js`
> ship prebuilt binaries, so no C++/Python build tools are needed. **macOS** needs Xcode
> Command Line Tools (`xcode-select --install`) for screenshots / vision; core
> accessibility-driven control still works without them. **Linux** needs a few system
> packages npm can't install: `tesseract-ocr` (OCR), `python3-gi` + `gir1.2-atspi-2.0`
> (accessibility tree), and — on Wayland — `ydotool` (synthetic input).

### 2 — Add it to your agent (pick your host)

**Claude Code**
```bash
claude mcp add clawdcursor -s user -- clawdcursor mcp --compact
```

**OpenAI Codex** — add to `~/.codex/config.toml`:
```toml
[mcp_servers.clawdcursor]
command = "clawdcursor"
args = ["mcp", "--compact"]
```

**Cursor / Windsurf / Zed / Claude Desktop** — add to the host's MCP config:
```json
{
  "mcpServers": {
    "clawdcursor": { "command": "clawdcursor", "args": ["mcp", "--compact"] }
  }
}
```

That's the whole setup. Ask your agent: *"open Outlook and reply to the latest email
from Sarah."*

### Or: one-command plugin (Claude Code)

Skip the manual config — this repo ships a plugin that registers the tools **and**
bundles the usage skill in one step. It resolves the package's `bin` (never a
hard-coded `dist/` path), so an upgrade can't break it:

```bash
claude plugin marketplace add AmrDab/clawdcursor
claude plugin install clawdcursor@clawdcursor
```

One-line installers (clone + build; handles the macOS native build)

```powershell
# Windows (PowerShell)
powershell -c "irm https://clawdcursor.com/install.ps1 | iex"
```
```bash
# macOS / Linux
curl -fsSL https://clawdcursor.com/install.sh | bash
```

> **Notes.** You never run `clawdcursor mcp` yourself — the host spawns it over stdio
> on demand. `clawdcursor doctor` is **not** part of MCP setup; it only configures the
> built-in LLM for the autonomous `agent` daemon. On macOS, **Accessibility is
> required** (primary control path); **Screen Recording is optional** (vision fallback
> only). For editor permission allowlists, use the server-level wildcard
> `mcp__clawdcursor` rather than per-tool entries — it survives tool renames.

---

## The engine

The perception + verification core (the **UI State Compiler**, since v1.5.0):

- **`compile_ui`** fuses the accessibility tree and OCR into one confidence-scored map of the screen, every element tagged with a stable `el_NN` id. Act on an element by `{element_id, snapshot_id}` instead of pixels — near-free in tokens, and it survives DPI, resize, and layout shifts. `find_button` / `find_field` locate a target by meaning and hand you the id.
- **Reactive verification.** `expect` on an action → Clawd Cursor confirms the outcome on the live screen and returns a **DEVIATION** when the UI didn't obey.
- **Cross-platform parity.** The compiler, secure-field redaction, and coordinate handling run on Windows, macOS, and Linux; the external-agent (MCP) surface resolves `el_NN` refs through the safety gate and discloses when it attached to your existing browser.

> Set-of-Mark-style element IDs and a11y/OCR fusion aren't new ideas on their own — what's rare is doing them **locally, a11y-first (no vision model required), with a built-in verification gate and one safety chokepoint**, across three operating systems, behind a single MCP config.

See the [changelog](CHANGELOG.md) for the full release history, or the [latest release](https://github.com/AmrDab/clawdcursor/releases/latest).

---

## How it works

**Where the brain lives** decides how you run it. Both modes can run side-by-side.

| Brain lives… | Mode | Command | What you call |
|---|---|---|---|
| In your editor (Claude Code, Cursor, Windsurf, Zed) | Direct tools | `clawdcursor mcp` | Each tool, via stdio MCP |
| In a headless agent with its own LLM (OpenClaw, Agent SDK, your loop) | Direct tools | `clawdcursor agent --no-llm` | Same, over HTTP MCP |
| Inside Clawd Cursor itself (scheduled / "submit and walk away") | Thin agent loop | `clawdcursor agent` + `doctor`-configured LLM | `task` / `submit_task` |
| External brain that delegates sub-tasks to the built-in loop | Direct + delegation | `clawdcursor agent` + your client | `task({instruction:…})` to hand off |

### The loop

Read the a11y tree (cheap) → act on named targets → verify from fresh observations → escalate perception only when needed (OCR → screenshot, the one tier that sends pixels to the model). Sparse a11y tree? `system.detect_webview` switches Electron/WebView2 apps to `browser.*` over CDP. Canvas-only (Paint, Figma, games)? Screenshot + coordinate click.

```mermaid
flowchart TB
    task["User task"] --> loop["Agent LLM loopplans · chooses tools · verifies"]
    loop --> observe{"Cheapest observationthat answers the question"}

    observe -- "obs·a11y — free" --> a11y["A11y tree(structured text + el_NN handles)"]
    observe -- "obs·ocr — cheap" --> ocr["OCR (OS-level, no vision LLM)"]
    observe -- "obs·dom — medium" --> dom["Browser DOM (CDP)"]
    observe -- "obs·vision — expensive" --> vision["Screenshot (image into context)"]

    a11y --> act
    ocr --> act
    dom --> act
    vision --> act

    act["Actclick/type/key/drag · invoke/set_value · open_app · batch"] --> safety
    safety["Single safety gatesafety.evaluate() → allow / confirm / block"] -- allowed --> tools["Tool registry98 granular + 7 compound"]
    safety -- needs user --> confirm["Human confirmation"] --> tools
    safety -- denied --> blocked["blocked"]

    tools --> desktop["Real desktop"]
    desktop --> verify{"expect → does state match?"}
    verify -- pass --> done["done"]
    verify -- "DEVIATION" --> loop

    classDef agentNode fill:#dbeafe,stroke:#2563eb,color:#0f172a;
    classDef gate fill:#ede9fe,stroke:#7c3aed,color:#0f172a;
    classDef obsNode fill:#fef9c3,stroke:#ca8a04,color:#0f172a;
    classDef actNode fill:#ffedd5,stroke:#ea580c,color:#0f172a;
    classDef stop fill:#fee2e2,stroke:#dc2626,color:#0f172a;
    class loop,verify agentNode;
    class safety,confirm,tools gate;
    class observe,a11y,ocr,dom,vision obsNode;
    class act actNode;
    class blocked stop;
```

**`batch` for deterministic stretches.** When the next N steps are known, collapse them into one call — each step still routes through the safety gate; on any guard miss or error the batch halts with a per-step trace.

**Task delegation.** With an LLM configured on the daemon, an external agent can hand off at any point: `task({"instruction":"…"})`. The built-in loop takes the wheel and reports back — offload grunt work to a cheaper model without burning your own context.

---

## The toolbox

Two catalogs ship side-by-side. The **toolbox** is 7 compound tools, each with an `action` enum covering ~10–20 verbs (~1,500 tokens total — about 12× smaller than granular, the `computer_20250124` shape editor hosts already know). The **granular** surface is the 98 underlying primitives, one schema per verb (for runtimes that need top-level tools, or for debugging). Both run through the same `safety.evaluate()` chokepoint; the full catalog is always visible via MCP `tools/list`.

| Toolbox | Actions |
|---|---|
| `computer` | `screenshot`, `click`, `double_click`, `right_click`, `triple_click`, `hover`, `move`, `scroll`, `scroll_horizontal`, `drag`, `drag_path`, `type`, `key`, `wait` |
| `accessibility` | `read_tree`, `find`, `get_element`, `focused`, `invoke`, `focus`, `set_value`, `get_value`, `expand`, `collapse`, `toggle`, `select`, `state`, `list_children`, `wait_for`, `compile_ui`, `find_button`, `find_field`, `smart_click`, `smart_type`, `smart_read` |
| `window` | `list`, `active`, `focus`, `maximize`, `minimize`, `restore`, `close`, `resize`, `list_displays`, `screen_size`, `open_app`, `open_file`, `open_url`, `switch_tab`, `navigate` |
| `system` | `clipboard_read`, `clipboard_write`, `system_time`, `ocr`, `undo`, `shortcuts_list`, `shortcuts_run`, `delegate`, `detect_webview`, `relaunch_with_cdp`, `system_prompt`, `build_uri`, `open_uri`, `open_app`, `open_file`, `open_url`, `detect_app`, `app_guide`, `learn_app` |
| `browser` | `connect`, `page_context`, `read_text`, `click`, `type`, `select_option`, `evaluate`, `wait_for`, `list_tabs`, `switch_tab`, `scroll` |
| `task` | `run` (default; **bounded-sync** — waits up to `timeout`s, returns `{status:"running"}` + progress if longer, re-call to keep waiting), `status`, `abort`. Delegates to the built-in loop. Requires `clawdcursor agent` with an LLM. |
| `batch` | `{steps:[…]}` — collapse N calls into one round-trip; each step `{name, arguments, expect?}`, re-perceived and safety-gated, halts with a trace on any miss. |

```js
computer({ action: "key", combo: "mod+s" })          // Cmd+S / Ctrl+S, resolved per-OS
accessibility({ action: "invoke", name: "Send" })    // click by name, not pixels
window({ action: "open_app", name: "Outlook" })
task({ instruction: "open Notepad and type hello" }) // hand off to the thin loop
```

---

## Cheapest-tier-first perception

Every observation has a cost. Start at the cheapest rung that works; climb only when it fails. The live log (`CLAWD_LOG=pretty`, default on a TTY) shows the ladder in real time via per-call badges.

| Tier | Badge | Cost | Source | When |
|---|---|---|---|---|
| **T1** structured | `obs·a11y` | ~free | `accessibility.*`, `window.*`, `browser.read_text`, clipboard | Default. Text + bounds, no image, no vision LLM. |
| **T2** OCR | `obs·ocr` | cheap | `system.ocr`, `smart_read` / `smart_click` / `smart_type` | A11y tree empty/sparse. OS-level text out, no image bytes. |
| **T3** DOM | `obs·dom` | medium | `browser.read_text` / `page_context` (CDP) | WebView / Electron / Chrome content. |
| **T4** screenshot (vision) | `obs·vision` | expensive | `computer.screenshot` | The only tier that puts pixels in the model's context. Canvas-only apps or spatial reasoning. Last resort. |

Acting tools log `act`. Watching `obs·a11y → act → obs·a11y

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [AmrDab](https://github.com/AmrDab)
- **Source:** [AmrDab/clawdcursor](https://github.com/AmrDab/clawdcursor)
- **License:** MIT
- **Homepage:** https://clawdcursor.com

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: flagged — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-amrdab-clawdcursor
- Seller: https://agentstack.voostack.com/s/amrdab
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.