Open Codex Computer Use

🚀 Open Claudex Computer Use

Background computer use for Claude Code, Codex, and MCP agents on macOS.

• Open Source Codex-style Computer Use • Native Swift MCP Server • App-Aware Virtual Cursor •

中文文档 • News • Features • Demo • Downloads • Quick Start • Compatibility • Agent Guide

> [!IMPORTANT] > 🖥️ From GUI to Agent UI. In December 1979, when Steve Jobs saw the GUI at Xerox PARC, it became obvious that computers needed a new interface. The first time I saw Codex Computer Use, I felt a smaller version of that: agents need their own lane to use real apps and coexist better with human UI work. > > 🧭 Background computer use. An agent should not have to steal your mouse and keyboard to get work done. It should operate in a separate lane, stay visible through an app-aware virtual cursor, and keep enough screenshot + Accessibility context to recover. > > 🔓 Why open source it? The official Codex Computer Use MCP is not open source. Open Claudex is an open macOS execution layer for Claude Code, Codex, and other MCP harnesses.

📮 News

• [2026.05.12] 🪽 Hermes Agent Computer Use is here too. The direction is getting clearer: agents need their own background lane, not your cursor. ✨
• [2026.05.11] 🦞 OpenClaw joined the computer-use wave with a Computer Use skill. Real-app operation is becoming a standard agent primitive.
• [Launch day] 🚀 Open Claudex Computer Use is public: an open-source macOS background computer-use layer for Claude Code, Codex, and MCP agents.

🧭 Quick Navigation

> [!TIP] > I'm a human -> Continue reading this README for demos, setup, compatibility, and project context. > > I'm an agent -> Read [CLAUDE.md](CLAUDE.md) for structured operating instructions, key files, and command quick reference.

claudex-computer-use is a native Swift MCP server that lets AI agents inspect and operate real Mac apps without moving your real mouse or requiring a cloud desktop.

• For Claude Code and Codex: local stdio MCP server plus a Codex plugin scaffold.
• For real Mac apps: Safari, Notes, Finder, Calculator, TextEdit, System Settings, and best-effort WebView-heavy apps such as Feishu/Lark.
• For demos and trust: app-aware virtual cursor overlay, post-action screenshots, and Codex-style responses.

Status: 0.1.0-alpha

> Not affiliated with Anthropic, OpenAI, Apple, or the official Codex Computer Use plugin.

🎬 Demos

| Native App Control | Background Cross-App Work | Feishu / Lark Best-Effort | |---|---|---| | | | | | Click and read native macOS apps through Accessibility, with a visible virtual cursor. | Let the agent work in Safari and Notes while you keep using the Mac. | Operate WebView-heavy enterprise apps with mixed AX and coordinate fallbacks. Sanitized demo data only. |

⚡ Quick Start

Tell your coding agent:

> Install Open Claudex Computer Use from https://github.com/OpenClaudex/open-claudex-computer-use and configure it as an MCP server for my agent.

Requires macOS 13+, Swift 5.9+, Accessibility permission, and Screen Recording permission. For manual setup, see [Installation & Integration](docs/install.md).

✨ Features

Open Claudex focuses on the native macOS execution layer:

• Reads app state through Accessibility and screenshots.
• Performs clicks, scrolling, dragging, keyboard input, text entry, and AX actions.
• Returns post-action state so agents can continue without excessive re-snapshotting.
• Shows a same-process virtual cursor for observation and recordings.
• Supports both NDJSON and Content-Length MCP stdio framing.

For agent-facing usage rules, tool behavior, and recovery patterns, read [Agent Guide](docs/agent-guide.md).

🧩 Compatibility

| Tier | Apps | Expected Behavior | |---|---|---| | Stable | Safari, Notes, TextEdit, Calculator, Finder, System Settings | Strong AX tree, screenshots, semantic clicks, set_value | | Limited | Chrome, Edge, VS Code, Slack, Cursor | Partial AX, coordinate fallback, pasteboard-heavy typing | | Best-effort | WeChat, Feishu/Lark, self-drawn or WebView-heavy surfaces | Sparse AX, unreliable frames, more fallback logic |

Details: [App Compatibility Matrix](docs/compatibility.md)

🧪 Why This Exists

This project started from two converging workflows: Codex-style background computer use and Claude Code-style MCP extensibility. The missing piece was a reusable open-source execution layer: a local macOS MCP server that any agent harness can plug into.

Open Claudex is not a full agent harness. It is the execution engine.

📚 Docs

• [Installation & Integration](docs/install.md)
• [Agent Guide](docs/agent-guide.md)
• [Demo Pack](docs/demos.md)
• [App Compatibility Matrix](docs/compatibility.md)
• [Testing](docs/testing.md)
• [Codex Native Trace Kit](docs/codex-native-trace-kit.md)
• [Roadmap](ROADMAP.md)

🌐 Related Projects

Open Claudex focuses on the native macOS execution layer. Related projects around computer use and agent desktops:

• iFurySt/open-codex-computer-use - open-source Codex-style computer-use MCP server.
• trycua/cua - sandbox, SDK, and infrastructure for full desktop computer-use agents.
• browser-use/macOS-use - making macOS apps accessible to AI agents.

⭐ Star History

📄 License

[MIT](LICENSE)

---

If this project helps you, please give it a ⭐ Star!

Report Issues · Feature Requests

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: OpenCodexLabs
• Source: OpenCodexLabs/open-codex-computer-use
• License: MIT

Install and usage instructions live in the source repository linked above.