Flutter Agent

Flutter Agent

The control plane your AI agent uses to drive a real Flutter app.

Flutter Agent runs a REST API on localhost:8000. AI coding agents (Claude Code, Cursor, Antigravity, your own) and ordinary HTTP clients use it to observe and control a Flutter app on an Android emulator or device — screenshots, UI tree inspection, taps / swipes / typing, hot reload, structured Dart logs, real heap + FPS + CPU profiles.

> v1.0.0 — production-ready. ADB control, multi-attribute UI matchers, > logcat + Dart-native log streaming, Dart VM Service (memory + FPS + CPU > profiling), Flutter daemon protocol (real source-level hot reload), and > a generated tool-definition file that lets any modern tool-calling > agent framework drive every endpoint. See [CHANGELOG.md](CHANGELOG.md).

---

Why this exists

AI coding agents can write Flutter code, but they're blind to what's actually rendering. Without a feedback loop, they iterate by guessing — "did my change work? did I break something? what state is the app in?"

Flutter Agent gives them eyes and hands:

[Claude Code / Cursor / Antigravity / your agent]
                │ HTTP / SSE
                ▼
         [Flutter Agent server]
                │
         ┌──────┴──────┐
         ▼             ▼
       ADB         Dart VM Service
         │             │
         ▼             ▼
   [Android emulator or device]
              │
        flutter run --machine
              │
        [your Flutter app]

The agent loop becomes:

1. Make a code change.
2. POST /hot_reload.
3. GET /screenshot to see what rendered.
4. GET /flutter_logs or GET /errors to see what happened.
5. Iterate.

Quick Start

Prerequisites: Python 3.10+, the Android SDK platform-tools (for adb), and a running emulator or USB-attached device.

pip install -r requirements.txt
uvicorn app:app --reload

Open http://localhost:8000/docs for the interactive OpenAPI explorer.

# Sanity check
curl http://localhost:8000/health

# Take a screenshot
curl http://localhost:8000/screenshot > screen.png

# Tap at (150, 250)
curl -X POST http://localhost:8000/tap \
  -H "Content-Type: application/json" \
  -d '{"x":150,"y":250}'

# Launch a Flutter app and auto-connect to its VM Service
curl -X POST http://localhost:8000/flutter_run/start \
  -H 'Content-Type: application/json' \
  -d '{"project_path":"/path/to/your/flutter_project"}'

# Tail Dart `print()` / `developer.log()` in real time
curl -N 'http://localhost:8000/flutter_logs/stream'

See [QUICKSTART.md](QUICKSTART.md) for the 5-minute walkthrough, [ARCHITECTURE.md](ARCHITECTURE.md) for the system design, and [APIREFERENCE.md](APIREFERENCE.md) for every endpoint.

Endpoints

| Endpoint | Method | Purpose | |----------|--------|---------| | /health | GET | Server + device status | | /screenshot | GET | Raw PNG of the device screen | | /app_state | GET | Screenshot (base64) + UI element list | | /find_element | GET | Locate an element by text / contentdesc / resourceid / classname | | /elements | GET | List elements with filters (clickable, enabled, textcontains, …) | | /tap | POST | Tap at coordinates | | /tap_element | POST | Find an element by query and tap it | | /swipe | POST | Swipe gesture | | /type_text | POST | Type into focused field | | /key_event | POST | Send Android keycode (BACK, HOME, ENTER, …) | | /wait_for_element | POST | Poll the UI tree until an element appears, or timeout | | /flutter_run/start | POST | Launch flutter run --machine and auto-connect the VM Service | | /flutter_run/stop | POST | Stop the running daemon | | /flutter_run/status | GET | Running / appId / wsUri / uptime | | /flutter_run/send_command | POST | Escape hatch — send any app.* / daemon.* RPC | | /vm_service/connect | POST | Attach to a Dart VM Service WebSocket (manual flow) | | /vm_service/disconnect | POST | Tear down the VM Service connection | | /vm_service/status | GET | URL + connected isolates | | /isolates | GET | List running Dart isolates | | /memory | GET | Real heap usage via getMemoryUsage | | /performance | GET | FPS / jank from the Dart VM Timeline | | /cpu_profile | GET | Top CPU hotspots from getCpuSamples over a window | | /hot_reload | POST | Source-level reload (via daemon) or reassemble fallback | | /hot_restart | POST | Engine restart | | /app_logs | GET | Recent logcat lines, with tag / level / regex filters | | /clear_logs | POST | Clear the device's logcat buffer | | /errors | GET | Parsed crashes (Flutter unhandled, AndroidRuntime fatals, ANRs) | | /frame_warnings | GET | Choreographer skipped-frame and Davey signals | | /logs/stream | GET (SSE) | Real-time logcat stream with optional filters | | /flutter_logs | GET | Buffered Dart print / developer.log entries since a cursor | | /flutter_logs/stream | GET (SSE) | Real-time stream of Dart log entries | | /flutter_logs/clear | POST | Empty the in-memory log buffer | | /app_info | GET | Package version + foreground state | | /full_diagnostics | GET | Combined snapshot |

UI matchers

find_element, tap_element, and wait_for_element all accept any combination of text, content_desc, resource_id, class_name, plus a match mode of exact (default), contains, or regex. content_desc is essential for Flutter — Flutter widgets expose their accessibility label there, not in text.

Driving a Flutter app

The recommended flow — Flutter Agent launches and owns flutter run --machine for you, one call end-to-end:

curl -X POST http://localhost:8000/flutter_run/start \
  -H 'Content-Type: application/json' \
  -d '{"project_path":"/path/to/your/flutter_project","device_id":"emulator-5554"}'

On success the response contains app_id, the live VM Service ws_uri, and vm_service_connected: true — meaning /memory, /performance, /hot_reload, and /hot_restart are all immediately wired up, and the reload endpoints route through the daemon's app.restart for real source-level reload rather than the widget-tree reassemble fallback.

Set FLUTTER_PROJECT_PATH to skip project_path in the body:

export FLUTTER_PROJECT_PATH=/path/to/your/flutter_project
curl -X POST http://localhost:8000/flutter_run/start -d '{}'

Manual VM Service flow

If you'd rather keep flutter run open in your own terminal:

flutter run -d emulator-5554
# A Dart VM Service on Pixel 9a is available at: http://127.0.0.1:64321/aBcDef=/

curl -X POST http://localhost:8000/vm_service/connect \
  -H 'Content-Type: application/json' \
  -d '{"url": "http://127.0.0.1:64321/aBcDef=/"}'

…or set VM_SERVICE_URL before starting the server.

In this mode /hot_reload falls back to ext.flutter.reassemble (widget-tree rebuild only — useful for setState / theme changes that have already been applied) and returns a hint field telling you to either /flutter_run/start or press r in your flutter run terminal. The response's mechanism field (flutter_daemon / vm_service_reassemble_fallback / vm_service_forced_reload) makes the choice transparent.

Flutter-native logs vs. logcat

Two different log surfaces, each better for a different job:

| Endpoint | Source | Use when | |---|---|---| | /app_logs, /logs/stream | Android logcat — everything on the device | You want native crash reports, Choreographer warnings, ANRs, or non-Flutter system noise | | /flutter_logs, /flutter_logs/stream | Dart VM Service Stdout / Stderr / Logging streams + Flutter daemon app.log | You want just your app's output — print(), debugPrint(), structured developer.log() records with level / name / error / stack-trace preserved |

The /flutter_logs endpoint maintains a 5,000-entry ring buffer indexed by a monotonic seq, so an agent can poll with ?since= and get exactly the new entries each call — no log loss across polls.

Features

• ADB bridge — screenshots, UI hierarchy parsing, tap / swipe / type,

Android keycodes, multi-attribute element matchers with exact / contains / regex modes, /elements filtered listing, /wait_for_element polling.

• Logs & observability — filtered logcat reads, SSE streaming,

parsed crash extraction (Flutter unhandled, AndroidRuntime fatals, ANRs), Choreographer frame warnings, package metadata + foreground detection.

• Dart VM Service — heap usage via getMemoryUsage, FPS + jank from

getVMTimeline, top CPU hotspots from getCpuSamples, isolate enumeration, VM-Service-backed hot reload + hot restart.

• Flutter daemon protocol — own flutter run --machine end-to-end,

auto-connect the VM Service, route hot reload through app.restart for real source-level reload, escape-hatch RPC for any app.* / daemon.* method.

• Flutter-native log streaming — subscribes to VM Service Stdout /

Stderr / Logging and the Flutter daemon's app.log events; both poll (since cursor) and SSE-push API.

• Agent SDK adapters — [examples/agent_tools/](examples/agent_tools/)

ships a generator (generate_tools.py) that converts the server's OpenAPI spec into 32 Anthropic-Messages-API tool definitions, plus a runnable agent example (harness_agent.py) that drives Flutter Agent via Claude tool use. The same JSON serves as the tool contract for Claude Code (via MCP), Cursor, Antigravity, and any other tool-calling framework.

Docker

docker-compose up -d
curl http://localhost:8000/health

On macOS/Windows the container reaches a host-side ADB server over TCP via host.docker.internal:5037 — run adb start-server on the host first. See the comments in [docker-compose.yml](docker-compose.yml) for details.

Development

pip install -r requirements.txt
pytest -v --cov=core --cov=app

Tests mock subprocess.run, so the suite runs without a real device. CI (GitHub Actions) enforces ≥80% coverage on core/ and app.py.

See [CONTRIBUTING.md](CONTRIBUTING.md).

License

MIT — see [LICENSE](LICENSE).

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: elysian12
• Source: elysian12/flutter-agent
• License: MIT

Install and usage instructions live in the source repository linked above.