Browser Mcp

Agent Web Interface

Agent Web Interface is an MCP server that gives AI agents a compact, semantic interface to the browser.

Website · npm · vs. Playwright MCP

Instead of exposing the full DOM or accessibility tree, it returns structured page snapshots: visible regions, readable content, interactive elements, stable element IDs, form context, screenshots, canvas inspection, and network activity. Agents can then navigate and act on pages using semantic IDs instead of brittle selectors or massive context dumps.

It is built for coding agents, browser agents, QA agents, research agents, and automation workflows that need reliable web interaction without wasting tokens on low-signal browser internals.

---

Why this exists

Browser automation is easy for scripts and hard for LLM agents.

Traditional browser tools expose either raw DOM, full accessibility trees, screenshots, or low-level selectors. That works for deterministic code, but it is inefficient for language models. The model has to spend context and reasoning budget separating useful UI intent from implementation noise.

Agent Web Interface changes the interface boundary.

The browser still runs through Puppeteer and Chrome DevTools Protocol, but the agent sees a smaller, more semantic representation of the page:

• What regions exist on the page
• What the user can read
• What the user can interact with
• Which elements are visible, enabled, selected, expanded, or required
• Which stable eid should be used for the next action
• What changed after the previous action

The goal is not to mirror the browser. The goal is to expose the page in the shape an agent can reason about.

---

The core abstraction

Agent Web Interface turns a browser page into an agent-readable snapshot.

A snapshot contains compact semantic information such as:

• Page regions: header, navigation, main content, footer
• Interactive elements: buttons, links, textboxes, checkboxes, radios, comboboxes
• Readable content: headings, paragraphs, alerts, labels
• Element state: visible, enabled, checked, selected, expanded, focused
• Layout hints: bounding boxes and screen zones
• Stable element IDs: eid values that can be reused by action tools

Agents act on these IDs:

{
  "eid": "btn-sign-in"
}

rather than reasoning from fragile CSS selectors or repeatedly scanning a large DOM tree.

This makes browser use more predictable for agents because observation and action are connected through a stable semantic contract.

---

What it is

Agent Web Interface is:

• An MCP server for browser automation
• A semantic observation layer over Puppeteer and CDP
• A compact page representation for LLM agents
• A stable eid-based action interface
• A toolset for navigation, interaction, forms, screenshots, canvas, readability, and network inspection

Agent Web Interface is not:

• A replacement for Puppeteer
• A general-purpose browser
• A visual testing framework
• A scraping framework
• A CAPTCHA or anti-bot bypass tool

Puppeteer and CDP remain the execution layer. Agent Web Interface changes what the agent sees and how it decides what to do next.

---

How it works

At a high level:

1. The agent calls a browser tool through MCP.
2. Agent Web Interface controls Chrome through Puppeteer and CDP.
3. The current page is reduced into semantic regions, readable content, and actionable elements.
4. The agent receives a compact snapshot instead of a raw browser dump.
5. The agent acts using stable element IDs.
6. Agent Web Interface waits for the page to stabilize and returns the updated state.

This keeps browser lifecycle, page representation, and action execution separated.

AI Agent
   ↓ MCP
Agent Web Interface
   ↓ semantic snapshots + stable eids
Puppeteer / Chrome DevTools Protocol
   ↓
Chrome / Chromium

---

Example agent loop

A typical browser-agent loop looks like this:

1. The agent calls navigate with a URL.
2. Agent Web Interface returns a compact page snapshot.
3. The agent calls find to locate a semantic element, such as a “Sign in” button or an email field.
4. The agent calls click, type, select, or press using the returned eid.
5. Agent Web Interface waits for the page to stabilize and returns the updated snapshot.
6. The agent continues from the changed page state instead of re-reading the entire DOM.

This gives the model a browser interaction loop based on semantic state transitions rather than raw page internals.

---

Example user journey

The following example shows how an agent might use Agent Web Interface inside a dashboard-style web app.

Task:

> Find the failed payment from client@example.com, open it, add an internal note saying “Customer contacted. Waiting for bank confirmation.”, and confirm the note was saved.

1. Agent navigates to the payments dashboard

Tool call:

{
  "tool": "navigate",
  "input": {
    "url": "https://dashboard.example.com/payments"
  }
}

Tool response:

  
  

  
    Home
    Payments
    Customers
    Reports
  

  
    Payments
    Create payment
    Export
    Search payments
    Status
    Date

    ₹12,500 succeeded nadeem@example.com Jun 6
    ₹8,999 failed client@example.com Jun 6
    ₹2,400 refunded test@example.com Jun 5
  

The agent does not need to inspect a DOM table. It sees the relevant row directly as semantic state.

2. Agent opens the failed payment

Tool call:

{
  "tool": "click",
  "input": {
    "eid": "payment-row-2"
  }
}

Tool response:

  
  

  
    Payment ₹8,999
    Failed

    Customer client@example.com
    Payment method UPI
    Failure reason Bank declined transaction

    Timeline
    Payment created Jun 6, 10:42 AM
    Payment failed Jun 6, 10:43 AM

    Internal notes
    Add an internal note
    Save note
  

  
    Refund
    Retry payment
    Copy payment ID
  

Because this is a navigation, the response is a new baseline. The previous row IDs are no longer assumed valid.

3. Agent types the internal note

Tool call:

{
  "tool": "type",
  "input": {
    "eid": "inp-internal-note",
    "text": "Customer contacted. Waiting for bank confirmation.",
    "clear": true
  }
}

Tool response:

  
  

  
    Add an internal note
  

The agent receives only the changed field, not the whole page again.

4. Agent saves the note

Tool call:

{
  "tool": "click",
  "input": {
    "eid": "btn-save-note"
  }
}

Tool response:

  
  
    Note saved.
  

  
    Note saved.
  

  
    Customer contacted. Waiting for bank confirmation.
    Save note
  

  
    Note saved.
  

The agent can now conclude that the note was saved because both signals are present:

1. A status appeared: Note saved.
2. A new note row appeared with the submitted text.

5. Agent final answer to the user

Done. I opened the failed ₹8,999 payment for client@example.com, added the internal note, and confirmed that the dashboard showed “Note saved.”

This journey demonstrates the core interaction model:

navigate → baseline state
click row → new page baseline
type note → small diff
click save → diff + observation + confirmation state

The agent does not receive raw DOM, brittle selectors, or a massive accessibility tree. It receives a compact state transition after every browser action.

---

Tool response format

Most browser interaction tools return a compact XML state response.

The response is designed for LLM agents, not for humans reading browser internals. It favors stable semantic IDs, short tags, page regions, and incremental diffs over full DOM dumps.

A typical response contains:

• ``: current page state, title, URL, and step number
• ``: viewport, scroll position, and active interaction layer
• ` or `: whether this is a full page state or an incremental update
• ``: important transient UI changes such as dialogs, alerts, and toasts
• ``: grouped actionable elements by semantic page region
• Short element tags such as `, , , , , and `

Agents should use the id attributes, not CSS selectors, when performing follow-up actions.

Baseline response

This is the shape after navigate when the agent has no previous state.

  
  

  
    Acme
    Docs
  

  
    Sign in
    Email
    Password
    Remember me
    Sign in
  

Diff response

For same-page interactions, Agent Web Interface returns only the meaningful change.

  
  

  
    Email
  

Validation error

Readable mutations are rendered inline inside ``.

  
  
    Invalid email or password.
  

  
    Password
    Sign in
  

  
    Invalid email or password.
  

Modal response

Overlay layers such as modals, popovers, and drawers show the complete active overlay so the agent can reason about available actions.

  
  

  
    
      Delete product?
      This action cannot be undone.
      Delete
    
  

  
    Delete product?
    Cancel
    Delete
  

Trimmed large region

When a page has many repeated elements, large regions may be trimmed. The agent can call find with the region to retrieve more.

  
  

  
    Invoice 1001
    Invoice 1002
    Invoice 1003
    Invoice 1004
    Invoice 1005
    
    Invoice 1048
    Invoice 1049
    Invoice 1050
  

---

Real-world response examples

These examples are illustrative, but shaped according to the actual response contract: `, , or , optional `, and region-grouped short element tags.

GitHub-style issue page

  
  

  
    GitHub
    Search or jump to...
    Pull requests
    Issues
    Create new...
  

  
    Code
    Issues
    Pull requests
    Actions
  

  
    Checkout flow fails on Safari
    Open
    nadeem
    Edit
    Copy link

    nadeem commented
    Safari users cannot complete checkout after selecting Apple Pay.
    Add reaction
    Comment options

    Leave a comment
    Comment
    Close issue
  

  
    Assignees
    Labels
    Projects
    Milestone
  

Linear-style issue list

  
  

  
    Inbox
    My issues
    Views
    Roadmaps
    Projects
  

  
    Active issues
    New issue
    Display options
    Filter

    In Progress
    CORE-61 Operations console
    CORE-62 GitHub Actions CI pipeline

    Todo
    CORE-47 KYB Step 2
    CORE-63 Fix Biome config drift
  

Payments dashboard

  
  

  
    Home
    Payments
    Customers
    Products
    Reports
  

  
    Payments
    Create payment
    Export
    Search payments
    Status
    Date

    ₹12,500 succeeded nadeem@example.com Jun 6
    ₹8,999 failed client@example.com Jun 6
    ₹2,400 refunded test@example.com Jun 5
  

---

Tool surface

Agent Web Interface exposes MCP tools across the main phases of browser use.

Session

• list_pages
• close_page

Navigation

• navigate
• go_back
• go_forward
• reload

Observation

• snapshot
• find
• get_element
• screenshot

Interaction

• click
• type
• press
• select
• hover
• scroll_to
• scroll
• drag
• wheel

Forms

• get_form
• get_field

Canvas

• inspect_canvas

Content

• read_page

Network

• list_network_calls
• search_network_calls

---

Quickstart

Run the interactive installer — it auto-detects which AI tools you have installed and registers the MCP server and agent skill in one step:

npx agent-web-interface install

Then ask your AI to use the browser:

Open https://example.com and summarize the main actions available to a user.

Target a specific harness

# Claude Code (also installs the agent skill)
npx agent-web-interface install --harness claude-code

# Cursor
npx agent-web-interface install --harness cursor

# VS Code
npx agent-web-interface install --harness vscode

# Claude Desktop (MCP only — no skill placement)
npx agent-web-interface install --harness claude-desktop

# Multiple at once
npx agent-web-interface install --harness cursor,vscode

# All detected harnesses
npx agent-web-interface install --harness all

Install flags

| Flag | Description | | -------------------------- | -------------------------------------------------------------------------------------------------- | | --harness | Target harness(es): claude-code, cursor, vscode, claude-desktop, all, or comma-separated | | --scope project\|user | Where to write the config (default: project) | | --global | Alias for --scope global | | --project | Alias for --scope project | | --browser-mode | Browser mode: auto (default), user, persistent, isolated | | --headless | Launch Chrome in headless mode | | --cdp-url | Connect to an existing Chrome DevTools Protocol endpoint | | --pin | Register an exact version instead of @latest | | --dry-run | Preview changes without writing files | | --yes | Skip interactive prompts (non-TTY mode) |

Check installation status

npx agent-web-interface doctor

Prints a per-harness status table showing whether the MCP server is registered and whether the agent skill is installed.

Skill-only installation (advanced)

The [agent-web-interface](skills/agent-web-interface/SKILL.md) skill can also be installed independently — without the MCP server — using npx skills:

npx skills add lespaceman/agent-web-interface

This copies only the skill into your agent's skills/ directory. You still need to register the MCP server separately (the install command above does both). npx skills is intentionally kept as a supported alternative for skill-only workflows — see [ADR-0003](docs/adr/0003-keep-npx-skills-alongside-installer.md).

---

Manual setup (Claude Desktop / Cursor / VS Code)

If you prefer to edit config files manually, add the server under the appropriate key:

{
  "mcpServers": {
    "agent-web-interface": {
      "command": "npx",
      "args": ["agent-web-interface@latest"]
    }
  }
}

VS Code uses servers instead of mcpServers and requires "type": "stdio":

{
  "servers": {
    "agent-web-interface": {
      "type": "stdio",
      "command": "npx",
      "args": ["agent-web-interface@latest"]
    }
  }
}

To force connection to your existing Chrome session:

{
  "mcpServers": {
    "agent-web-interface": {
      "command": "npx",
      "args": ["agent-web-interface@latest"],
      "env": {
        "AWI_BROWSER_MODE": "user"
      }
    }
  }
}

---

Browser modes

Browser initialization happens automatically on the first browser tool call.

Set AWI_BROWSER_MODE to control how Chrome is started.

| Mode | Behavior | Profile | | ------------ | ---------------------------------------------------- | --------------------------------------------- | | unset | Auto: try user, then persistent, then isolated | Depends on fallback

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: drisplabs
• Source: drisplabs/browser-mcp
• License: MIT
• Homepage: https://agent-web-interface.com

Install and usage instructions live in the source repository linked above.