Page Agent

Alibaba PageAgent — Client-Side GUI Agent Framework

> Auto-activation: This skill activates when page-agent is detected in project dependencies or when implementing natural language web automation.

What is PageAgent?

PageAgent is a purely client-side web automation framework powered by LLMs. It converts natural language commands into DOM operations through an iterative Re-act (Reasoning + Acting) loop. Unlike server-side tools (Selenium, Playwright) or screenshot-based approaches, PageAgent:

• Runs entirely within the webpage as JavaScript code
• Uses text-based DOM analysis (no screenshots, no vision models)
• Requires zero backend infrastructure
• Supports any OpenAI-compatible LLM provider

┌─────────────────────────────────────────────────────────────────┐
│                     PageAgent Architecture                       │
│                                                                  │
│  ┌──────────────┐     ┌──────────────┐     ┌──────────────────┐ │
│  │  User Input  │────▶│ PageAgent    │────▶│  LLM Provider    │ │
│  │  (Natural    │     │ Core         │◀────│  (OpenAI, Qwen,  │ │
│  │   Language)  │     │ (Re-act Loop)│     │   Claude, etc.)  │ │
│  └──────────────┘     └──────┬───────┘     └──────────────────┘ │
│                              │                                   │
│                       ┌──────▼───────┐                          │
│                       │ Page         │                          │
│                       │ Controller   │                          │
│                       │ (DOM ↔ HTML) │                          │
│                       └──────┬───────┘                          │
│                              │                                   │
│                       ┌──────▼───────┐                          │
│                       │  Live DOM    │                          │
│                       │  (Browser)   │                          │
│                       └──────────────┘                          │
└─────────────────────────────────────────────────────────────────┘

Key Concepts — The Re-act Loop

Each agent step follows this cycle:

1. Observe: PageController.getBrowserState() extracts DOM state → simplified indexed HTML
2. Think: System prompt + user task + history + browser state → sent to LLM via MacroTool (forces reflection-before-action structure: evaluation_previous_goal, memory, next_goal, then action)
3. Act: Selected tool executed against the live DOM (click, type, scroll, etc.)
4. Update: Result recorded in history, events emitted, lifecycle hooks called
5. Check: If action is done, return result; otherwise loop back to Observe

The MacroTool pattern wraps all tools into a single "macro" tool that forces the LLM to reflect before acting — outputting evaluation, memory, and next_goal fields before selecting any action. This significantly reduces impulsive errors.

Quick Start

Option 1: NPM Install (Recommended for Production)

npm install page-agent

import { PageAgent } from 'page-agent'

const agent = new PageAgent({
  // LLM Configuration (required)
  model: 'gpt-4.1-mini',           // or 'qwen3.5-plus', 'claude-haiku-4.5', etc.
  baseURL: 'https://api.openai.com/v1',  // OpenAI-compatible endpoint
  apiKey: 'YOUR_API_KEY',

  // Optional Configuration
  language: 'en-US',               // UI language
  maxSteps: 20,                    // Max reasoning steps (default: 20)
  temperature: 0.1,                // LLM temperature
  maxRetries: 3,                   // Retry on LLM failures
})

// Show the built-in chat panel
agent.panel.show()

// Or execute tasks programmatically
const result = await agent.execute('Click the login button and fill username as "admin"')
console.log(result.success, result.data)

Option 2: CDN Script (Quick Testing Only)

Option 3: Chrome Extension (Multi-Tab)

The Chrome extension (@page-agent/ext) enables multi-tab workflows, accessible via window.PAGE_AGENT_EXT. Useful when tasks span across multiple pages. Requires a token for authorization.

Package Architecture (Monorepo)

| Package | NPM Name | Purpose | |---------|-----------|---------| | page-agent | page-agent | Complete package with UI panel (use this for standard integration) | | @page-agent/core | @page-agent/core | Headless agent core — Re-act loop, tools, state management (no UI) | | @page-agent/llms | @page-agent/llms | LLM client — OpenAI-compatible API, retry logic, model patches | | @page-agent/page-controller | @page-agent/page-controller | DOM extraction, simplified HTML generation, action execution | | @page-agent/ui | @page-agent/ui | React-based Panel component for human-in-the-loop interaction | | @page-agent/ext | Private | Chrome extension for multi-tab automation |

Dependency chain: page-agent → @page-agent/core → @page-agent/llms + @page-agent/page-controller

Supported LLM Models

Models must support OpenAI-compatible API and tool calls (function calling).

| Provider | Tested Models | Recommended | |----------|---------------|-------------| | Qwen (Alibaba) | qwen3.5-plus, qwen3.5-flash, qwen-3-max | ⭐ qwen3.5-plus, qwen3.5-flash | | OpenAI | gpt-5.1, gpt-4.1-mini | ⭐ gpt-5.1 | | DeepSeek | deepseek-3.2 | ⭐ deepseek-3.2 | | Google | gemini-3-flash | ⭐ gemini-3-flash | | Anthropic | claude-haiku-4.5 | ⭐ claude-haiku-4.5 | | xAI | Grok (via patches) | — | | Ollama | qwen3:14b (local) | Use with 64K context |

Ollama Local Setup

# Required environment variables
export OLLAMA_CONTEXT_LENGTH=64000
export OLLAMA_HOST=0.0.0.0:11434
export OLLAMA_ORIGINS="*"

ollama pull qwen3:14b

const agent = new PageAgent({
  model: 'qwen3:14b',
  baseURL: 'http://localhost:11434/v1',
  apiKey: 'ollama',  // any non-empty string
})

LLM Configuration

interface LLMConfig {
  model: string       // Model name (e.g., 'gpt-4.1-mini')
  baseURL: string     // OpenAI-compatible API endpoint
  apiKey: string      // API key (⚠️ exposed client-side — use proxy in production)
  temperature?: number // Default: model-dependent
  maxTokens?: number   // Max response tokens
  maxRetries?: number  // Retry count on failure (default: 3)
}

🔐 Production Security: Backend Proxy Pattern

CRITICAL: Never expose API keys in client-side code for production. Use a backend proxy:

const agent = new PageAgent({
  model: 'gpt-4.1-mini',
  baseURL: '/api/llm-proxy',  // Your backend endpoint
  apiKey: '',                 // Empty — backend handles auth
})

Backend proxy options:

1. Reverse proxy through your existing API server
2. Edge functions (Vercel Edge, Netlify Edge)
3. Serverless functions (AWS Lambda, Cloud Functions)

The proxy receives the OpenAI-compatible request, injects the real API key, and forwards to the LLM provider.

Custom Tools — Extending Agent Capabilities

Custom tools are the primary way to make PageAgent do domain-specific work. Tools use Zod schemas for type-safe input validation.

Zod Version

PageAgent uses Zod for tool schemas. Both Zod 3 (≥3.25.0) and Zod 4 are supported. Always import from zod/v4 subpath. Zod Mini is NOT supported.

import { z } from 'zod/v4'  // ✅ Always use this import

Defining Custom Tools

import { z } from 'zod/v4'
import { PageAgent, tool } from 'page-agent'

const agent = new PageAgent({
  // ... LLM config ...
  customTools: {
    // Add a new tool
    add_to_cart: tool({
      description: 'Add a product to the shopping cart by its product ID.',
      inputSchema: z.object({
        productId: z.string().describe('The unique product identifier'),
        quantity: z.number().min(1).default(1).describe('Number of items to add'),
      }),
      execute: async function (input) {
        // `this` is the PageAgent instance — access DOM state, panel, etc.
        await fetch('/api/cart', {
          method: 'POST',
          body: JSON.stringify(input),
        })
        return `✅ Added ${input.quantity}x ${input.productId} to cart.`
      },
    }),
  },
})

Tool Anatomy

interface PageAgentTool {
  name: string              // Auto-set from the key in customTools record
  description: string       // Shown to LLM — be specific about when to use
  inputSchema: ZodSchema    // Zod validation schema (also generates JSON Schema for LLM)
  execute: (this: PageAgent, input: ValidatedInput) => Promise | string
  pageFilter?: {            // (Beta) Restrict tool to specific URL patterns
    include?: string[]      // Glob patterns — tool only available on matching pages
    exclude?: string[]      // Glob patterns — tool hidden on matching pages
  }
}

Overriding & Removing Built-in Tools

const agent = new PageAgent({
  customTools: {
    scroll: null,               // Remove the scroll tool entirely
    execute_javascript: null,   // Remove JS execution for security
    click_element: tool({ ... }), // Override the built-in click with custom implementation
  },
})

Runtime Tool Registration

// Add tools dynamically after initialization
agent.registerTool(tool({
  name: 'submit_feedback',
  description: 'Submit user feedback to the backend',
  inputSchema: z.object({ rating: z.number().min(1).max(5), comment: z.string() }),
  execute: async function (input) {
    await fetch('/api/feedback', { method: 'POST', body: JSON.stringify(input) })
    return '✅ Feedback submitted successfully.'
  },
}))

Page Filters (Beta) — Context-Aware Tools

Restrict tools to specific pages to reduce LLM token usage and improve accuracy:

const agent = new PageAgent({
  customTools: {
    approve_order: tool({
      description: 'Approve a pending order.',
      inputSchema: z.object({ orderId: z.string() }),
      pageFilter: {
        include: ['/admin/orders', '/admin/orders/*'],
        exclude: ['/admin/orders/archived'],
      },
      execute: async function (input) {
        await fetch(`/api/orders/${input.orderId}/approve`, { method: 'POST' })
        return `✅ Order ${input.orderId} approved.`
      },
    }),
  },
})

Best Practices for Custom Tools

1. Description Writing: Start with action verbs ("Add...", "Submit...", "Navigate to..."). Be specific about when to use.
2. Schema Design: Use .describe() on Zod fields — these appear in the LLM's tool schema. Use defaults for common values.
3. Error Handling: Catch exceptions, return descriptive error messages with emoji prefixes (✅, ❌, ⏳). Never expose sensitive data.
4. Return Values: Return strings that help the LLM understand what happened and decide next steps.
5. Performance: Keep tool execution fast. The agent waits for tool results before the next reasoning step.

Custom Instructions — Guiding Agent Behavior

System Instructions (Global)

const agent = new PageAgent({
  instructions: {
    system: `You are a professional e-commerce assistant.
    Guidelines:
    - Always confirm before placing orders
    - Never modify payment information
    - Use polite, professional language`
  },
})

Page-Specific Instructions (Dynamic)

const agent = new PageAgent({
  instructions: {
    system: 'You are an order management assistant.',
    getPageInstructions: (url) => {
      if (url.includes('/checkout')) {
        return `This is the checkout page. Important fields: shipping address, payment method, promo code.
        Never auto-submit — always ask user to confirm before clicking "Place Order".`
      }
      if (url.includes('/products')) {
        return `This is the product listing page. Help users find products by name, category, or price range.`
      }
      return undefined  // No special instructions for other pages
    },
  },
})

Data Masking — Protecting Sensitive Information

Use transformPageContent to sanitize the DOM content before it's sent to the LLM:

const agent = new PageAgent({
  transformPageContent: async (content) => {
    // Mask phone numbers: 13812345678 → 138****5678
    content = content.replace(/\b(1[3-9]\d)(\d{4})(\d{4})\b/g, '$1****$3')
    
    // Mask emails: john.doe@example.com → j***@example.com
    content = content.replace(
      /\b([a-zA-Z0-9._%+-])[^@]*(@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,})\b/g,
      '$1***$2'
    )
    
    // Mask credit card numbers
    content = content.replace(/\b(\d{4})\s?\d{4}\s?\d{4}\s?(\d{4})\b/g, '$1 **** **** $2')
    
    return content
  },
})

PageController Configuration

The PageController handles all DOM interaction. Configure it for optimal element detection:

const agent = new PageAgent({
  // PageController options
  enableMask: true,           // Show blocking overlay during automation (default: true via PageAgent)
  viewportExpansion: -1,      // -1 = index ALL elements (full page), 0 = viewport only, >0 = viewport + N pixels
  highlightOpacity: 0.0,     // Visual highlight on elements (0 = transparent, 1 = opaque)
  highlightLabelOpacity: 0.1,// Label opacity for indexed elements

  // Element filtering
  interactiveBlacklist: [     // Elements AI should NEVER interact with
    document.getElementById('admin-panel'),
    () => document.querySelector('.dangerous-button'),
  ],
  interactiveWhitelist: [],   // Force elements to be interactive even if not detected
  
  // HTML attributes to include in simplified DOM
  includeAttributes: ['aria-label', 'data-testid', 'placeholder', 'title', 'alt'],
})

Key: How PageController Represents the DOM

PageController converts the live DOM into simplified, indexed HTML for the LLM:

[0]Home
[1]Click Me
[2]
Welcome to the page
[3]
  Option A
  Option B

The LLM references elements by numeric index (not CSS selectors), making it:

• Deterministic: Indices are stable within a single DOM snapshot
• LLM-friendly: Simple numeric references in responses
• No selector fragility: No XPath or CSS selector generation

BrowserState Output Format

Each step, the LLM sees a structured prompt including:

Current Page: [Example Site](https://example.com)
Page info: 1920x1080px viewport, 1920x2400px total page size, 0.0 pages above, 1.2 pages below
Interactive elements from top layer of the current page inside the viewport:
[Start of page]
[0]Home
[1]Click Me
[2]
Welcome to the page
... 1320 pixels below (1.2 pages) - scroll to see more ...

Using PageAgentCore (Headless / Custom UI)

For custom UIs or headless automation, use PageAgentCore directly:

import { PageAgentCore } from '@page-agent/core'
import { PageController } from '@page-agent/page-controller'

// Create the controller manually
const pageController = new PageController({
  enableMask: false,
  viewportExpansion: -1,
})

// Create headless agent
const agent = new PageAgentCore(pageController, {
  model: 'gpt-4.1-mini',
  baseURL: 'https://api.openai.com/v1',
  apiKey: 'YOUR_API_KEY',
})

// Listen to events for custom UI
agent.addEventListener('statuschange', (e) => {
  console.log('Agent status:', e.detail.status)  // 'idle' | 'running' | 'paused' | 'done' | 'error'
})

agent.addEventListener('historychange', (e) => {
  // Historical events — persisted and sent to LLM as context
  const history = e.detail.history
  console.log('Latest step:', history[history.length - 1])
})

agent.addEventListener('activity', (e) => {
  // Transient activity events — real-time UI updates
  console.log('Activity:', e.detail.type, e.detail.data)
})

// Execute
const result = await agent.execute('Find and click the Submit button')

React Hook Example for Custom UI

import { useEffect, useState } from 'react'
import { PageAgentCore } from '@page-agent/core'

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [tn5052](https://github.com/tn5052)
- **Source:** [tn5052/page-agent-skills](https://github.com/tn5052/page-agent-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.