Browser Automation

Browser Automation

Control a dedicated Chromium browser via playwright-cli from the terminal. Token-efficient: commands return concise output, not verbose accessibility trees.

App guides (LOAD BEFORE interacting with a specific app)

Before automating a specific web app, read the corresponding guide in apps/. These guides contain validated selectors, event sequences, and gotchas that save you from trial-and-error.

| App | Guide | When to load | |---|---|---| | Gmail | apps/gmail.md | Before any Gmail operation (compose, reply, read inbox, search, delete) | | LinkedIn | apps/linkedin.md | Before any LinkedIn operation (messaging, connections, jobs, Easy Apply, notifications) | | Microsoft Teams | apps/teams.md | Before any Teams operation (send/delete messages via chatsvc API, token extraction) | | Jira | apps/jira.md | Before any Jira operation (create issue, add comment, transition status) | | Teamtailor | apps/teamtailor.md | Before applying to jobs on Teamtailor-based career sites | | Humand.co | apps/humand.md | Before applying to jobs on Humand.co-based career sites |

How to load: read the file with your read tool. Example: read .agents/skills/browser-automation/apps/gmail.md

Check for existing scripts first: the consuming repo may already have scripts that wrap common operations (e.g. scripts/linkedin-inbox.js, scripts/send-email.js). Run ls scripts/ to see what's available. Prefer existing scripts over manual UI automation — they're faster, more reliable, and handle edge cases. The app guides list common scripts to look for.

NEVER edit scripts to hardcode personal data. Scripts should auto-detect values at runtime or accept them as arguments. If a script has a placeholder like ``, it's a bug — fix the script to auto-detect, don't replace the placeholder with a real value. Hardcoding personal data in tracked files violates repo portability.

If the app you need is not listed: use the generic patterns in this file. Consider creating a new apps/.md guide after validating your approach.

Setup

Install

npm install -g @playwright/cli@latest
playwright-cli install-browser        # downloads chromium

Profile directory (NEVER commit)

Use a persistent profile to preserve cookies/logins across sessions. Always gitignore it.

# Create a profile dir at repo root (or anywhere outside git tracking)
mkdir -p .browser-profile

# Add to .gitignore IMMEDIATELY
echo ".browser-profile/" >> .gitignore
echo ".playwright-cli/" >> .gitignore
echo "*.session-state.json" >> .gitignore
echo ".browser-config.json" >> .gitignore

# Open with persistent profile
playwright-cli open "https://example.com" --persistent --profile ./.browser-profile

Rules:

• Profile dir contains cookies, localStorage, session data. NEVER commit it.
• If you clone a repo, create the profile dir fresh. Never share profiles.
• Use --headed for manual login (captchas, 2FA). Default is headless.

The safe wrapper (recommended)

The wrapper script (scripts/browser.js in this skill) guarantees the profile is always used, prevents race conditions, manages parallel sessions, and reads browser mode from config. Always use the wrapper for open/goto/close. Never call playwright-cli open directly.

Copy the wrapper to your repo:

# After installing this skill, copy the wrapper to your repo's scripts/ dir
cp .agents/skills/browser-automation/scripts/browser.js scripts/browser.js

Create a config file (optional, defaults to headless):

# .browser-config.json in your repo root
echo '{"browser_mode": "headed_logins_only"}' > .browser-config.json

Config resolution for browser_mode (headed vs headless):

1. --headed / --headless flag passed to open
2. .browser-config.json: { "browser_mode": "headed" }
3. BROWSER_MODE environment variable
4. Default: headless

Mode values:

• headless — always headless
• headed — always headed (visible browser)
• headed_logins_only — headless by default, caller passes --headed for manual logins

# Core commands (always use the wrapper for these)
node scripts/browser.js open  [--headed|--headless] [--session ]
node scripts/browser.js goto  [--tab ] [--session ]
node scripts/browser.js close [--session ] [--force]
node scripts/browser.js close-all [--force]
node scripts/browser.js ensure [--session ]

# Tab management (wrapper commands, NOT exec)
node scripts/browser.js tab-list
node scripts/browser.js tab-new  --name 
node scripts/browser.js tab-select 
node scripts/browser.js tab-close 

# Passthrough to playwright-cli (for click, fill, snapshot, eval, etc.)
# "exec" forwards the REST of the args to playwright-cli.
# DO NOT write "playwright-cli" again. DO NOT wrap the command in quotes.
# Just write the subcommand name and its args directly after "exec".
node scripts/browser.js exec snapshot
node scripts/browser.js exec click 
node scripts/browser.js exec fill  "text"
node scripts/browser.js exec eval "js expression"
node scripts/browser.js exec find "text to search"
node scripts/browser.js exec press Enter
node scripts/browser.js exec screenshot --filename=page.png

# WRONG — these all fail:
#   node scripts/browser.js exec "playwright-cli tab-list"    (don't write playwright-cli)
#   node scripts/browser.js exec "playwright-cli snapshot"    (don't write playwright-cli)
#   node scripts/browser.js exec "click '[data-tooltip=Redactar]'"  (don't quote the command)
#   node scripts/browser.js exec "snapshot"                   (don't quote the command)
# CORRECT:
#   node scripts/browser.js exec snapshot
#   node scripts/browser.js exec click 
#   node scripts/browser.js exec tab-list                     (playwright-cli tab-list via exec)

# Sessions for parallel subagents
node scripts/browser.js attach --session 
node scripts/browser.js detach --session 
node scripts/browser.js who                         # list active attached agents

# Tab management
node scripts/browser.js tab-new  --name  [--session ]
node scripts/browser.js tab-select  [--session ]
node scripts/browser.js tab-close  [--session ]
node scripts/browser.js tab-close-all [--session ]
node scripts/browser.js tab-list [--session ] [--json]

# Auth state persistence
node scripts/browser.js save-state [--filename ] [--session ]
node scripts/browser.js load-state [--filename ] [--session ]

# Debugging
node scripts/browser.js dashboard
node scripts/browser.js trace-start [--session ]
node scripts/browser.js trace-stop [--session ]
node scripts/browser.js video-start [--filename ] [--session ]
node scripts/browser.js video-stop [--session ]
node scripts/browser.js console [level] [--session ]
node scripts/browser.js requests [--session ]
node scripts/browser.js request  [--session ]

# Info
node scripts/browser.js list
node scripts/browser.js status

Key wrapper behaviors:

• --profile=.browser-profile is hardcoded. Cannot be omitted.
• If a session is already running, open auto-navigates instead of failing.
• Lockfile prevents race conditions when multiple processes open the browser.
• Health check detects zombie sessions before reuse.
• Ref-count prevents one agent from killing the browser while others work.
• For all playwright-cli commands (click, fill, snapshot, eval) use exec or call playwright-cli directly AFTER opening via the wrapper.

Headed vs headless

# Headless (default) — for automation
node scripts/browser.js open "https://example.com"

# Headed — for manual login, captcha solving, visual debugging
node scripts/browser.js open "https://example.com" --headed

When a session expires or a captcha appears: open headed, let the user log in manually, save state, then continue headless. See [references/profile-management.md](references/profile-management.md).

Golden rules (validated empirically)

These rules were validated through extensive testing. Breaking them causes failure.

Rule 1: eval > ref-based clicks

Refs ([ref=e123]) are per-snapshot and do not persist between separate playwright-cli CLI calls. A ref from one snapshot call is invalid by the next click call.

Wrong:

snap = snapshot()
ref = findRef(snap, "Message")
clickRef(ref)  # may fail, ref may be stale

Right:

# Use eval to find and click by text in one atomic call
node scripts/browser.js exec eval "(function(){
  const els = document.querySelectorAll('a, button, [role=\"link\"]');
  for (const el of els) {
    if (el.textContent.includes('Message')) { el.click(); return 'clicked'; }
  }
  return 'not_found';
})()"

Rule 2: In-page polling > shell sleep

Shell sleep between browser commands kills the playwright-cli daemon session. The session dies within 5-10 seconds of inactivity.

Wrong:

node scripts/browser.js goto "https://example.com"
sleep 4                        # session may die here
node scripts/browser.js exec snapshot        # fails: "No active session"

Right:

# Use eval with in-page polling (keeps connection alive)
node scripts/browser.js goto "https://example.com"
node scripts/browser.js exec eval "(async function(){
  for (let i = 0; i  setTimeout(r, 200));
  }
  return 'timeout';
})()"

Exception: Very short sleeps (1-2s) within a single shell command using && chaining are safe. The session stays alive as long as the shell process is running.

Rule 3: Read snapshot file as fallback

When exec snapshot fails (session briefly busy), the open/goto commands auto-generate a snapshot YAML file in .playwright-cli/. Read it directly.

# Try exec snapshot first
node scripts/browser.js exec snapshot
# If it fails, read the latest snapshot file
ls -t .playwright-cli/page-*.yml | head -1 | xargs cat

Rule 4: Use URLs directly, not clicks for navigation

Navigating to a specific URL is more reliable than clicking navigation links.

Wrong: Click "Messaging" icon in header Right: node scripts/browser.js goto "https://www.linkedin.com/messaging/"

Wrong: Click "Saved Jobs" menu item Right: node scripts/browser.js goto "https://www.linkedin.com/jobs-tracker/?stage=saved"

Rule 5: Verify with DOM content, not URL

SPAs (LinkedIn, Gmail, React apps) update the right panel without changing the URL.

Wrong: Check if URL changed after clicking a conversation Right: Check if the target container exists and matches expected content

node scripts/browser.js exec eval "(function(){
  const panel = document.querySelector('.msg-s-message-list-container');
  const header = document.querySelector('h2');
  if (panel && header && header.textContent.includes('Person Name')) return 'ok';
  return 'not_loaded';
})()"

Rule 6: Batch operations into a single eval call

Doing wait + click + verify in one eval call is more robust than multiple separate CLI calls. Each separate call risks session death between steps and adds latency.

Wrong:

node scripts/browser.js exec eval "document.querySelector('#btn')"
node scripts/browser.js exec eval "document.querySelector('#btn').click()"
node scripts/browser.js exec eval "document.querySelector('#result')"

Right:

node scripts/browser.js exec eval "(function(){
  const btn = document.querySelector('#btn');
  if (!btn) return 'not_found';
  btn.click();
  const result = document.querySelector('#result');
  return result ? result.textContent : 'no_result';
})()"

Chaining: open + eval in a single shell command

Chain open && eval in a single shell command to prevent session death between calls.

Wrong:

node scripts/browser.js open "https://mail.google.com"
# session may die here
node scripts/browser.js exec eval "document.title"

Right:

node scripts/browser.js open "https://mail.google.com" && \
  node scripts/browser.js exec eval "document.title"

Core commands

Open / navigate / close

playwright-cli open [url]              # open browser (headless by default)
playwright-cli open [url] --headed     # open visible browser
node scripts/browser.js goto               # navigate current tab
playwright-cli go-back                 # browser back button
playwright-cli go-forward              # browser forward button
playwright-cli reload                  # reload page
playwright-cli close                   # close browser

Snapshot (the most important command)

node scripts/browser.js exec snapshot                # capture page structure with element refs
node scripts/browser.js exec snapshot           # snapshot a specific element (smaller output)
node scripts/browser.js exec snapshot "#main"        # snapshot a CSS selector

Returns a tree of the page with [ref=eXXX] identifiers. Use these refs for click/fill/select. Always take a fresh snapshot or find before interacting — refs change after any page mutation.

Warning: Full snapshots of complex SPAs (Gmail, LinkedIn, Facebook) are HUGE and get truncated in the output. Prefer find or snapshot a specific element instead.

Find (search for elements — PREFERRED over full snapshot)

node scripts/browser.js exec find "text"             # search for text, returns matching elements with refs
node scripts/browser.js exec find "regex"            # search with regex
node scripts/browser.js exec find --regex "/sign (in|up)/i"  # with flags

Returns matching nodes with refs. Much smaller output than a full snapshot. Use this as your primary way to locate elements on complex pages.

Interact

node scripts/browser.js exec click              # click an element
node scripts/browser.js exec click  right       # right-click
node scripts/browser.js exec dblclick           # double-click
node scripts/browser.js exec fill  "text"       # fill input/textarea (replaces content)
node scripts/browser.js exec fill  "text" --submit  # fill + press Enter
node scripts/browser.js exec type "text"             # type into focused element (appends)
node scripts/browser.js exec select  "value"    # select dropdown option
node scripts/browser.js exec check              # check checkbox/radio
node scripts/browser.js exec uncheck            # uncheck checkbox
node scripts/browser.js exec hover              # hover over element
node scripts/browser.js exec press Enter             # press keyboard key
node scripts/browser.js exec upload            # upload file to file chooser
node scripts/browser.js exec drag    # drag and drop

Eval (run JavaScript in page context)

# Simple expression
node scripts/browser.js exec eval "() => document.title"

# Access DOM
node scripts/browser.js exec eval "() => document.querySelector('h1')?.textContent"

# Run inline IIFE for complex logic
node scripts/browser.js exec eval "(function(){ return JSON.stringify({url: location.href, title: document.title}) })()"

# Eval on a specific element (ref becomes 'element' inside)
node scripts/browser.js exec eval "() => element.textContent" 

# Async eval (fetch with cookies)
node scripts/browser.js exec eval "(async () => { const r = await fetch('/api/data'); return JSON.stringify(await r.json()) })()"

Eval runs in the page context with all cookies. Use it for:

• Extracting data not visible in snapshots
• Calling site APIs (fetch with cookies)
• Checking page state (URL, title, DOM)
• Clicking by text (more reliable than ref-based clicks, see Rule 1)
• Waiting for elements (in-page polling, see Rule 2)

Screenshot / PDF

node scripts/browser.js exec screenshot              # full page screenshot
node scripts/browser.js exec screenshot         # screenshot specific element
node scripts/browser.js exec screenshot --filename=page.png
node scripts/browser.js exec pdf --filename=page.pdf # save page as PDF

Tab management (parallelization)

There are TWO tab systems. Don't mix them:

1. Wrapper named tabs (RECOMMENDED — use --tab fl

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: galiprandi
• Source: galiprandi/job-seeker
• License: MIT
• Homepage: https://galiprandi.github.io/job-seeker/

Install and usage instructions live in the source repository linked above.