Browser Automation

Browser Automation Skill

Playwright CSS Selector Reference

Basic Selectors

| Selector | Description | Example | |----------|-------------|---------| | #id | By ID | #checkout-btn | | .class | By class | .add-to-cart | | tag | By element | button, input | | [attr=val] | By attribute | [data-testid="submit"] | | tag.class | Combined | button.primary | | parent child | Descendant | div.container button | | parent > child | Direct child | ul > li | | :nth-child(n) | Nth element | li:nth-child(2) | | :first-child | First element | ul > li:first-child | | :last-child | Last element | ul > li:last-child | | [attr*=val] | Attribute contains | [class*="price"] | | [attr^=val] | Attribute starts with | [href^="https"] | | [attr$=val] | Attribute ends with | [href$=".pdf"] | | :not(sel) | Negation | button:not(.disabled) | | sel1, sel2 | Multiple selectors | #submit, button[type="submit"] |

Form Selectors

| Selector | Use Case | |----------|----------| | input[type="email"] | Email fields | | input[type="password"] | Password fields | | input[type="search"] | Search boxes | | input[name="q"] | Google/search query | | textarea | Multi-line text areas | | select[name="country"] | Dropdown menus | | input[type="checkbox"] | Checkboxes | | input[type="radio"] | Radio buttons | | button[type="submit"] | Submit buttons | | input[type="file"] | File upload fields | | input[type="date"] | Date pickers | | input[type="tel"] | Phone number fields | | input[autocomplete="cc-number"] | Credit card fields | | [contenteditable="true"] | Rich text editors |

Navigation Selectors

| Selector | Use Case | |----------|----------| | a[href*="cart"] | Cart links | | a[href*="checkout"] | Checkout links | | a[href*="login"] | Login links | | nav a | Navigation menu links | | .breadcrumb a | Breadcrumb links | | [role="navigation"] a | ARIA nav links | | a[href*="account"] | Account/profile links | | a[href*="register"], a[href*="signup"] | Registration links | | header a[href="/"] | Logo/home link | | footer a | Footer links |

E-commerce Selectors

| Selector | Use Case | |----------|----------| | .product-price, [data-price] | Product prices | | .add-to-cart, #add-to-cart | Add to cart buttons | | .cart-total, .order-total | Cart total | | .quantity, input[name="quantity"] | Quantity selectors | | .checkout-btn, #checkout | Checkout buttons | | [data-product-id] | Product identifiers | | .product-title, h1.product-name | Product names | | .product-image img, [data-zoom-image] | Product images | | .star-rating, [data-rating] | Review ratings | | .in-stock, .availability | Stock status | | select[name="size"], .size-selector | Size selectors | | [data-variant], .color-swatch | Variant selectors |

---

Generic Selector Strategies (Priority Order)

Use selectors that are resilient to UI redesigns. Prefer semantic and accessibility-based selectors over class names.

Tier 1 — Test Attributes (most stable)

| Selector | Description | |----------|-------------| | [data-testid="value"] | Explicit test ID — survives refactors | | [data-test="value"] | Alternative test attribute convention | | [data-cy="value"] | Cypress test attribute | | [data-qa="value"] | QA-specific test attribute |

Tier 2 — Accessibility Attributes

| Selector | Description | |----------|-------------| | [aria-label="Search"] | Accessible name, framework-agnostic | | [aria-labelledby="id"] | References a labelling element | | [role="button"] | ARIA role — semantic intent | | [role="link"] | ARIA link role | | [role="textbox"] | ARIA textbox role | | [role="dialog"] | Modals and popups | | [role="navigation"] | Navigation landmarks | | [role="search"] | Search landmarks | | [aria-expanded="true"] | Open dropdowns/menus | | [aria-selected="true"] | Selected tabs/options | | [aria-checked="true"] | Checked checkboxes/radios | | [aria-disabled="true"] | Disabled elements (do not click) |

Tier 3 — Semantic HTML

| Selector | Description | |----------|-------------| | button[type="submit"] | Form submit buttons | | input[name="fieldname"] | Form fields by name | | input[type="email"] | Email input by type | | label[for="fieldid"] | Label linked to input | | nav a | Navigation links | | main, article, section | Content landmarks | | header, footer | Page structure | | h1, h2, h3 | Headings for orientation |

Tier 4 — ID and Visible Text

| Strategy | When to use | |----------|-------------| | #unique-id | When ID is human-readable and stable | | Visible text content | When no good attribute selectors exist | | a:has-text("Sign In") | Playwright-specific text matching |

Tier 5 — Class Selectors (least stable)

| Risk | Pattern | |------|---------| | Low risk | .btn-primary, .nav-link (design-system classes) | | Medium risk | .header-search-input (component-specific) | | High risk | .css-1a2b3c, .sc-fAbCdE (auto-generated by CSS-in-JS) |

Rule: Never rely on auto-generated class names (random strings like .css-xyz123). These change on every build.

Accessibility-Based Interaction Patterns

Modern web apps expose accessibility attributes that are more stable than CSS classes.

Finding Interactive Elements by Role

Buttons:     [role="button"], button
Links:       [role="link"], a[href]
Text inputs: [role="textbox"], input[type="text"], textarea
Checkboxes:  [role="checkbox"], input[type="checkbox"]
Radio:       [role="radio"], input[type="radio"]
Comboboxes:  [role="combobox"] (autocomplete/typeahead fields)
Tabs:        [role="tab"] (tab navigation)
Menus:       [role="menu"], [role="menuitem"]
Dialogs:     [role="dialog"], [role="alertdialog"]

Reading Page Structure via Landmarks

[role="banner"]       → site header (logo, global nav)
[role="navigation"]   → navigation sections
[role="main"]         → primary page content
[role="search"]       → search functionality
[role="contentinfo"]  → footer (copyright, legal links)
[role="complementary"] → sidebar content
[role="form"]         → form regions

Label-Based Field Identification

Instead of guessing input selectors, find labels first:
1. browser_read_page → look for label text (e.g., "Email Address")
2. Use: label:has-text("Email") + input  (sibling)
   Or:  input[aria-label="Email Address"]
   Or:  #

SPA Framework Detection & Handling

Detecting the Framework

| Signal | Framework | Notes | |--------|-----------|-------| | ` or | React / Next.js | Content rendered client-side | | with data-v- attributes | Vue.js / Nuxt | data-v-xxxxx are scoped style markers | | or custom element tags | Angular | Uses web component-like tags | | or compiled class names | Svelte / SvelteKit | Minimal runtime footprint | | URL contains #/ hash routing | Any SPA | Client-side routing via hash | | __NEXTDATA_ script tag | Next.js | Server-side rendering with hydration | | __NUXT__ or __NUXTDATA_` in page | Nuxt.js | Vue SSR framework |

Framework-Specific Interaction Tips

React apps:

• State updates are batched — wait 500ms-2s after interactions for re-renders
• Look for data-testid attributes (common in React Testing Library projects)
• Portal-rendered content (modals, tooltips) may be at the end of ``, not nested in the component tree
• React-Select dropdowns: click the container, then look for [class*="option"] in the menu that appears

Vue apps:

• v-if elements may not exist in DOM until conditions are met — re-read page after state changes
• Vue transitions: wait for CSS transitions to complete before interacting
• Vuetify/Element UI components have predictable class prefixes (.v-btn, .el-input)

Angular apps:

• Elements often have _ngcontent- or _nghost- attributes (do not use these as selectors — they change per build)
• Angular Material components: use [role] and [aria-label] attributes instead of classes
• Forms may use reactive validation — errors appear only after interaction (blur event)

General SPA rules:

• After clicking a navigation element, wait 1-3 seconds before reading the page
• If content is missing, check for loading indicators: .loading, .spinner, [aria-busy="true"], .skeleton
• Retry browser_read_page up to 3 times with 2-second intervals before giving up
• URL changes without full page reload confirm SPA routing — do not expect browser_navigate events

Site-Specific Selector Patterns

These are reference selectors for common sites. They change frequently — always verify with browser_read_page if a selector fails, then construct a fresh selector from the live DOM.

Google Search

| Element | Selector | |---------|----------| | Search input | input[name="q"], textarea[name="q"] | | Search button | input[name="btnK"], button[type="submit"] | | Result titles | h3 (within #search) | | Result links | #search a[href^="http"] | | "Next" pagination | a#pnnext |

Amazon

| Element | Selector | |---------|----------| | Search input | #twotabsearchtextbox | | Search button | #nav-search-submit-button | | Add to cart | #add-to-cart-button | | Quantity dropdown | #quantity | | Cart count | #nav-cart-count |

GitHub

| Element | Selector | |---------|----------| | Search | input[name="q"] | | Repository name | [itemprop="name"] a | | Star button | button[aria-label*="Star"] | | Submit button | button[type="submit"] |

Note: When a saved selector fails, use browser_read_page to discover the current DOM, then build a new selector from live content. Prefer [data-testid], [aria-label], or visible text over fragile class-based selectors.

---

Common Workflows

Product Search & Purchase

1. browser_navigate → store homepage
2. browser_type → search box with product name
3. browser_click → search button or press Enter
4. browser_read_page → scan results
5. browser_click → desired product
6. browser_read_page → verify product details & price
7. browser_click → "Add to Cart"
8. browser_navigate → cart page
9. browser_read_page → verify cart contents & total
10. STOP → Report to user, wait for approval
11. browser_click → "Proceed to Checkout" (only after approval)

Account Login

1. browser_navigate → login page
2. browser_read_page → identify form fields and any CAPTCHA
3. browser_type → email/username field
4. browser_type → password field
5. browser_click → login/submit button
6. browser_read_page → verify successful login (check for dashboard/profile elements)
7. If MFA required → inform user, wait for code input

Form Submission

1. browser_navigate → form page
2. browser_read_page → understand form structure
3. browser_type → fill each field sequentially
4. browser_click → checkboxes/radio buttons as needed
5. browser_screenshot → visual verification before submit
6. browser_click → submit button
7. browser_read_page → verify confirmation

Price Comparison

1. For each store:
   a. browser_navigate → store URL
   b. browser_type → search query
   c. browser_read_page → extract prices
   d. memory_store → save price data
2. memory_recall → compare all prices
3. Report findings to user

Multi-Page Data Extraction

1. browser_navigate → starting page
2. browser_read_page → extract data from current page
3. memory_store → save extracted data
4. Check for pagination:
   a. browser_click → "Next" button or page number link
   b. browser_read_page → verify new page loaded (check for changed content)
   c. Repeat from step 2
5. If no more pages → compile and report results

Account Registration

1. browser_navigate → registration page
2. browser_read_page → identify required fields
3. browser_type → fill name, email, password fields sequentially
4. browser_click → accept terms checkbox (if required)
5. browser_screenshot → verify all fields before submission
6. browser_click → submit/register button
7. browser_read_page → check for:
   - Success page → registration complete
   - Email verification prompt → inform user
   - Validation errors → read errors, correct fields, retry

File Download Monitoring

1. browser_navigate → page with download link
2. browser_read_page → identify download button/link
3. browser_click → download trigger
4. browser_read_page → check for download confirmation or redirect
5. If download requires additional steps (accept terms, choose format):
   a. browser_click → required selections
   b. browser_click → final download button
6. Report download status to user

---

Wait Strategies & Timing

When to Wait

Proper waiting prevents most automation failures. Never use fixed sleep times when a condition-based wait is possible.

| Scenario | Strategy | Notes | |----------|----------|-------| | Page navigation | Wait for load event | browser_navigate handles this automatically | | After clicking link | Read page to confirm new content | Check for expected elements on destination | | AJAX/dynamic content | Re-read page after delay | Some SPAs load content asynchronously | | Form submission | Read page for confirmation | Check for success message or redirect | | Slow networks | Retry with backoff | 3s, 6s, 12s intervals | | Animation/transition | Brief pause before interaction | Modal fade-in, dropdown expansion |

Detecting Page Load Completion

After browser_navigate or browser_click that triggers navigation:
1. browser_read_page → check if expected content is present
2. If content missing → wait 2-3 seconds → browser_read_page again
3. If still missing after 3 retries → page may have changed structure
4. Use browser_screenshot to visually confirm page state

SPA (Single Page Application) Handling

SPAs (React, Angular, Vue, Svelte) do not trigger traditional page loads. Client-side routing means the browser URL changes but no network navigation occurs.

1. browser_click → triggers route change (URL updates but no page reload)
2. browser_read_page → may return stale content from previous view
3. Check for loading indicators in the output:
   - Text: "Loading...", "Please wait", skeleton placeholders
   - Attributes: [aria-busy="true"]
   - Classes: .loading, .spinner, .skeleton, .placeholder
4. If loading detected OR content stale → wait 2 seconds
5. browser_read_page → retry (attempt 2 of 3)
6. If still stale → wait 3 seconds → browser_read_page (attempt 3 of 3)
7. If content never updates:
   a. browser_screenshot → check if content is visually present but not captured as text
   b. The content may be inside an iframe or shadow DOM — try alternative access
   c. Report the issue to the user with the screenshot

Iframe Content Access

When target content is inside an iframe:
1. browser_read_page → look for  elements and their src attributes
2. browser_navigate → directly to the iframe src URL (if same-origin)
3. Interact with the content normally
4. browser_navigate → back to the parent page when done
Note: Cross-origin iframes may block direct access. Inform the user if this occurs.

Shadow DOM Awareness

Web components using shadow DOM hide their internals from normal CSS selectors:
1. If a known element is not found by any selector, suspect shadow DOM
2. browser_screenshot → visually confirm the element exists on the page
3. Try interacting via visible text content (may pierce shadow boundaries)
4. If interaction fails, inform the user that the element is inside a shadow root

---

Error Recovery Strategies

Error Recovery Decision Tree

When any interaction fails, walk through this decision tree top-to-bottom:

INTERACTION FAILED
│
├─ Is this the correct page?
│  ├─ NO → browser_read_page to check URL
│  │       ├─ Redirected to login? → re-authenticate, then retry
│  │       ├─ Redirected to error page? → handle HTTP erro

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [librefang](https://github.com/librefang)
- **Source:** [librefang/librefang-registry](https://github.com/librefang/librefang-registry)
- **License:** MIT

Install and usage instructions live in the source repository linked above.