Laya Browser Agent

laya-browser-agent

Browser agent decisions powered by Laya — the open-source System 1 model. A local alternative to TypeSafe Jev: no cloud, no API key, no screenshots.

[English](README.md) | [中文](README.zh-CN.md) | [日本語](README.ja.md) | [Español](README.es.md)

[](https://github.com/ChenneyZhuang/laya-browser-agent/actions/workflows/tests.yml/badge.svg) [](LICENSE) [](https://huggingface.co/ichenney/laya-browser-v32b)

A local, open-weight alternative to TypeSafe Jev for the browser-driving use case — running Laya, the open-source "System One" decision model, fully on your own machine. Works with browser-use/jev-ultrafast via the same wire format, and speaks TypeSafe's /v1/systemone dialect, so existing Jev tooling points at it by changing one base URL.

A decision model answers typed questions about a state and returns calibrated probabilities. It never writes text, so it cannot hallucinate an instruction. That makes it exactly the right shape for the deciding half of a browser agent: hand it a numbered table of the controls on a page, and it tells you which operation to run and which element to act on.

This project wires those models into that role, locally, for whatever agent you already use.

> 🚀 Fine-tuned checkpoint included: this project also trains > ichenney/laya-browser-v32b, > which beats the official browser-tuned Laya checkpoint on 6 of 8 benchmarks — > holdout 0.7125 vs 0.425, MiniWoB 0.9138 vs 0.6638, JevBench hard > 0.4144 vs 0.243 — and runs at 27 ms/decision on a 3080 (31× faster than > the hosted Jev API). One line to use: > LayaTorchBackend(model="ichenney/laya-browser-v32b", subfolder="v32b"). > [Full comparison →](#benchmarks)

from localdecide import BrowserDecider
from localdecide.drivers import PlaywrightDriver

with PlaywrightDriver(start_url="https://en.wikipedia.org/wiki/Main_Page") as driver:
    run = BrowserDecider().run(driver, "Click the 'Random article' link in the navigation.")
    print(run.stopped, run.summary()["median_decision_ms"], "ms/decision")
# -> done 142 ms/decision

Measured on an M4 MacBook Air, 16 GB (see [Benchmarks](#benchmarks)):

| | | |---|---| | Decision latency | 10–30 ms steady state, ~150 ms with a 60-element page (upstream p50: 38 ms for 1 question) | | Throughput | up to ~100 decisions/second | | Cost | $0.00 — no API, no metering | | Model size | 322 M params (~644 MB) on disk | | Calibration | upstream reports ECE 0.030 across 13 task families (post-temperature-scaling) | | Page content sent to a server | none |

---

How this relates to Jev and Laya

| | TypeSafe Jev | Laya | localdecide | |---|---|---|---| | Weights | closed, API only | open, Apache-2.0 | runs Laya's open weights | | Where it runs | TypeSafe's cloud | anywhere PyTorch runs | your machine — MLX on Apple Silicon, PyTorch elsewhere | | Wire format | POST /v1/systemone | same contract | speaks it too (POST /v1/systemone) | | Cost | $0.042/M input tokens | free | free | | Browser harness | jev-ultrafast (18.8k★) | — | included: loop, drivers, guards, skills | | Page leaves your machine | yes | no | no |

Verified against the real Jev API

This repo's systemone dialect was validated end-to-end against TypeSafe's production endpoint (api.typesafe.ai/v1/systemone, model jev-1.13.0) on 2026-09-22. A working request looks like this — note that criteria is required for every question type (the API rejects questions without it), and for choice it is a map of option → rubric description, not a string:

{
  "state": "Hi, my pool pump stopped working...",
  "model": "jev-latest",
  "questions": {
    "is_pool_lead": {
      "type": "noul",
      "instructions": "Is this a swimming-pool related service request?",
      "criteria": {
        "true": "Related to pool maintenance, construction, or supplies",
        "false": "Not pool related"
      }
    },
    "urgency": {
      "type": "choice",
      "instructions": "Which urgency level?",
      "criteria": {
        "low": "Routine inquiry",
        "medium": "Wants service soon",
        "high": "Emergency or explicitly time-sensitive"
      }
    }
  }
}

Response: {"model":"jev-1.13.0","answers":{"is_pool_lead":{"noul":0.99}, "urgency":{"choice":"high","confidence":1.0,...}},"usage":{"input_tokens":398,"output_tokens":58}}

The same payload, with url pointed at the bundled localdecide serve (POST /v1/systemone), produces the same answer shape from the local Laya checkpoint — so code written against one works against the other by changing one base URL. score questions take criteria as an array of level names.

Head-to-head vs hosted Jev: measured, not claimed (v10s era, kept for provenance)

examples/diagnostics/jev_head_to_head.py runs the same 12 single-step element-table decisions through both engines — the local Laya v10s checkpoint and TypeSafe's production jev-1.13.0 — on the same fixture pages with the same question contract. examples/diagnostics/jev_flow_h2h.py does the same for six full task flows driven through the real browser loop (history, scoping, guards all active for both engines).

Single-step, zero-context (12 goals, 3 fixtures, 6 languages):

| | local v10s | hosted jev-1.13.0 | |---|---|---| | strict element hits | 4/12 | 8/12 | | cross-lingual goals | 1/6 | 5/6 | | median decision latency | 618 ms | 716 ms | | mean confidence | 0.90 (overconfident) | 0.81 | | engines pick the same element | 2/12 | — |

Multi-step flows (6 flows x 2 engines): neither engine solves the scripted shop flow unaided today. The hard state is the one right after typing a search query: the goal names a product the page does not show yet, and both engines lose the thread there — local v10s clicks Search again at p=0.84 even with the product visible, hosted Jev answers BLOCKED or picks the right element at p=0.45. Local v10s did solve the Chinese navigation flow end-to-end (帮助中心 -> DONE); hosted Jev reached the same element but never emitted DONE.

What this means in practice:

• **If you want accuracy out of the box, especially cross-lingual, hosted Jev

is measurably better.** At ~$0.042/M input tokens a typical decision costs ~$0.000017.

• **If you want privacy, offline, or free at volume, the local checkpoint is

competitive on latency and honest about confidence** (0.90 vs 0.81 mean — calibration work helps here), but it needs the harness loop to hit its trained regime, and its multilingual grounding is the weakest axis.

• The headline "62% task success" for the browser-tuned checkpoint comes from

goals whose wording overlaps the page's own vocabulary. Goals that require the model to bridge a vocabulary gap (type a word the page never shows) are the open problem for both engines. This battery exists so you can re-run the comparison yourself; numbers here are from 2026-09-22, jev-1.13.0.

Two more batteries round out the picture:

Text classification (jev_text_h2h.py — real business texts, no browser):

| family (21 cases) | local v10s | hosted jev-1.13.0 | |---|---|---| | pool-lead triage: relevant noul | 3/7 labelled correct | 7/7 | | pool lead_quality score (0-4) | low-biased (1.0-2.0) | calibrated (2.4-3.7) | | Chinese SMS: transaction / type | 6/8 | 6/8 | | Chinese SMS: phishing detection | 0/3 | 3/3 | | robustness (empty / 5k chars / adversarial) | 3/3 | 3/3 | | median latency | 36 ms | 738 ms |

The phishing row deserves a stare: local v10s scored the classic "妈妈，我手机坏了…快转5000" scam at p=0.14 and the lucky-red-packet scam at p=0.23 — it would wave both through. Hosted Jev put both at p=0.96. For any safety-adjacent routing (fraud, abuse, self-harm), local v10s in its current form is not safe to trust alone.

Browser edge cases (jev_edge_h2h.py — goals where restraint is the right answer, 9 cases): local 4/9, hosted 5/9, and they fail in opposite directions. Local v10s is a fire-and-act model: it clicks "Delete my account" (p=0.93) when asked to delete the entire website, unticks an already-unticked checkbox, and clicks a disabled button — near-certain confidence every time. Hosted Jev blocks the impossible goals but also over-blocks legitimate ones (missed "Delete my account" as a real goal). Neither engine has a trustworthy notion of "this goal cannot be done here" yet; the harness's own guards (disabled-element checks, confirmation gates) are what catch these today.

Practical summary across all four batteries: use hosted Jev when accuracy and safety calibration matter and per-call cost is fine; use local v10s when latency (10-20x faster), privacy, or free-at-volume matters, and let the harness guards compensate for its overconfidence. Fine-tuning data for the weakest axes (Chinese grounding, phishing, restraint) is exactly what the training recipe in this repo's diagnostics produces.

The phishing gap is a data problem, not an architecture ceiling: a September 2026 arXiv study (2609.23959) LoRA-tunes a 4B model to output a single calibrated P(scam) in one forward pass and reaches AUROC .974 with calibration error .052 on scam-call screening — the same readout this repo runs, better data. And Laya's upstream publishes the calibration numbers to aim for: accuracy 0.753 at ECE 0.030 across 13 task families, which the v10s browser checkpoint does not inherit on text outside its browser training distribution (see the pool/SMS rows above).

If you have read about Jev's "System One" model and want the same idea — typed, calibrated decisions instead of generated text — running locally for your browser agents, this is the wiring for it. It uses the browser-tuned Laya checkpoint (cklxx/laya-browser, which itself documents 0% → 62% task success after fine-tuning) and adds the parts neither project ships: element-table observation, answer validation, confidence gating, loop guards, and a TypeSafe-compatible server.

Why this exists

The "System One model" idea — a non-autoregressive model that returns typed, calibrated decisions instead of prose — went from research to production-worthy in

1. TypeSafe's Jev made it famous; Convai Innovations' Laya

shipped the same architecture as Apache-2.0 open weights; and a remarkable amount of work went into making these models drive browsers.

What was missing was the boring part: a neutral, local, agent-agnostic harness. Something you can point Claude Code, Codex, Cursor, Hermes, or your own script at — that loads a local checkpoint, keeps the model's output inside a safe action space, and speaks the dialects agents already talk.

That is this repo.

What a decision model may and may not do here

| The model decides | Your code decides | |---|---| | which operation (CLICK/TYPE_TEXT/SELECT/SCROLL/WAIT/DONE/BLOCKED) | what each operation means | | which element index to act on | what that index maps to in the DOM | | how confident it is | whether the confidence is good enough |

The model's output is validated against the option set it was given before anything acts on it: a key outside the offered set, a probability vector that does not sum to one, or a choice that is not the argmax is rejected and the decision fails open (your agent takes its own fallback path instead of acting on junk). Model output can never become a selector, a coordinate, or executable code — it is only ever an index into a table your code built.

---

Install

Step by step

1. Install the package with the extras for your platform. Install from a git clone (the PyPI release is catching up to the repo; if pip install laya-browser-agent works for you, prefer it):

git clone https://github.com/ChenneyZhuang/laya-browser-agent && cd laya-browser-agent

# Apple Silicon Mac (M1–M4) — MLX runtime, fastest path:
pip install -e '.[all]'

# Linux / Windows / Intel Mac — same checkpoints through PyTorch:
pip install -e '.[torch]'

# Linux + NVIDIA GPU — PyTorch will pick the CUDA wheel if one is present:
pip install -e '.[torch]'

(Once the package is on PyPI, pip install 'laya-browser-agent[mlx]' / [torch] will be the one-liner. localdecide is the import and CLI name on both paths.)

2. Install a browser driver (only needed for the browser loop):

pip install 'laya-browser-agent[playwright]' && playwright install chromium
# Or attach to a Chrome you already have open and logged in — no download:
pip install 'laya-browser-agent[cdp]'

3. Run the hardware check. doctor detects your chip and memory, picks the right runtime, and runs a one-decision smoke test — so a broken install shows up here, not in your agent:

localdecide doctor

Expected output on an M4:

python      3.12.13 (arm64, Darwin)
hardware    Apple M4, 16 GB unified memory
laya-mlx    installed
playwright  installed
backend     laya-mlx
smoke test  OK (85 ms, first call includes model load)

4. From source (for development):

git clone https://github.com/ChenneyZhuang/laya-browser-agent
cd laya-browser-agent
python3.12 -m venv .venv && source .venv/bin/activate
pip install -e '.[all,playwright,cdp]' pytest
python -m pytest tests/test_contract.py -q     # 55 tests, no model needed

On an M4 it prints the chip, memory, runtime, and runs a one-decision smoke test so a broken install shows up here instead of in your agent:

python      3.12.13 (arm64, Darwin)
hardware    Apple M4, 16 GB unified memory
laya-mlx    installed
playwright  installed
backend     laya-mlx
smoke test  OK (85 ms, first call includes model load)

What to expect on different devices

Everything here is measured on real hardware or stated as a limit. The harness is identical everywhere (pure Python, verified by CI on 6 platform/Python combinations); what changes by device is which runtime you install and how big a decision you can afford.

| Device | Runtime | Expected experience | |---|---|---| | Apple Silicon M-series, 16 GB+ (M1–M4) | laya-mlx | The reference experience: 10–30 ms short decisions, ~330 ms scoped browser steps, everything local. This is what the benchmarks above measure. | | Apple Silicon, 8 GB (M1/M2 base) | laya-mlx | Works, but the 1.3 GB v32b checkpoint (or 650 MB browser-legacy) + Chromium is tight. The subprocess design keeps one model OR one browser resident; close heavy apps. Expect swap pressure on big pages. | | Intel Mac | laya (PyTorch) | laya-mlx does not run here. Model works; expect ~2–4× the Apple Silicon latency on CPU. Browser loop fine. (Note: GitHub retired the macos-13 runner image in Dec 2025; Intel macOS CI now runs on macos-15-intel, which GitHub itself plans to retire in 2027 — Intel macOS support has a countdown.) | | Linux server, CPU only | laya (PyTorch) | Good for batch deciding (no browser needed for classification). Browser loops work headless. Latency similar to Intel Mac CPU. | | Linux + NVIDIA GPU | laya (PyTorch, CUDA) | Best PyTorch path — GPU inference cuts latency well below CPU. Also the only place you can fine-tune (the laya-browser recipe needs CUDA). | | Windows | laya (PyTorch) | Works; same expectations as Linux CPU. Playwright supports it natively. | | Below 8 GB total / Raspberry Pi class | — | Not supported. The v32b checkpoint alone is 1.3 GB (v10s legacy: 650 MB) and the decision heads want ~1 GB resident. Use the HTTP backend to reach another machine instead. | | Any device, model elsewhere | `

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: ChenneyZhuang
• Source: ChenneyZhuang/laya-browser-agent
• License: Apache-2.0
• Homepage: https://github.com/ChenneyZhuang/laya-browser-agent#readme

Install and usage instructions live in the source repository linked above.