Browser Control

Usejarvis controls a real browser instance using the Chrome DevTools Protocol (CDP). It auto-detects Chrome, Brave, Edge, or Chromium on your system, launches it with a dedicated profile, attaches, and exposes ten browser tools to the agent.

How It Works

The browser launches lazily the first time a browser tool is used, not at daemon startup:

1. If a browser is already running with a reachable CDP endpoint, Usejarvis attaches to it
2. Otherwise it scans for a supported browser binary and launches it with remote debugging
3. Pages are snapshotted as a DOM scan of interactive elements, and the agent interacts using numbered element IDs from that snapshot rather than fragile CSS selectors

Browser tools also route to sidecars: every browser tool accepts a target, and when omitted the call auto-routes to a connected browser-capable sidecar before falling back to the daemon’s local browser. On hosted or Docker brains, the sidecar path is the default, since the container has no display.

Navigation is headed by default; pass headless: true to run hidden. Switching modes relaunches the browser.

Supported Browsers

On Linux, Usejarvis probes absolute paths in order: Google Chrome (/usr/bin/google-chrome, google-chrome-stable, chrome), Brave, Microsoft Edge, Chromium (/usr/bin/chromium, chromium-browser, /snap/bin/chromium). macOS and Windows check the standard install locations for the same four browsers, and WSL can fall back to the Windows Chrome executable (still dialing CDP on 127.0.0.1).

There is no environment variable override for the binary path; the port and profile are internal defaults.

To disable local browser control entirely (for example on a headless server where only sidecars should browse), set:

browser:
  local: false

Launch Flags

The browser is launched with a dedicated profile at ~/.jarvis/browser/profile and flags including:

--remote-debugging-port --user-data-dir --no-first-run
--no-default-browser-check --disable-sync --disable-background-networking
--disable-component-update --disable-features=Translate,MediaRouter
--disable-session-crashed-bubble --hide-crash-restore-bubble
--password-store=basic --disable-blink-features=AutomationControlled

On Linux, --no-sandbox and --disable-dev-shm-usage are always added. On WSL, --ozone-platform=x11 is used; on non-WSL Linux without a $DISPLAY, the browser starts with --headless=new.

Stealth

A stealth script is injected once per connection (applied to every new document) that patches navigator.webdriver, navigator.plugins, navigator.languages, and permissions.query, removes window.cdc_* artifacts, and stubs window.chrome.runtime.

The Browser Tools

Tool	Description
browser_navigate	Navigate to a URL; returns page text plus interactive elements
browser_snapshot	Re-scan the current page; returns text plus numbered elements
browser_click	Click an element by element_id from the last snapshot; supports button (left/right) and double
browser_type	Type into an element (element_id required). Replaces content by default; append: true keeps it, submit: true presses Enter
browser_hover	Hover an element by element_id
browser_press_key	Send a key or combination to the page
browser_scroll	Scroll the page or an element
browser_upload_file	Set a file on an input via CDP, bypassing the native picker (local browser only)
browser_evaluate	Execute JavaScript in the page context
browser_screenshot	Capture the page as PNG

There is no separate extract tool: browser_navigate and browser_snapshot already return the page text.

Snapshots

Snapshots are a DOM scan (querySelectorAll) over interactive elements: links, buttons, inputs, selects, textareas, ARIA roles, [data-testid], and contenteditable nodes. The scan traverses same-origin iframes (up to depth 3, max 10 frames). Cross-origin iframes and closed shadow roots are not visible to it.

Webapp Templates

When the browser lands on a known domain, a matching per-app playbook (from the built-in library of 100 webapp templates, extendable under ~/.jarvis/webapp-templates/) is injected into the model context, teaching it the app’s roles and flows. Templates are keyed by domain and only load when you actually navigate there.

Background Monitor

A separate background browser instance (CDP port 9223, profile ~/.jarvis/browser/bg-profile) is used for background monitoring tasks so it never fights the agent’s main browser session.

Troubleshooting

Browser fails to launch on a server

• No display available: non-WSL Linux without $DISPLAY auto-falls back to headless, but the better setup on servers is browser.local: false plus a sidecar on a desktop machine

Agent clicks the wrong element

• Element IDs come from the last snapshot. If the page changed, take a fresh browser_snapshot first

Sites detect automation

• The stealth patches cover the common checks, but some sites use additional signals. Use your normal browser for those sites, or a sidecar browser session