camofox reaps a tab after TAB_INACTIVITY_MS (5 min) without API tool calls;
desktop-mode input goes straight to Selkies and screenshots don't count, so
tabs disappeared under the user during takeover.
- BrowserKeepalive (in-process): sessions held by an open desktop WebSocket
(ref-counted), a human takeover lease, or a running/queued agent run that
used browser.* get a keepalive every BROWSER_KEEPALIVE_MS (default 120s,
0 = off). Holds are re-checked each round; release/lease expiry/agent end/
session close stop it and camofox reaps as usual.
- Keepalive call: camofox 1.17 has no keepalive/touch endpoint and /stats does
not bump toolCalls; use GET /tabs/{id}/downloads without includeData/consume
(read-only metadata). Goes through BrowserService.run so restarts or reaped
tabs self-heal via BR-3 reconnect; no lastAutomationAt, no control needed.
- Runbook browser.md §12, deploy env table, compose env; tests against the
testkit mock camofox.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- browser:launching hook generates a Windows fingerprint once (camoufox-js,
os=windows), persists it to /root/.camofox/umt-identity.json and reuses it
on every launch; Firefox/Camoufox upgrades only bump UA/version fields
- Windows fontconfig with absolute font dir (upstream path was broken)
- GET /umt/identity (public UA/OS/version) and PUT /umt/sessions/:id/proxy
(per-context proxy; Firefox no longer uses container HTTP_PROXY env)
- adapter identity()/setEgress(), splitBrowserOnlyCookies for cf_clearance/__cf_bm
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Any session operation that hits 404/410 (or finds no open tab) reconciles
tabs; if none survive it re-injects the account cookies (BR-2), opens a new
tab under the same userId/sessionKey, returns to the recorded URL (site home
otherwise), marks old tabs lost and retries the operation once. Failure
returns 503 browser_reconnect_failed with the reason, with a 10s backoff.
- Restart detection: umt plugin GET /umt/instance (fallback /health fields);
a changed instance id marks every open session needsReconnect.
- POST /browser/sessions/{id}/reconnect (idempotent); session view exposes
needsReconnect and reconnect {status, trigger, at, reason, humanNotice}.
- Controller and lease are kept; reconnecting during takeover asks the human
to double-check the reloaded page.
- Web: screenshot and desktop modes show "reconnecting", auto-reconnect once,
pause polling, offer a reconnect button with the reason on failure; desktop
mode rebuilds the Selkies connection after a reconnect.
- Mock camofox restart(), tests, smoke step and runbook section 10.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Inject the site account Cookie (manual + auto-login session, session wins)
into the camofox profile before the first tab opens; password accounts
without a session run refresh-login first; API Key sites are skipped.
- Cookie expired after injection -> credential.status=expired; cookie
accounts go to waiting/login_required ("log in once, applies site-wide").
- Write back site-domain cookies from the browser on handover, session close,
every 10 min of activity, and from refresh-login when the stored cookie is
invalid (verified before saving). user_info.cookieOrigin records
source=browser; source shown as manual/login/browser in the origin view.
- New camofox plugin umt-cookies (GET/POST /umt/sessions/:userId/cookies
under the access key); profile dir 0700/files 0600 in the desktop image.
- Single entry: sources-identity/service/credentials.ts.
- Real check: HHClub cookie injected into a fresh profile -> home page
logged in (logout link present), 5 cookies read back.
Requires rebuilding umt-browser-desktop (docker compose build camofox &&
docker compose up -d camofox) to take effect.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- browser.screenshot returns the page as an image content block (≤1280px wide, JPEG quality
configurable) for vision-capable upstreams; non-vision upstreams get a redacted text snapshot.
Login pages / password fields are marked sensitive and withheld unless a
browser.sensitive_screenshot grant exists and allowSensitive is passed.
- browser.click_at / type_at / scroll / wait act in screenshot pixels; the server converts via
viewport/DPR, requires automation control and an allowlisted host, and writes a thumbnail
evidence file (DATA_DIR/tmp/evidence) referenced from agent_run_event.
- browser.solve_challenge runs a bounded child agent_run (vision caps only, maxSteps ≤ 12,
budget), judges success from page facts, stops immediately on 2FA / SMS / device confirmation,
and otherwise moves the session to waiting with evidence and a takeover link.
- PiAgentRuntime passes modelImage outputs as pi-ai image content when model.input has image;
the ledger never stores image data; coordinate actions use per-invocation idempotency keys.
- Tests: image codec, runtime image content, E14 e2e (login solved, 2FA stop, step limit,
sensitive withheld). Real camofox 1.17.0 check on example.com recorded in the runbook.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- compose: camofox service on by default (internal only, volume umt-camofox,
healthcheck, CAMOFOX_REPLICAS=0 to skip); app depends on it with
service_started/required=false and auto-registers camofox-env from
BROWSER_PROVIDER_URL / BROWSER_ACCESS_KEY / BROWSER_ENABLED
- docker/camofox/Dockerfile: builds upstream tag v1.17.0 (389c996); fixes the
upstream Dockerfile (missing postinstall.js before npm ci, sh -c not
forwarding SIGTERM so logins were never persisted on restart)
- adapter fixes from the real container: coordinate click via evaluate +
selector, focus typing with mode=keyboard, password values redacted from
ARIA snapshots/extract, downloads field names, 410 after browser restart
treated as lost tab; navigate reopens a tab in the same session
- verified-capability table keyed by upstream version (1.17.0 all ten);
other versions stay unverified; probe reports auth failures
- settings: browser providers block (create/edit/delete/probe, capability
table); API POST/PATCH/DELETE /browser/providers, POST .../probe
- scripts/browser-verify: umtbr stack + mock NexusPHP run of REQ 4.7.4 items
- docs: runbook browser §1/§7 (real-instance record), deploy §2.1
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- GET /system/health splits processes into current instances (hosts with
a beat in the last 10 min, newest per host+kind, not stopped) and
retiredProcesses[]; 45s stale and level only consider current ones
- heartbeat writer marks stopped_at on graceful exit (migration 0043),
upsert clears it; pure api role now stops its writer on SIGTERM
- scheduled job system.heartbeat_cleanup (maintain, 1h) deletes rows
without a beat for 24h and samples older than 7 days
- status page: per-role alarms ("io worker 45 秒没有心跳"), collapsible
retired instances; glossary/runbook updated
- bump all members to 0.2.2, CHANGELOG [0.2.2]
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- contracts: isSensitiveQueryParam (whole-name match on SENSITIVE_QUERY_PARAMS
or isSensitiveKey), the single source for query credential names
- rss/redact: drop local SENSITIVE_QUERY_KEYS; param checks go through
isSensitiveQueryParam (+ caller extras); REDACTED re-exported from contracts
- browser/redact: CREDENTIAL_PARAM built from SENSITIVE_QUERY_PARAMS, JSON keys
also checked with isSensitiveKey; keeps only its own rules (path passkey,
cookie names, a11y password fields); remove stray regex statement
- redact-consistency.test: 24 credential spellings (pass_key, c_secure_pass,
refresh_token, password, uid_key, auth, ...) masked by contracts, rss and
browser alike; plain params untouched; rss extract/unredact round-trip
Co-Authored-By: Claude Opus 5.5 <[email protected]>
