camofox reaps a tab after TAB_INACTIVITY_MS (5 min) without API tool calls;
desktop-mode input goes straight to Selkies and screenshots don't count, so
tabs disappeared under the user during takeover.
- BrowserKeepalive (in-process): sessions held by an open desktop WebSocket
(ref-counted), a human takeover lease, or a running/queued agent run that
used browser.* get a keepalive every BROWSER_KEEPALIVE_MS (default 120s,
0 = off). Holds are re-checked each round; release/lease expiry/agent end/
session close stop it and camofox reaps as usual.
- Keepalive call: camofox 1.17 has no keepalive/touch endpoint and /stats does
not bump toolCalls; use GET /tabs/{id}/downloads without includeData/consume
(read-only metadata). Goes through BrowserService.run so restarts or reaped
tabs self-heal via BR-3 reconnect; no lastAutomationAt, no control needed.
- Runbook browser.md §12, deploy env table, compose env; tests against the
testkit mock camofox.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- adapters: overviewLanguages() limits overview/tagline chain to Chinese
variants (zh-CN→zh-SG→zh-TW→zh-HK); titles keep full chain; season
supplement fetch skips languages useless for overview
- setting metadata.overviewNonChineseFallback (default off) + UI switch
- media detail shows "暂无简介" for empty entity/episode overview
- scripts/overview-recompute.ts: recompute stored non-Chinese overviews
from kv_cache only (no TMDB requests), dry-run by default, --apply
- tests: chain unit tests, Heart Signal (103147) fixture, server
recompute test, MP differential for episode overview
- runbook + CHANGELOG
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- every outbound site request (adapter list/search/detail/download/health,
RSS, login, user data) carries the account UA via SourcesRuntime.siteFetch;
no more Deno default UA
- priority: account manual > pending > global setting > browser identity
(fallback: same-shape Windows Firefox UA); settings UA default is empty
- migration 0056: existing cookie/password accounts keep their old UA as
ua_pending until re-login or browser cookie write-back switches cookie+UA
in one transaction; browser_provider.identity caches the browser identity
- cf_clearance/__cf_bm stripped on browser write-back (names kept only)
- browser sessions get per-site egress (same proxy as HTTP, or direct)
- site account page shows UA source; raw UA in tech info
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- browser:launching hook generates a Windows fingerprint once (camoufox-js,
os=windows), persists it to /root/.camofox/umt-identity.json and reuses it
on every launch; Firefox/Camoufox upgrades only bump UA/version fields
- Windows fontconfig with absolute font dir (upstream path was broken)
- GET /umt/identity (public UA/OS/version) and PUT /umt/sessions/:id/proxy
(per-context proxy; Firefox no longer uses container HTTP_PROXY env)
- adapter identity()/setEgress(), splitBrowserOnlyCookies for cf_clearance/__cf_bm
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- GET /download-tasks/board?state=active|completed: per-torrent dl/up speed,
ETA, seeds/leechs, downloader total speed (transfer/info), counts for both
groups, latest organize result (status, reason code, open decision)
- live qB data cached per instance for 2 s with in-flight coalescing
- panel: total speed line, 进行中 / 已完成 segments remembered locally,
4 s polling while the page is visible, mobile layout, "—" for zero speed
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Adapters (pure functions + tests):
- detail-description: strip HTML/BBCode residue, drop site template blocks and
ad/signature lines (configurable rules), parse PT info blocks (◎ / 【】 / 『』 /
「键:」) into infoBlock and body paragraphs
- mediainfo: MediaInfo and BDInfo (quick summary + full report) into a summary
of container/duration/bitrate/size, video (HDR/DV), audio and subtitle tracks
- file-tree: file list into a flattened directory tree with sizes and
main/sample/subtitle/extra roles
- formatDetailPage combines them; the server computes it when reading the
cached detail page, so existing caches benefit without a refetch
Contracts: SiteReleasePage gains infoBlock, paragraphs, media, fileTree.
Web: drawer now reads top-down: meta row (promo/H&R, size, peers, upload time,
IMDb/TMDB/豆瓣 tags) → 资料 → 简介 (collapsible) → 技术规格 (raw folded) →
截图 grid with keyboard lightbox → 文件 tree; inventory overlay moved into the
fixed footer next to 下载.
Fixtures: 6 desensitized samples from the local cache (site.detail and RSS
descriptions) plus synthetic multi-track MediaInfo, BDInfo, bracket blocks and
directory file lists.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
recallTmdbCandidates searches cn then en names in the preferred language by
type and fetches details for a bounded set of candidates (MP alias fallback);
translations become aliases; bindAuto no longer registers generic or
alias-only site titles; snapshot/feed title changes reparse and force a
re-resolve that unbinds when it can no longer confirm.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- RSS: split NexusPHP "[category]name[subtitle][size][uploader]" titles in the
site adapter layer (category id/name/media type, clean title, subtitle,
size); snapshot fields win on merge; lazily renormalize old RSS rows and
re-emit release.observed; unknown list fields show an em dash
- Detail: HHClub section layout (douban info block as description,
screenshots section, IMDb/TMDB from MediaInfo lines, upload time field,
on-demand viewfilelist.php once); site CDN screenshots get a Referer
- Site request count now includes site-bound RSS fetches
- Origin view shows "自动化" when idle and "Agent" only after recent
automation actions (lastAutomationAt); agent step titles no longer get
squeezed to a sliver by long notes
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Any session operation that hits 404/410 (or finds no open tab) reconciles
tabs; if none survive it re-injects the account cookies (BR-2), opens a new
tab under the same userId/sessionKey, returns to the recorded URL (site home
otherwise), marks old tabs lost and retries the operation once. Failure
returns 503 browser_reconnect_failed with the reason, with a 10s backoff.
- Restart detection: umt plugin GET /umt/instance (fallback /health fields);
a changed instance id marks every open session needsReconnect.
- POST /browser/sessions/{id}/reconnect (idempotent); session view exposes
needsReconnect and reconnect {status, trigger, at, reason, humanNotice}.
- Controller and lease are kept; reconnecting during takeover asks the human
to double-check the reloaded page.
- Web: screenshot and desktop modes show "reconnecting", auto-reconnect once,
pause polling, offer a reconnect button with the reason on failure; desktop
mode rebuilds the Selkies connection after a reconnect.
- Mock camofox restart(), tests, smoke step and runbook section 10.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- SiteReleaseDetail.page: description, MediaInfo/BDInfo, screenshots,
IMDb / Douban / TMDB ids, file list, tags, uploader, plus a parse probe
(labels found) for layout-change diagnosis. Generic NexusPHP rules
(field label -> next sibling) starting from the MovieClaw detail
selectors; site overrides under umt.detail.page (HHClub: div labels).
M-Team builds the same structure from descr BBCode + mediainfo.
- Fetched only on demand (drawer open without cache, "复核详情",
release.inspect); content in kv_cache region site.detail for the
site detail cache period (>= 24 h); within the window the site is not
requested again. External ids persist in source_release.raw._detail,
which list snapshots do not overwrite.
- Screenshots only as proxy paths /api/v1/img/site-detail/{rid}/{i}:
public-address check on every hop, no site credentials, magic-byte
image check, 8 MB cap. Poster proxy shares the same helper.
- Identity: site ids count as explicit-id evidence; when TMDB is
queried, a TMDB id goes straight to details and an IMDb id to
/find (no title search on a hit). Douban only for local recall.
- Release drawer "站点详情页" section; fixtures and tests for HHClub div
layout, rowhead layout, BBCode, cache/proxy/identity end to end.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Re-lands 901b01d, which was reverted wholesale by 28974d1 (BR-2 committed
from the shared index). The agent UI files 901b01d had rolled back are
left at their AGUI-2 versions; contracts, routes, db types and the runbook
part already reached main through later commits, so this carries the rest.
HHClub (憨憨) was checked read-only against a real account (one detail
request; list, user data and health came from the product's own fetch).
Parsing fixes:
- list titles carried the site's new-torrent marker "[新]" (58 of 100
rows; the detail page title has none); stripped with a replace filter
- refreshing a detail overwrote seeders / leechers / snatched / category
name / H&R with null because the HHClub div detail page lacks them;
detail values now fall back to the list snapshot (mergeDetailOverList),
and an unknown detail promotion keeps the list promotion
- level "贵宾" is VIP (umt.vipLevels)
- sanitized minimal fixture HHANCLUB_LIST_PAGE_2026 + tests; README marks
HHClub as verified 2026-09-29
Size, counts, category ids/names, +08:00 time and subtitle were correct.
Request restraint:
- site definitions may carry umt.defaults; merged with settings.sites
(longer intervals, fewer pages): new settings defaultSnapshotMaxPages 1,
defaultHealthCheckIntervalSec 6 h, defaultDetailCacheSec 24 h; default
burst 2. HHClub: snapshot 60 min page 1 only, health 6 h, search
concurrency 1, detail cache 24 h, 1 token / 3 s burst 2
- periodic / manual snapshots are capped to that page count; scheduled
health checks skip sites checked within the interval; detail refresh is
served from cache inside the window; live searches on one site queue
- every rate-limiter token is counted per site per hour
(site_request_count, migration 0046); GET /sites/{id}/request-stats;
site overview and resource list show requests in the last 24 hours
Resource list: two-layer rows. Main line: title + spec tags (episodes,
resolution, source, codec, HDR, group, 国语/中字, site tags) + promotion
with time left + H&R. Second line: subtitle (one line, ellipsis) and
fixed-width category / size / seed-leech-snatched / upload time
(relative, absolute on hover) columns, stale badge, inventory overlay;
folds into a card on phones.
Also aligns identity_candidate.normalized_title order in db types with
the migrations (schema drift test).
Takes effect after the stack is rebuilt (migration 0046).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- metadata.enrich (propose, low, replay safe): normalized-title cache (kv_cache
region enrich, 7 days) -> daily budget -> web.search (title + 短剧 / 电视剧 平台,
second query limited to whitelist via site:) -> web.fetch of up to 3 relevant
whitelisted pages (registered site domains never read, redirects out of the
whitelist dropped) -> upstream model extraction with rule fallback -> server-side
evidence check (rule_confirmed / multi_evidence / needs_review). Suggestion only.
- POST /identity-pool/{id}/enrich (suggest) and /enrich/apply (console session):
provisional entity or same-title merge, season/episode units, field-level
metadata with source web/agent + evidence, aliases, release assignment.
Poster proxy GET /img/enrich/{id} (og:image from whitelisted pages only).
- Auto-enrich once per pool entry when subscription/rule lookups find nothing on
TMDB (identity.autoEnrich, off by default); identity.enrichDailyBudget,
identity.enrichDomains settings; web.fetch now returns og:image.
- Pool sheet "元数据补全" section; entity detail shows evidence links.
- Migration 0049_enrichment (metadata_enrichment, metadata_field.evidence, source
web). Requires image rebuild / api + worker restart to take effect.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Inject the site account Cookie (manual + auto-login session, session wins)
into the camofox profile before the first tab opens; password accounts
without a session run refresh-login first; API Key sites are skipped.
- Cookie expired after injection -> credential.status=expired; cookie
accounts go to waiting/login_required ("log in once, applies site-wide").
- Write back site-domain cookies from the browser on handover, session close,
every 10 min of activity, and from refresh-login when the stored cookie is
invalid (verified before saving). user_info.cookieOrigin records
source=browser; source shown as manual/login/browser in the origin view.
- New camofox plugin umt-cookies (GET/POST /umt/sessions/:userId/cookies
under the access key); profile dir 0700/files 0600 in the desktop image.
- Single entry: sources-identity/service/credentials.ts.
- Real check: HHClub cookie injected into a fresh profile -> home page
logged in (logout link present), 5 cookies read back.
Requires rebuilding umt-browser-desktop (docker compose build camofox &&
docker compose up -d camofox) to take effect.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
HHClub (憨憨) was checked read-only against a real account (one detail
request; list, user data and health came from the product's own fetch).
Parsing fixes:
- list titles carried the site's new-torrent marker "[新]" (58 of 100
rows; the detail page title has none); stripped with a replace filter
- refreshing a detail overwrote seeders / leechers / snatched / category
name / H&R with null because the HHClub div detail page lacks them;
detail values now fall back to the list snapshot (mergeDetailOverList),
and an unknown detail promotion keeps the list promotion
- level "贵宾" is VIP (umt.vipLevels)
- sanitized minimal fixture HHANCLUB_LIST_PAGE_2026 + tests; README marks
HHClub as verified 2026-09-29
Size, counts, category ids/names, +08:00 time and subtitle were correct.
Request restraint:
- site definitions may carry umt.defaults; merged with settings.sites
(longer intervals, fewer pages): new settings defaultSnapshotMaxPages 1,
defaultHealthCheckIntervalSec 6 h, defaultDetailCacheSec 24 h; default
burst 2. HHClub: snapshot 60 min page 1 only, health 6 h, search
concurrency 1, detail cache 24 h, 1 token / 3 s burst 2
- periodic / manual snapshots are capped to that page count; scheduled
health checks skip sites checked within the interval; detail refresh is
served from cache inside the window; live searches on one site queue
- every rate-limiter token is counted per site per hour
(site_request_count, migration 0046); GET /sites/{id}/request-stats;
site overview and resource list show requests in the last 24 hours
Resource list: two-layer rows. Main line: title + spec tags (episodes,
resolution, source, codec, HDR, group, 国语/中字, site tags) + promotion
with time left + H&R. Second line: subtitle (one line, ellipsis) and
fixed-width category / size / seed-leech-snatched / upload time
(relative, absolute on hover) columns, stale badge, inventory overlay;
folds into a card on phones.
Also aligns identity_candidate.normalized_title order in db types with
the migrations (schema drift test).
Takes effect after the stack is rebuilt (migration 0046).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- browser.screenshot returns the page as an image content block (≤1280px wide, JPEG quality
configurable) for vision-capable upstreams; non-vision upstreams get a redacted text snapshot.
Login pages / password fields are marked sensitive and withheld unless a
browser.sensitive_screenshot grant exists and allowSensitive is passed.
- browser.click_at / type_at / scroll / wait act in screenshot pixels; the server converts via
viewport/DPR, requires automation control and an allowlisted host, and writes a thumbnail
evidence file (DATA_DIR/tmp/evidence) referenced from agent_run_event.
- browser.solve_challenge runs a bounded child agent_run (vision caps only, maxSteps ≤ 12,
budget), judges success from page facts, stops immediately on 2FA / SMS / device confirmation,
and otherwise moves the session to waiting with evidence and a takeover link.
- PiAgentRuntime passes modelImage outputs as pi-ai image content when model.input has image;
the ledger never stores image data; coordinate actions use per-invocation idempotency keys.
- Tests: image codec, runtime image content, E14 e2e (login solved, 2FA stop, step limit,
sensitive withheld). Real camofox 1.17.0 check on example.com recorded in the runbook.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- compose: camofox service on by default (internal only, volume umt-camofox,
healthcheck, CAMOFOX_REPLICAS=0 to skip); app depends on it with
service_started/required=false and auto-registers camofox-env from
BROWSER_PROVIDER_URL / BROWSER_ACCESS_KEY / BROWSER_ENABLED
- docker/camofox/Dockerfile: builds upstream tag v1.17.0 (389c996); fixes the
upstream Dockerfile (missing postinstall.js before npm ci, sh -c not
forwarding SIGTERM so logins were never persisted on restart)
- adapter fixes from the real container: coordinate click via evaluate +
selector, focus typing with mode=keyboard, password values redacted from
ARIA snapshots/extract, downloads field names, 410 after browser restart
treated as lost tab; navigate reopens a tab in the same session
- verified-capability table keyed by upstream version (1.17.0 all ten);
other versions stay unverified; probe reports auth failures
- settings: browser providers block (create/edit/delete/probe, capability
table); API POST/PATCH/DELETE /browser/providers, POST .../probe
- scripts/browser-verify: umtbr stack + mock NexusPHP run of REQ 4.7.4 items
- docs: runbook browser §1/§7 (real-instance record), deploy §2.1
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- GET /system/health splits processes into current instances (hosts with
a beat in the last 10 min, newest per host+kind, not stopped) and
retiredProcesses[]; 45s stale and level only consider current ones
- heartbeat writer marks stopped_at on graceful exit (migration 0043),
upsert clears it; pure api role now stops its writer on SIGTERM
- scheduled job system.heartbeat_cleanup (maintain, 1h) deletes rows
without a beat for 24h and samples older than 7 days
- status page: per-role alarms ("io worker 45 秒没有心跳"), collapsible
retired instances; glossary/runbook updated
- bump all members to 0.2.2, CHANGELOG [0.2.2]
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Migration 0042_cache adds UNLOGGED kv_cache. PgCache mirrors MP
runtime/cache.py (region, per-key TTL, TLRU maxsize eviction, @cached as
cachedCall); only the storage moves from memory to PG. TMDB request
responses (MP TMDb.request @cached) and recognize results (MP TmdbCache:
same key, {id:0} negative cache with EMPTY_RESULT_CACHE_TTL, type-conflict
checks, backfill after manual assignment) live in kv_cache, shared by all
processes. Settings cache.tmdbTtlHours / tmdbMaxSize / emptyResultTtlSec
map to META_CACHE_EXPIRE / CONF.tmdb / EMPTY_RESULT_CACHE_TTL. TMDB
details now also store posterPath / backdropPath.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Migration 0041_rate_limit adds rate_limit_bucket. PgTokenBucket (@umt/db)
refills and takes tokens under SELECT ... FOR UPDATE on the DB clock and
polls with jitter within a wait budget; penalize() carries Retry-After.
Site adapters get createSharedSiteRateLimiter (key site:<id>), shared by
api / worker / worker:agent, with an in-process serial queue in front and
a local fallback when the bucket is unavailable.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- G6: FileRoot.pendingCount/pendingPartial for download roots (same rule as the
unidentified filter, recursive, cached 5 min, invalidated by files.changed)
- G7: FileEntry.parsedHint from parseReleasePath (file + parent dirs); the web
subline uses it and no longer parses names itself
- G8: FileIdentifyCandidate.posterUrl (TMDB poster_path via adapter and
catalog.lookup, local entities from metadata.posterPath); thumbnails in the
identify panel
- S-7 contract: FilesIdentifyResult.remoteTaskId/remotePending,
MediaServerInventory.partial, EventType identify.completed with
IdentifyCompletedPayload; identify panel (useIdentifyLive) and organize wizard
merge remote results by root + path, buffering events that beat the response
Co-Authored-By: Claude Opus 5.5 <[email protected]>
loginNexusPhp (captcha / two-factor / bad credentials / blocked / Cloudflare
classification, TOTP), SiteUserInfo.vip via profile.vip_css or umt.vipLevels,
maxSeedingPages option, MTorrent parser honours apiUrl; testkit mock NexusPHP
site gains takelogin with login guards and user detail pages.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The throttle could drop the final copy heartbeat when it landed inside the
interval (fsproxy.test saw 6291456 instead of 6291461 under ci load). Phase
completion (bytes >= total) is now sent immediately like phase changes.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- contracts: isSensitiveQueryParam (whole-name match on SENSITIVE_QUERY_PARAMS
or isSensitiveKey), the single source for query credential names
- rss/redact: drop local SENSITIVE_QUERY_KEYS; param checks go through
isSensitiveQueryParam (+ caller extras); REDACTED re-exported from contracts
- browser/redact: CREDENTIAL_PARAM built from SENSITIVE_QUERY_PARAMS, JSON keys
also checked with isSensitiveKey; keeps only its own rules (path passkey,
cookie names, a11y password fields); remove stray regex statement
- redact-consistency.test: 24 credential spellings (pass_key, c_secure_pass,
refresh_token, password, uid_key, auth, ...) masked by contracts, rss and
browser alike; plain params untouched; rss extract/unredact round-trip
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- main.ts: `--role fsworker` dispatched before the rest of the module graph is
loaded (other imports became dynamic); runs fsWorkerMain from the new
`@umt/adapters/fsworker` export (~150 ms startup in the compiled binary)
- fsproxy: default worker command picked per runtime (deno CLI -> `deno run
fsworker.ts`, compiled binary -> `<execPath> --role fsworker`); stats expose
workerKind; UMT_FS_TEST_HANG_AFTER_BYTES acceptance switch for containers
- compile: --allow-run adds /app/umt-server (spawn self); start task allows deno
- organize: replaced old file moved into the file-manager trash layout
(<root>/.umt-trash/<day>/<uuid>/<rel>) and registered in trash_item, ledger
result follows; organize.transfer log carries fsproxy stats
- settings: autoInvestigateMode (investigate/recover) + max recover attempts
- deploy.md: container verification (subprocess self, TIMEOUT -> SIGKILL x3)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- adapters/fs: FsProxy + fsworker run each file op in a killable subprocess;
progress-stall watchdog (FS_HANG_TIMEOUT_MS) for copy/hash, total timeout for
quick ops; SIGTERM -> SIGKILL -> FsError('TIMEOUT'); stat/checkRoot timeouts
never read as missing
- copyWithVerify: onExisting conflict/skip/replace (hardlink backup + atomic
rename, old file to .umt-trash or deleted), sweep stale partials before write,
document why permissions/owner are not copied
- filesMatch/filesMatchSha256 throw FsError on read failure instead of
match=false; crossDeviceMove skips redundant re-hash after sha256 copy
- FsErrorCode TIMEOUT + fsErrorClass; organize executeRecord routes fs ops
through fsproxy, retries TIMEOUT/IO up to FS_TRANSFER_MAX_ATTEMPTS then opens
a decision without deleting the record; reconcile errors no longer reported
as EEXIST
- domain: subtitle .default marker (ORGANIZE_DEFAULT_SUBTITLE_LANG) and
findOtherVersions (same season/episode/part, video only) as upgrade
disposal candidates
- scripts/bench/fs-bench.ts and benchmark record in porting doc
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- POST /approvals and PATCH /approvals/{id} require a console admin session
(EndpointDef auth:"session"); API keys incl. admin scope get 403, so an
organize.execute key can no longer approve its own plan (S1)
- tests for single-use approval checks: expired, revision/payload hash
mismatch, consumed, revoked, pending/rejected, plan mismatch; re-approve
after plan change succeeds once and cannot be replayed (S4)
- one redact key table in @umt/contracts shared by server logger and agent
redactForModel: rsskey, torrent_pass, authkey, downhash, c_secure_pass,
passkey variants, x-api-key, cookie headers, nested objects/arrays (S2)
- TMDB network errors: message and cause redacted via redactUrl (S3)
- web Agent ?mock=1 only in DEV builds or when pi.provider is fake (S7)
- EndpointDef auth annotations for /auth/*, asserted against
PUBLIC_AUTH_PATHS (S10); doc drift fixes (S11)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
