From 55841402a6fc66fb906bdc2a57066ef683f26f47 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 6 Nov 2026 00:17:37 +0800 Subject: [PATCH] docs: align Bun, routing, browser validation and MoviePilot site provenance --- CHANGELOG.md | 9 + CLAUDE.md | 4 +- README.md | 8 +- apps/server/README.md | 2 +- docker/camofox/README.md | 10 +- docs/adr/ADR-002-task-engine.md | 4 +- docs/adr/ADR-003-pi-integration.md | 4 +- docs/adr/ADR-007-bun-runtime.md | 4 +- docs/design/console-ia.md | 4 +- docs/design/ui-controls.md | 2 +- docs/plan/README.md | 2 +- docs/plan/subtitles.md | 4 +- docs/porting/license-and-provenance.md | 2 + .../moviepilot-downloader-and-sites.md | 2 + docs/porting/mp-sync-ledger.md | 110 ++++++----- docs/runbook/browser.md | 4 +- docs/runbook/deploy.md | 2 +- docs/runbook/dev.md | 10 +- docs/runbook/sites.md | 4 + docs/runbook/storage.md | 2 +- scripts/mp-sync/ledger.json | 181 ++++++++++-------- scripts/mp-sync/ledger.manual.json | 55 +++++- scripts/mp-sync/lib.mjs | 2 +- 23 files changed, 269 insertions(+), 162 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 12713b58..7bd56abf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,15 @@ 版本号遵循 [SemVer](https://semver.org/lang/zh-CN/)。所有成员包(`apps/*`、`packages/*`)与根 `package.json` 使用同一个版本号;`/api/v1/health` 与备份清单里的版本取自 `apps/server/package.json`。 +## 未发布 + +- 运行时、开发工具、Bun 类型及镜像统一锁定 Bun 1.4.4。 +- 控制台从 React Router 迁到 TanStack Router,保留已有页面地址、旧链接重定向和记录抽屉导航,构建继续输出到 `build/client`。 +- 升级前后端依赖,Pi 1.3 的工具返回适配为 `output`;OpenWorkflow 升级到 0.11,数据库测试改为文件串行以避开并发迁移锁等待。 +- 修复重复搜索同一设置字段时不再定位的问题,保留中文路由错误页;整理身份事项并发确认时复用同一计划,冲突作品选择明确拒绝。 +- 更新浏览器独立构建的 Camoufox / Playwright / MCP 依赖锁与桌面镜像 apt 源处理;新组合的容器验收状态见浏览器运行手册。 +- 对照 MoviePilot v3.2.4 同步站点适配:新增 Milkie,更新 API 地址、分类、分页、用户数据与频控错误处理;下载复用站点共享出口与限流,M-Team 字幕接入现有在线补字幕入口。 + ## [0.3.47] - 2026-10-13 - 整理默认展示作品与执行摘要,可直接开始,预览仍可单独打开;手动设置收进高级选项,去掉空来源选择步骤和重复处理入口,部分整理不再显示为完整成功。 diff --git a/CLAUDE.md b/CLAUDE.md index f430f2a0..53a1feb2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,6 @@ # UHD Media Tool -影视自动化工具:站点 / RSS 获取 → 识别 → 下载(qB)→ 整理入库 → 刮削,带 Agent。后端 Bun 1.4.3 + Hono + Kysely(PostgreSQL),前端 React Router 8 + Vite,同一个 workspace。业务逻辑以 MoviePilot v3 为参考(`docs/porting/`)。 +影视自动化工具:站点 / RSS 获取 → 识别 → 下载(qB)→ 整理入库 → 刮削,带 Agent。后端 Bun 1.4.4 + Hono + Kysely(PostgreSQL),前端 TanStack Router + Vite,同一个 workspace。业务逻辑以 MoviePilot v3 为参考(`docs/porting/`)。 面向团队内部使用,目前只有本机测试环境:**避免过度设计和过度防御**,只处理已确认会影响正常使用的问题。回复和文档用中文。 @@ -35,7 +35,7 @@ REST 端点定义按领域分在 `packages/contracts/src/api/*.ts`,`api.ts` - 位置:server 的测试统一放 `apps/server/test/<模块>/`(不再用 `src/**/__tests__`);web 与 adapters 测试和源码放同目录 `*.test.ts`;其它包放 `test/`。 - 命名按测的行为(`crash-recovery.test.ts`、`two-stage-search.test.ts`),不要用任务编号(`e4-`、`w5a2`、`-md1`)。 -- server 分三组(`apps/server/vitest.config.ts`):`unit` 不碰 PG;`pg` 真实 PG + 假引擎,文件并行;`serial` 起真实引擎 / worker / 子进程的集成与 e2e,串行。新文件缺省进 `pg`,起真实 worker / 引擎 / 子进程的必须加进 `SERIAL`。 +- server 分三组(`apps/server/vitest.config.ts`):`unit` 不碰 PG,文件并行;`pg` 真实 PG + 假引擎,文件串行,避免 OpenWorkflow 0.11 并发索引迁移等待其它测试的长事务快照;`serial` 起真实引擎 / worker / 子进程的集成与 e2e,串行。新文件缺省进 `pg`,起真实 worker / 引擎 / 子进程的必须加进 `SERIAL`。 - server 测试的建库、schema 锁、假引擎统一用 `apps/server/test/support/`(`pg.ts`、`engine.ts`),各模块 harness 只写自己的装配。 - 每个 PG 测试文件用自己的 schema;等待异步结果用轮询到条件,不写固定 sleep。 - 真机测试用 `*.real.test.ts`,靠环境变量开启,缺省跳过。 diff --git a/README.md b/README.md index d04863b1..74e73bde 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ 已实现(有自动化回归及部分真实环境 / 脱敏样本记录,四站与真实下载、媒体服务器的完整验收尚未完成): -- 来源:多 RSS(条件请求、重叠窗口、多层去重、补漏)、站点中心(观众 / 馒头 / HHClub / CHD 等 23 个站点定义,凭证、账号数据、健康检查)、站点快照与实时补查、全局搜索。 +- 来源:多 RSS(条件请求、重叠窗口、多层去重、补漏)、站点中心(观众 / 馒头 / HHClub / CHD 等 23 个 MovieClaw 定义、Milkie API 适配与 5 个公开 BT 站,凭证、账号数据、健康检查)、站点快照与实时补查、全局搜索。 - 识别:MP 对齐的标题解析、TMDB + 本地别名召回、本地实体优先、临时季集单元、待识别池、人工纠错与字段锁。 - 订阅与规则:单次 / 持续追更、多版本目标(Slot)、总集数账本、覆盖矩阵、日历;规则组(兼容 MP 规则字符串)、偏好标签与抑制记录;新作品 / 追更 / 升级三种收录模式。 - 下载:多 qB、管理标签接管、缓存卷硬阈值与恢复水位、无进展只记录不换源、缓存清理(入库已验证 + 保留期 + H&R)。 @@ -25,7 +25,7 @@ ## 快速启动 -需要 Bun 1.4.3(`curl -fsSL https://bun.sh/install | bash -s "bun-v1.4.3"`)和 Docker。解析沙箱使用 Linux seccomp;本地构建还需要 C 编译器及 Node-API 头文件(Debian/Ubuntu: `apt-get install build-essential libnode-dev`)。 +需要 Bun 1.4.4(`curl -fsSL https://bun.sh/install | bash -s "bun-v1.4.4"`)和 Docker。解析沙箱使用 Linux seccomp;本地构建还需要 C 编译器及 Node-API 头文件(Debian/Ubuntu: `apt-get install build-essential libnode-dev`)。 ```sh cp .env.example .env @@ -70,7 +70,7 @@ docker compose --env-file .env.deploy exec app umt backup # 备份 | 路径 | 职责 | |---|---| -| `apps/web` | React Router 8 SPA 控制台,沿用 uhd_web 设计系统 | +| `apps/web` | TanStack Router SPA 控制台,沿用 uhd_web 设计系统 | | `apps/server` | Hono API(`/api/v1`)+ SSE + worker 入口;业务按 `src/modules//` 分模块(见 apps/server/README.md) | | `packages/contracts` | zod schema:API/事件/能力的唯一定义 | | `packages/db` | Kysely 客户端、表类型与迁移 | @@ -89,6 +89,6 @@ docker compose --env-file .env.deploy exec app umt backup # 备份 | `scripts/` | 备份 / 恢复脚本(镜像内 `/app/scripts/`) | | `spikes/movieclaw-src/` | MovieClaw 只读源码克隆(不进 git;`scripts/mp-sync/movieclaw-import.mjs` 导入站点定义用) | -技术选型:Bun 1.4.3(锁定)+ TypeScript 7.0.2 strict,package.json workspace,npm 依赖由 bun.lock 锁定。前端 Vite 8 + React Router 8 由 Bun 执行。决策见 [ADR-007](docs/adr/ADR-007-bun-runtime.md)(取代 ADR-004)。 +技术选型:Bun 1.4.4(锁定)+ TypeScript 7.0.2 strict,package.json workspace,npm 依赖由 bun.lock 锁定。前端 Vite 8 + TanStack Router 由 Bun 执行。决策见 [ADR-007](docs/adr/ADR-007-bun-runtime.md)(取代 ADR-004)。 解析沙箱预加载解析器和 WASM 后,通过 Node-API 原生模块安装 Linux seccomp 系统调用白名单,覆盖全部 Bun 线程。任务执行阶段禁止路径文件访问、网络、执行程序和创建子进程;隔离无法建立时直接失败。超时和 RSS 超限仍由父进程 SIGKILL。生产镜像同时安装编译二进制、沙箱策略模块与内置 skills。 diff --git a/apps/server/README.md b/apps/server/README.md index 67ecd098..3f179e73 100644 --- a/apps/server/README.md +++ b/apps/server/README.md @@ -1,6 +1,6 @@ # @umt/server -服务进程(Bun 1.4.3):Hono HTTP API(`/api/v1`)、SSE 与 worker 入口。默认 `--role all` 单进程运行;部署按 `api`、`worker`(IO)、`worker:agent` 分进程。OpenAPI 见 `/api/openapi.json`、`/api/docs`。业务编排和写入放在模块 service,确定性规则放在 `@umt/domain`。 +服务进程(Bun 1.4.4):Hono HTTP API(`/api/v1`)、SSE 与 worker 入口。默认 `--role all` 单进程运行;部署按 `api`、`worker`(IO)、`worker:agent` 分进程。OpenAPI 见 `/api/openapi.json`、`/api/docs`。业务编排和写入放在模块 service,确定性规则放在 `@umt/domain`。 ## 模块骨架 diff --git a/docker/camofox/README.md b/docker/camofox/README.md index 0ebe125b..06a3d14f 100644 --- a/docker/camofox/README.md +++ b/docker/camofox/README.md @@ -10,15 +10,15 @@ docker build --network=host --build-arg APT_MIRROR=mirrors.ustc.edu.cn \ 桌面底座含 Docker HTTPS apt 源;需要时可另传 `--build-arg APT_HTTPS_PROXY=http://127.0.0.1:7890` 使用现有代理。该参数只作用于构建中的 apt HTTPS 请求,默认直连,不写入容器 apt 配置;Debian HTTP 镜像仍直连。 -固定组合:Node 24.21.0(基础镜像固定摘要)、Camoufox 152.0.4-beta.30(amd64/arm64 官方资产 SHA-256)、camoufox-js 0.12.0、playwright-core 1.60.0、better-sqlite3 13.0.3、MCP SDK 1.31.0。桌面底座继续使用已核验的 Selkies 摘要;Cookie 插件、身份、代理、运行用户、umask 和 exec 信号转发保留。 +固定组合:Node 24.21.0(基础镜像固定摘要)、Camoufox 152.0.4-beta.30(amd64/arm64 官方资产 SHA-256)、camoufox-js 0.13.0、playwright-core 1.62.1、better-sqlite3 13.0.3、MCP SDK 1.32.1。桌面底座继续使用已核验的 Selkies 摘要;Cookie 插件、身份、代理、运行用户、umask 和 exec 信号转发保留。 下载较慢时,可先把对应架构的官方资产存成某目录中的 `camoufox.zip`,设置 `CAMOUFOX_ASSET_DIR=/path/to/asset` 再运行 `build.sh`。可选命名上下文直接提供 ZIP,但仍通过完全相同的 SHA-256 校验、解压与二进制存在检查;默认仍在构建时从官方 URL 下载。 -Playwright 1.60.0 是满足 camoufox-js peer 的最新稳定版。官方 [修复 #312](https://github.com/apify/camoufox-js/commit/3fe80d8448) 说明:1.61 起 `Browser.setDefaultViewport` 的 `viewport.isMobile` 不被 Camoufox 的 Juggler scheme 接受,导致 newContext/newPage 失败。因此没有 override peer,也没有升级到 1.63 后隐藏冲突。 +camoufox-js 0.13.0 的 peer 要求 playwright-core `<1.63.0`,因此固定到 1.62.1。Camoufox 152.0.4-beta.30 已加入 Playwright 1.61 协议支持,0.13.0 也固定使用该浏览器版本。 -本地 manifest 从上游生产入口派生,去掉开发测试、OpenClaw 可选 peer 和发布脚本;保留服务、postinstall 和生产可选 MCP。安全 overrides 固定 `adm-zip` 0.6.1、`brace-expansion` 5.0.12、`fast-uri` 3.1.8、`hono` 4.13.12、`ip-address` 10.7.2;fast-uri 保留 AJV 所需的 3.x API。其它生产依赖由独立 lock 固定。这里是显式依赖补丁,没有在构建时运行 `npm audit fix`。 +本地 manifest 从上游生产入口派生,去掉开发测试、OpenClaw 可选 peer 和发布脚本;保留服务、postinstall 和生产可选 MCP。安全 overrides 固定 `adm-zip` 0.6.1、`brace-expansion` 5.0.12、`fast-uri` 3.1.9、`hono` 4.13.13、`ip-address` 10.7.2;fast-uri 保留 AJV 所需的 3.x API。其它生产依赖由独立 lock 固定。这里是显式依赖补丁,没有在构建时运行 `npm audit fix`。 -MCP 和内置插件从服务根目录的同一依赖树解析模块。镜像中的 MCP manifest 同步为 SDK 1.31.0,不复制上游未安装的独立旧 lock。`audit-2026-10-01.json` 记录此生产 lock 的 npm 审计:174 个依赖、0 条漏洞;该结果不代表操作系统包审计。两个镜像均在安装依赖前更新容器系统包,宿主机软件不变。 +MCP 和内置插件从服务根目录的同一依赖树解析模块。镜像中的 MCP manifest 同步为 SDK 1.32.1,不复制上游未安装的独立旧 lock。2026-10-22 对当前 lock 运行 `npm audit` 得到 0 条漏洞;`audit-2026-10-01.json` 是旧锁文件的历史记录。两个镜像均在安装依赖前更新容器系统包,宿主机软件不变。 需要更新依赖时,先编辑明确版本,再用固定 Node 镜像生成 lock 并审计,审查 lock 差异后构建: @@ -32,3 +32,5 @@ docker run --rm --network=host -v "$PWD/docker/camofox:/work:ro" -w /work \ ``` 隔离验收使用独立容器、端口和 profile 卷,不挂生产 `umt-camofox`。`verify-upgrade.mjs` 接受测试地址和访问 key,支持重启前后两次运行以核验 Cookie 落盘和固定身份。此验收覆盖本地行为;真实 PT 挑战、远程桌面客户端兼容和 ARM64 执行仍需各自验收。 + +2026-11-05 已在本机独立桌面容器验收上述固定组合:重启前 12 项、正常重启后 13 项通过,覆盖鉴权、固定身份、Cookie、页面创建、evaluate、快照、PNG、选择器点击、可信坐标点击与 profile 隔离。重启后实例 ID 改变,固定身份保持,Cookie 从持久化状态恢复。测试容器与专用卷已清理,未更改现有部署;日志保存在 `.local/review-20261105/browser-*.log`。 diff --git a/docs/adr/ADR-002-task-engine.md b/docs/adr/ADR-002-task-engine.md index a21ba53a..babd7c56 100644 --- a/docs/adr/ADR-002-task-engine.md +++ b/docs/adr/ADR-002-task-engine.md @@ -1,4 +1,4 @@ -# ADR-002:持久任务引擎采用 OpenWorkflow 0.10.1(锁版本),封装在 `TaskEngine` 端口之后 +# ADR-002:持久任务引擎采用 OpenWorkflow(锁版本),封装在 `TaskEngine` 端口之后 - 状态:已采纳(2026-09-27,W0-B spike 后) - 关联:REQ §11、`00-first-version-plan`(已删除,见 git `ceb66f8`) §1 与 §3.1-8、验证记录 spike `w0b-openworkflow`(已删除,见 git `ceb66f8`)、代码 `spikes/openworkflow/`(已删除,见 git `ceb66f8`) @@ -9,6 +9,8 @@ v0.1 采用 **`openworkflow@0.10.1`(精确锁版本)+ `postgres@^3.4.9`**,表放 暂不实现自研租约任务表。下文附录给出备选设计,只在触发复议条件时启用(REQ §11.1:"只保留清晰的执行接口,不提前实现第二套引擎")。 +2026-11-05 更新:当前精确锁定 `openworkflow@0.11.0` 与 `postgres@3.4.9`,原有端口和业务状态归属不变。新版本使用并发索引迁移,可能等待其它测试 schema 的长事务快照;server 的 `pg` 组改为文件串行,独立 schema 的隔离约定保留。下表记录最初 0.10.1 的验证结论,当前回归以 `bun run ci` 的任务引擎与 server 测试为准。 + ## 验证结论摘要 | 项 | 结果 | 对接入的约束 | diff --git a/docs/adr/ADR-003-pi-integration.md b/docs/adr/ADR-003-pi-integration.md index fa64d44f..39ce3e54 100644 --- a/docs/adr/ADR-003-pi-integration.md +++ b/docs/adr/ADR-003-pi-integration.md @@ -84,7 +84,7 @@ faux.setResponses([ | 重试 | `retry: { enabled, maxRetries, baseDelayMs }`、`streamOptions.timeoutMs` | | 子代理开关 | **SDK 不提供**。由我们实现一个 `agent.delegate` 能力:开新 lane 或新会话,继承父任务的授权上限、预算和取消状态;默认关闭 | -## 已知限制 +## 初次接入时的限制(历史) - 0.x 版本,存储格式 4 仍是 WIP,可能原地改变。锁精确版本,升级后跑 agent 与 server 的 Pi 相关测试(`deno task test:all`)。 - 没有子代理,没有 PG 会话存储,`watchSession` 未实现。 @@ -94,6 +94,8 @@ faux.setResponses([ ## 2026-10-08 迁到 pi-durable +2026-11-05 更新:当前 `pi-durable`、`pi-ai`、`chord` 均精确锁定 1.3.0。适配层按新版 `ToolExecutionResult` 返回 `output` 内容块,正常结果与错误结果使用同一字段;对外 `AgentRuntimePort` 与会话目录约定保留。以下记录 1.1.0 迁入时的设计。 + pi-agent-core 1.0 删掉了 harness(AgentHarness、会话存储、压缩、skills、系统提示等),只剩 `Agent` 与循环;harness 搬到 `@earendil-works/pi-durable`(实验性,版本之间 API 会变)。我们改为: - 依赖锁精确版本:`@earendil-works/pi-durable@1.1.0`、`@earendil-works/pi-ai@1.1.0`、`@earendil-works/chord@1.1.0`(pi-durable 的 Context / 文档状态),**移除** pi-agent-core。Pi 的类型仍只出现在 `packages/agent/src/pi/*`(另有 `prompt/skills.ts` 的格式约定),对外只有 `AgentRuntimePort`,端口与 `AgentRunEvent` 合同不变(`tool_call_end` 只新增可选 `durationMs`)。 diff --git a/docs/adr/ADR-007-bun-runtime.md b/docs/adr/ADR-007-bun-runtime.md index 632812fc..2f6a50a8 100644 --- a/docs/adr/ADR-007-bun-runtime.md +++ b/docs/adr/ADR-007-bun-runtime.md @@ -1,8 +1,8 @@ -# ADR-007: Bun 1.4.3 与 Linux 解析沙箱 +# ADR-007: Bun 1.4.4 与 Linux 解析沙箱 状态:采纳。取代 ADR-004 的运行时、依赖安装与编译方式;更新 ADR-005 的权限隔离实现。 -后端、前端开发工具及 Vitest 使用 Bun 1.4.3。根 package.json 管理 workspace 与统一依赖,bun.lock 锁定解析结果。.bun-version、packageManager 与 Docker 构建镜像固定同一版本。现有业务依赖版本保持不变。 +后端、前端开发工具及 Vitest 使用 Bun 1.4.4。根 package.json 管理 workspace 与统一依赖,bun.lock 锁定解析结果。.bun-version、packageManager 与 Docker 构建镜像固定同一版本。运行时迁移时保持原业务依赖版本;后续升级在 manifest 与锁文件中显式维护,Pi 1.3 与 OpenWorkflow 0.11 的适配见 ADR-003、ADR-002。 Deno 全局调用改为 Node API 或 @umt/runtime 中的 Bun HTTP、WebSocket、子进程与流适配。HTTP 出口使用 proxy:false 强制直连,显式代理使用 respectNoProxy:false;环境代理由现有出口选择函数处理。 diff --git a/docs/design/console-ia.md b/docs/design/console-ia.md index c180edb1..79ac1b55 100644 --- a/docs/design/console-ia.md +++ b/docs/design/console-ia.md @@ -193,7 +193,7 @@ | 路 | 范围 | 文件 | 预估 | | --- | --- | --- | --- | -| A 导航 / 路由 / 重定向 | 新 `NAV_GROUPS`、新路由表、旧链接重定向、`findNavItem` 与面包屑、全站 `consolePath` 调用点替换 | `components/console/nav.ts`、`nav.test.ts`、`app/routes.ts`、`routes/console/layout.tsx`、新 `routes/console/redirects.ts`、`home.tsx` 等内部链接 | 0.5–1 天 | +| A 导航 / 路由 / 重定向 | 新 `NAV_GROUPS`、新路由表、旧链接重定向、`findNavItem` 与面包屑、全站 `consolePath` 调用点替换 | `components/console/nav.ts`、`nav.test.ts`、`app/router.tsx`、`routes/console/layout.tsx`、新 `routes/console/redirects.ts`、`home.tsx` 等内部链接 | 0.5–1 天 | | B 设置页 + contracts | `settings.tsx` 改为 `settings/:section` 布局;按 3.4 重组现有表单卡片;新增站点登记表单、元数据分区、账户分区;`SETTINGS_SECTIONS` 与字段归属测试 | `routes/console/settings*.tsx`、`routes/console/settings/*.tsx`、`packages/contracts/src/settings.ts`(+ test)、`api/settings.ts`、`api/sites.ts` | 1.5–2 天 | | C 页面合并 / 拆分 | 任务页拆成 `downloads`(含 plans)与 `tasks`;订阅页拆出 `rules`;站点详情加"快照与 Feed / 账号"Tab,来源页收窄为健康总表;身份页"未识别"标记待处理;下载队列页头缓存水位条 | `routes/console/tasks*.tsx` → 新 `downloads*.tsx`、`subscriptions.tsx` → 新 `rules.tsx`、`sites.$siteId.tsx`、`sources.tsx`、`identity.tsx`、`components/workbench/*` | 2 天 | @@ -249,7 +249,7 @@ | `/console/sources?tab=snapshots&siteId=x` | `/console/sites/x?tab=snapshots` | `sources.tsx` clientLoader;无 siteId 留在来源健康 | | `/console/settings?tab=*` | `/console/settings#<分区>[/<节>]` | IA-2:`settings.tsx` clientLoader;W8-A 起 `settingsPath()` 直接生成 hash | -`nav.test.ts` 断言每个导航项、每个重定向目标都在 `routes.ts` 里;冒烟(`scripts/smoke.mjs` 的 `REDIRECT_ROUTES`)逐条打开旧地址,确认落到新地址且 200。 +`nav.test.ts` 断言每个导航项、每个重定向目标都在 `router.tsx` 里;冒烟(`scripts/smoke.mjs` 的 `REDIRECT_ROUTES`)逐条打开旧地址,确认落到新地址且 200。 ### 6.3 页面合并 / 拆分 diff --git a/docs/design/ui-controls.md b/docs/design/ui-controls.md index f5983f98..5a17ca24 100644 --- a/docs/design/ui-controls.md +++ b/docs/design/ui-controls.md @@ -2,7 +2,7 @@ > 迁入自 /opt/uhd_web `docs/features/`(2026-09-27),2026-10-08 与 uhd_web 同步。文中流媒体、播放器、落地页、套餐等段落是 uhd_web 的原始上下文,本项目没有这些页面,只作材质与 token 用法参考;以本目录 README 的约定为准。 -配合[设计 token](design-tokens.md)与[界面材质](interface-materials.md)。开发态的陈列页 `/dev/ui`(只在开发构建注册,[routes.ts](../../apps/web/app/routes.ts))把三块壳下的 token、材质和控件排在一页,换皮或改 token 先看它。 +配合[设计 token](design-tokens.md)与[界面材质](interface-materials.md)使用。控制台的实际控件以 `apps/web/app/components/ui/` 为准。 ## 按钮只有一套 diff --git a/docs/plan/README.md b/docs/plan/README.md index a4984c64..a6fa02da 100644 --- a/docs/plan/README.md +++ b/docs/plan/README.md @@ -32,7 +32,7 @@ ### 进行中 - 系统梳理后的待办:[system-review.md](system-review.md),T0~T6 按优先级。 -- 补字幕:[subtitles.md](subtitles.md),第 1、2 步已完成;待办:以后接进整理 / 订阅工作流。 +- 补字幕:[subtitles.md](subtitles.md),手动上传、SubHD/字幕库及 M-Team 在线搜索下载已完成;待办:接进整理 / 订阅工作流。 - 沙箱第二批:站点网页、RSS、网页抓取的 HTML / XML 解析迁入常驻子进程池(ADR-005)。 ### 业务(需要真实环境或决定) diff --git a/docs/plan/subtitles.md b/docs/plan/subtitles.md index 52f0fe25..5827142f 100644 --- a/docs/plan/subtitles.md +++ b/docs/plan/subtitles.md @@ -25,7 +25,9 @@ - 语言:先看文件名标签(chs / cht / 简体 / 繁體 / eng / 双语 …),再看内容(CJK、假名、谚文、繁体常用字比例);中英双语按简中处理(与整理规则一致)。 - 季集:用文件名解析(复用资源名解析器);电影只有一个目标。对不上的留空让人选;同一集多个字幕按语言区分,同语言重复的提示。 -## 在线(SubHD、字幕库) +## 在线(SubHD、字幕库、M-Team) + +- M-Team:已启用的 PT 站沿用站点账号、API 地址、代理、UA 和共享限流参与搜索,无须再登记字幕站。搜索结果使用稳定字幕 ID,不预先获取下载签名;点击下载时现场换签,后续沙箱解析、配对、写入与撤销沿用同一流程。历史来源为 `mteam`,迁移 `0096` 扩展来源约束;未接入自动补字幕。 - 搜索词:作品中文名、原名、年份,剧集带季号;结果按标题与季集匹配度排序,对不上的不列。 - 验证码:① 代码识别(SubHD SVG 路径长度表、字幕库 BMP 采样模板,在沙箱里);② 没过就由智能体视觉在浏览器里接管;③ 智能体也没过就转人工。 diff --git a/docs/porting/license-and-provenance.md b/docs/porting/license-and-provenance.md index 6ed4c67b..6e76caf2 100644 --- a/docs/porting/license-and-provenance.md +++ b/docs/porting/license-and-provenance.md @@ -4,6 +4,8 @@ ## 1. 核对结论 +2026-11-06 站点适配增量使用 MoviePilot v3.2.4(`3b156a545f14fea0285cd3ed3ea4aa49d97faa57`),根 LICENSE 仍为 GPLv3。按公开代码行为维护 TS 实现,没有复制 Python 文件、解密站点数据或引入闭源模块;新增 Milkie 为手写 API 定义,来源随定义登记。逐文件基线见 [mp-sync](mp-sync-ledger.md),以下表格保留最初来源核查的历史上下文。 + | 项 | MoviePilot | MovieClaw | | --- | --- | --- | | 仓库 | `https://github.com/jxxghp/MoviePilot`(分支 `v3`) | `https://github.com/movieclaw/movieclaw`(`main`) | diff --git a/docs/porting/moviepilot-downloader-and-sites.md b/docs/porting/moviepilot-downloader-and-sites.md index a85698be..3dbaf0db 100644 --- a/docs/porting/moviepilot-downloader-and-sites.md +++ b/docs/porting/moviepilot-downloader-and-sites.md @@ -3,6 +3,8 @@ > 格式:REQ §14.1.2 迁移交付规范。只做行为对照与 TS 重写,**不复制上游文件**(许可边界见 `license-and-provenance.md`)。 > 本文件是研发对照记录,不表示对应 TS 模块已实现或已逐函数核验。 +2026-11-06 站点框架已增量核对到 MoviePilot v3.2.4(`3b156a5`),当前实现、来源文件与保留差异见 [跟随移植账本](mp-sync-ledger.md#3-当前对照总表)。下文保留首次移植的 `e6820a8` 函数和行号记录,不作为当前全部适配状态。下载端口已改用共用 `SourcesRuntime` 的站点出口、有效凭据与 PG 限流;M-Team 字幕 API 接入现有在线补字幕流程。 + ## 0. 来源与版本 | 项 | 值 | diff --git a/docs/porting/mp-sync-ledger.md b/docs/porting/mp-sync-ledger.md index 7ae3f745..4ddb2dfd 100644 --- a/docs/porting/mp-sync-ledger.md +++ b/docs/porting/mp-sync-ledger.md @@ -10,83 +10,101 @@ ## 2. 用法 -MP 本地仓库默认 `/opt/moviepilot-plus/upstream/MoviePilot`(浅克隆,当前 `e6820a8`)。以下命令 Node 24 或 `deno run -A` 均可(不依赖根 package.json / deno.json 任务)。 +MP 本地仓库默认 `/opt/moviepilot-plus/upstream/MoviePilot`(早期浅克隆,`e6820a8`)。用仓库锁定的 Bun 运行脚本;核查其他 checkout 时显式传 `--mp <路径>`,每个条目的已同步提交由下表单独记录。 ```bash # 1) 取上游新提交(浅克隆) git -C /opt/moviepilot-plus/upstream/MoviePilot fetch --depth=1 origin <新提交或 v3> # 2) 查差异:已变更 / 已删除 / 监视目录新增,附逐文件 git diff 命令;有变化退出码 1 -node scripts/mp-sync/check.mjs --to <新提交> # 或 --mp <路径> --json +bun scripts/mp-sync/check.mjs --to <新提交> # 或 --mp <路径> --json # 3) 按 diff 增量移植 TS 文件,把文件头 @mp-source 的提交号改为新提交(新文件加标签) # 4) 重建 ledger 与下表 -node scripts/mp-sync/build-ledger.mjs --commit <新提交> +bun scripts/mp-sync/build-ledger.mjs --commit <新提交> # 5) 跑适配器测试 -cd packages/adapters && deno run -A npm:vitest run src/sites +cd packages/adapters && bun x --bun vitest run src/sites ``` MovieClaw 站点定义(与 MP 无关,但同属站点框架跟随工具): ```bash -node scripts/mp-sync/movieclaw-import.mjs # 从 v0.27.0 重新转换 23 个站点定义(覆盖层:scripts/mp-sync/movieclaw-overrides/) -node scripts/mp-sync/movieclaw-import.mjs --check # 只比较,有差异退出码 1 +bun scripts/mp-sync/movieclaw-import.mjs # 从 v0.27.0 重新转换 23 个站点定义(覆盖层:scripts/mp-sync/movieclaw-overrides/) +bun scripts/mp-sync/movieclaw-import.mjs --check # 只比较,有差异退出码 1 ``` ## 3. 当前对照总表 +2026-11-06 核对 GitHub 最新稳定发布 [v3.2.4](https://github.com/jxxghp/MoviePilot/releases/tag/v3.2.4),固定提交 `3b156a545f14fea0285cd3ed3ea4aa49d97faa57`。本轮同步站点索引、用户解析与共享频控涉及的 42 个来源文件;其余 12 个媒体服务器、RSS、下载编排、文件管理、TMDB 与通用语料条目保留原基线,不据此宣称全应用已追平。 + +| 范围 | 本次同步 | +| --- | --- | +| 资源搜索 | 新增 Milkie;M-Team、HDDolby、Yema 分类透传与 API 地址覆盖;TNode 会话隔离;通用 XML/RSS、空 Feed 与无服务端分页的关键词窗口 | +| 用户数据 | NexusPHP UUID、专用做种入口与流量回填;Audiences 顶栏用户识别;Discuz 分享率、Unit3D 用户名 ID 与积分入口;M-Team 等级名、空做种页停止;HDDolby API 前缀及受限 schema 回退 | +| 公共请求 | 复用既有 `site:<数据库 ID>` 的 PG 桶;种子下载使用同一出口、账号 UA 和有效凭据;远端频控保留原因与冷却,错误页面标题脱敏 | +| M-Team 字幕 | API 搜索、按种子列字幕和下载接入现有补字幕流程;来源枚举与迁移 `0096`,解析、写入、撤销沿用现有服务 | + +保留项目边界:站点定义继续来自已核实的 MovieClaw MIT 快照与手写定义,不使用 MP 加密资源;MoviePilot 的采集器迁到 [MoviePilot-Site-Collector](https://github.com/jxxghp/MoviePilot-Site-Collector),该工具的迁移不代表站点配置变为可直接导入。Milkie 默认未启用。API Key、Cookie 和下载签名不进入搜索结果或历史;M-Team 字幕搜索只返回稳定 ID,下载时现场换签。无 Cookie 的签名下载不复用浏览器 profile,CDN 跳转也不转发账号凭据。 + +Audiences 的 compact 指标使用局部 XPath,比上游仍可能回到整文档的绝对 XPath 更严格,避免列表卡片污染当前用户数据。分页上限、ID 字符串化、缺失数值归一及确定性绝对日期解析延续既有合同。Python 类型整理、httpx 连接池恢复和 dateparser 语言加载属于上游实现细节,不另造对应 TS 机制。 + +已用 21 个自编 HTML/JSON 案例动态运行固定上游解析器做差分,按合同归一后 21/21 一致;自动化站点与字幕回归使用模拟请求及隔离数据库。真实 PT 账号、站点挑战和端到端下载仍需各自现场验收,未计入这些自动化结论。 + -共 51 个上游文件(`adapters/` = `packages/adapters/src/`)。 +共 54 个上游文件(`adapters/` = `packages/adapters/src/`)。 | 上游文件 | 本项目文件 | 移植提交 | sha256(前 12 位) | 来源 | | --- | --- | --- | --- | --- | -| `app/adapters/network/cloudflare.py` | `adapters/sites/user-parsers/common.ts` | `e6820a8` | `a23a2c44da05` | tag | +| `app/adapters/network/cloudflare.py` | `adapters/sites/user-parsers/common.ts` | `3b156a5` | `c50274ad9e69` | tag | | `app/application/mediaserver.py` | `adapters/emby/client.ts` | `e6820a8` | `f034f0be8ba6` | manual | | `app/application/rss.py` | `adapters/rss/fetch.ts`
`adapters/rss/parse.ts`
`adapters/rss/rss-link.ts` | `e6820a8` | `8ea70f712512` | manual | -| `app/application/site/sites.pyi` | `adapters/sites/definitions/schema.ts`
`adapters/sites/registry.ts`
`adapters/sites/spider/schema.ts` | `e6820a8` | `8fea9a239354` | manual | +| `app/application/site/sites.pyi` | `adapters/sites/definitions/schema.ts`
`adapters/sites/registry.ts`
`adapters/sites/spider/schema.ts` | `3b156a5` | `b3f0ded80227` | manual | +| `app/application/site/throttle.py` | `apps/server/src/modules/acquisition-organize/ports.ts`
`adapters/sites/rate-limit.ts` | `3b156a5` | `6ed524a1c2b4` | manual | +| `app/application/torrent/download.py` | `adapters/sites/http.ts` | `3b156a5` | `11affe81a5af` | manual | | `app/chain/download/submission.py` | `adapters/sites/mteam/adapter.ts`
`adapters/sites/spider/adapter.ts`
`adapters/sites/spiders/common.ts` | `e6820a8` | `f7cb870e47d6` | manual+tag | -| `app/domain/site.py` | `adapters/sites/user-parsers/common.ts` | `e6820a8` | `650c795a4def` | tag | -| `app/foundation/dom.py` | `adapters/sites/user-parsers/dom.ts` | `e6820a8` | `6d6accf2cef1` | tag | -| `app/foundation/size.py` | `adapters/sites/spider/spider.ts` | `e6820a8` | `7b0b8f50588c` | manual | -| `app/foundation/temporal.py` | `adapters/sites/spiders/common.ts`
`adapters/sites/user-parsers/common.ts` | `e6820a8` | `74941f460e40` | tag | -| `app/foundation/text.py` | `adapters/sites/user-parsers/common.ts` | `e6820a8` | `a88c75dc8431` | tag | +| `app/domain/site.py` | `adapters/sites/user-parsers/common.ts` | `3b156a5` | `5e6b8ebcce64` | tag | +| `app/foundation/dom.py` | `adapters/sites/user-parsers/dom.ts` | `3b156a5` | `38b63b12ee03` | tag | +| `app/foundation/size.py` | `adapters/sites/spider/spider.ts` | `3b156a5` | `ee3289e2cbdd` | manual | +| `app/foundation/temporal.py` | `adapters/sites/spiders/common.ts`
`adapters/sites/user-parsers/common.ts` | `3b156a5` | `237d9f55de13` | tag | +| `app/foundation/text.py` | `adapters/sites/user-parsers/common.ts` | `3b156a5` | `8d8c5ed7aa81` | tag | | `app/modules/__init__.py` | `adapters/qbittorrent/paths.ts` | `e6820a8` | `2edada0ea591` | manual | | `app/modules/_base/downloader.py` | `adapters/qbittorrent/torrent.ts` | `e6820a8` | `84bd8946c8b2` | manual | | `app/modules/emby/emby.py` | `adapters/emby/client.ts` | `e6820a8` | `e6468ea34ef2` | manual | | `app/modules/filemanager/storages/local.py` | `adapters/fs/copy.ts`
`adapters/fs/move.ts` | `e6820a8` | `806a1970ca8e` | manual | | `app/modules/filemanager/transhandler.py` | `adapters/fs/move.ts` | `e6820a8` | `13e2bba491ac` | manual | -| `app/modules/indexer/__init__.py` | `adapters/sites/registry.ts`
`adapters/sites/spider/adapter.ts`
`adapters/sites/spiders/adapter.ts`
`adapters/sites/spiders/index.ts`
`adapters/sites/spiders/map.ts`
`adapters/sites/spiders/types.ts`
`adapters/sites/user-parsers/registry.ts` | `e6820a8` | `e9c5d15fdaa5` | manual+tag | -| `app/modules/indexer/parser/__init__.py` | `adapters/sites/user-parsers/base.ts`
`adapters/sites/user-parsers/types.ts` | `e6820a8` | `0e6c4215b713` | tag | -| `app/modules/indexer/parser/bitpt.py` | `adapters/sites/user-parsers/bitpt.ts` | `e6820a8` | `871d25e570f1` | tag | -| `app/modules/indexer/parser/discuz.py` | `adapters/sites/user-parsers/discuz.ts` | `e6820a8` | `12277ba955ce` | tag | -| `app/modules/indexer/parser/file_list.py` | `adapters/sites/user-parsers/file-list.ts` | `e6820a8` | `f92e65ed7327` | tag | -| `app/modules/indexer/parser/gazelle.py` | `adapters/sites/user-parsers/gazelle.ts` | `e6820a8` | `c10e5246724b` | tag | -| `app/modules/indexer/parser/hddolby.py` | `adapters/sites/user-parsers/hddolby.ts` | `e6820a8` | `a4336d92d603` | tag | -| `app/modules/indexer/parser/ipt.py` | `adapters/sites/user-parsers/ipt.ts` | `e6820a8` | `2b9d04d76606` | tag | -| `app/modules/indexer/parser/mtorrent.py` | `adapters/sites/user-parsers/mtorrent.ts` | `e6820a8` | `c5d5a3c6f9f5` | tag | -| `app/modules/indexer/parser/nexus_audiences.py` | `adapters/sites/user-parsers/nexus-audiences.ts` | `e6820a8` | `1a2d104db769` | tag | -| `app/modules/indexer/parser/nexus_hhanclub.py` | `adapters/sites/user-parsers/nexus-hhanclub.ts` | `e6820a8` | `8850619069d4` | tag | -| `app/modules/indexer/parser/nexus_php.py` | `adapters/sites/user-parsers/nexus-php.ts` | `e6820a8` | `39303ae96d9c` | tag | -| `app/modules/indexer/parser/nexus_project.py` | `adapters/sites/user-parsers/nexus-project.ts` | `e6820a8` | `2b5508cfb2f8` | tag | -| `app/modules/indexer/parser/nexus_rabbit.py` | `adapters/sites/user-parsers/nexus-rabbit.ts` | `e6820a8` | `09149b89a7c7` | tag | -| `app/modules/indexer/parser/rousi.py` | `adapters/sites/user-parsers/rousi.ts` | `e6820a8` | `1b1ea93de084` | tag | -| `app/modules/indexer/parser/small_horse.py` | `adapters/sites/user-parsers/small-horse.ts` | `e6820a8` | `b73b5de97737` | tag | -| `app/modules/indexer/parser/sunnypt.py` | `adapters/sites/user-parsers/sunnypt.ts` | `e6820a8` | `937f8453fa2b` | tag | -| `app/modules/indexer/parser/tnode.py` | `adapters/sites/user-parsers/tnode.ts` | `e6820a8` | `c9cb809fc4bf` | tag | -| `app/modules/indexer/parser/torrent_leech.py` | `adapters/sites/user-parsers/torrent-leech.ts` | `e6820a8` | `72d1473a3bba` | tag | -| `app/modules/indexer/parser/unit3d.py` | `adapters/sites/user-parsers/unit3d.ts` | `e6820a8` | `9a4d579deca5` | tag | -| `app/modules/indexer/parser/yema.py` | `adapters/sites/user-parsers/yema.ts` | `e6820a8` | `0f98fd580bad` | tag | -| `app/modules/indexer/parser/zhixing.py` | `adapters/sites/user-parsers/zhixing.ts` | `e6820a8` | `90c8954e7387` | tag | -| `app/modules/indexer/spider/__init__.py` | `adapters/sites/spider/map.ts`
`adapters/sites/spider/schema.ts`
`adapters/sites/spider/spider.ts`
`adapters/sites/spider/template.ts` | `e6820a8` | `2513d962fa43` | manual | -| `app/modules/indexer/spider/haidan.py` | `adapters/sites/spiders/haidan.ts` | `e6820a8` | `75eb802f19f0` | tag | -| `app/modules/indexer/spider/hddolby.py` | `adapters/sites/spiders/hddolby.ts` | `e6820a8` | `46a50bd8d49f` | tag | -| `app/modules/indexer/spider/mtorrent.py` | `adapters/sites/mteam/adapter.ts`
`adapters/sites/mteam/client.ts`
`adapters/sites/mteam/spider.ts`
`adapters/sites/spiders/mtorrent.ts` | `e6820a8` | `a8ab2b822cb2` | manual+tag | -| `app/modules/indexer/spider/rousi.py` | `adapters/sites/spiders/rousi.ts` | `e6820a8` | `aa13c386a560` | tag | -| `app/modules/indexer/spider/sunnypt.py` | `adapters/sites/spiders/sunnypt.ts` | `e6820a8` | `735b0d721e65` | tag | -| `app/modules/indexer/spider/tnode.py` | `adapters/sites/spiders/tnode.ts` | `e6820a8` | `a3634a268dba` | tag | -| `app/modules/indexer/spider/torrentleech.py` | `adapters/sites/spiders/torrentleech.ts` | `e6820a8` | `c965f2bcaea4` | tag | -| `app/modules/indexer/spider/yema.py` | `adapters/sites/spiders/yema.ts` | `e6820a8` | `560b62c1ff10` | tag | +| `app/modules/indexer/__init__.py` | `adapters/sites/registry.ts`
`adapters/sites/spider/adapter.ts`
`adapters/sites/spiders/adapter.ts`
`adapters/sites/spiders/index.ts`
`adapters/sites/spiders/map.ts`
`adapters/sites/spiders/types.ts`
`adapters/sites/user-parsers/registry.ts` | `3b156a5` | `4366034bdc25` | manual+tag | +| `app/modules/indexer/parser/__init__.py` | `adapters/sites/user-parsers/base.ts`
`adapters/sites/user-parsers/types.ts` | `3b156a5` | `c6d7e45058cf` | tag | +| `app/modules/indexer/parser/bitpt.py` | `adapters/sites/user-parsers/bitpt.ts` | `3b156a5` | `daccec01e01d` | tag | +| `app/modules/indexer/parser/discuz.py` | `adapters/sites/user-parsers/discuz.ts` | `3b156a5` | `f056cb1f3691` | tag | +| `app/modules/indexer/parser/file_list.py` | `adapters/sites/user-parsers/file-list.ts` | `3b156a5` | `1e101fe651f5` | tag | +| `app/modules/indexer/parser/gazelle.py` | `adapters/sites/user-parsers/gazelle.ts` | `3b156a5` | `37d6b2017708` | tag | +| `app/modules/indexer/parser/hddolby.py` | `adapters/sites/user-parsers/hddolby.ts` | `3b156a5` | `1a9f885333c3` | tag | +| `app/modules/indexer/parser/ipt.py` | `adapters/sites/user-parsers/ipt.ts` | `3b156a5` | `fffccd341ace` | tag | +| `app/modules/indexer/parser/mtorrent.py` | `adapters/sites/user-parsers/mtorrent.ts` | `3b156a5` | `1f35d41add01` | tag | +| `app/modules/indexer/parser/nexus_audiences.py` | `adapters/sites/user-parsers/nexus-audiences.ts` | `3b156a5` | `7b53f3c76c5e` | tag | +| `app/modules/indexer/parser/nexus_hhanclub.py` | `adapters/sites/user-parsers/nexus-hhanclub.ts` | `3b156a5` | `320c0979c192` | tag | +| `app/modules/indexer/parser/nexus_php.py` | `adapters/sites/user-parsers/nexus-php.ts` | `3b156a5` | `8581be9deb8f` | tag | +| `app/modules/indexer/parser/nexus_project.py` | `adapters/sites/user-parsers/nexus-project.ts` | `3b156a5` | `5bf6662060df` | tag | +| `app/modules/indexer/parser/nexus_rabbit.py` | `adapters/sites/user-parsers/nexus-rabbit.ts` | `3b156a5` | `3d5434742282` | tag | +| `app/modules/indexer/parser/rousi.py` | `adapters/sites/user-parsers/rousi.ts` | `3b156a5` | `f1ee95dfef78` | tag | +| `app/modules/indexer/parser/small_horse.py` | `adapters/sites/user-parsers/small-horse.ts` | `3b156a5` | `adb5efa1f74e` | tag | +| `app/modules/indexer/parser/sunnypt.py` | `adapters/sites/user-parsers/sunnypt.ts` | `3b156a5` | `c5449afb4930` | tag | +| `app/modules/indexer/parser/tnode.py` | `adapters/sites/user-parsers/tnode.ts` | `3b156a5` | `699061470045` | tag | +| `app/modules/indexer/parser/torrent_leech.py` | `adapters/sites/user-parsers/torrent-leech.ts` | `3b156a5` | `cf0debbfacd9` | tag | +| `app/modules/indexer/parser/unit3d.py` | `adapters/sites/user-parsers/unit3d.ts` | `3b156a5` | `cfa271284dcb` | tag | +| `app/modules/indexer/parser/yema.py` | `adapters/sites/user-parsers/yema.ts` | `3b156a5` | `da2a09e2d855` | tag | +| `app/modules/indexer/parser/zhixing.py` | `adapters/sites/user-parsers/zhixing.ts` | `3b156a5` | `00de9540751f` | tag | +| `app/modules/indexer/spider/__init__.py` | `adapters/sites/spider/map.ts`
`adapters/sites/spider/schema.ts`
`adapters/sites/spider/spider.ts`
`adapters/sites/spider/template.ts` | `3b156a5` | `2456e822939b` | manual+tag | +| `app/modules/indexer/spider/haidan.py` | `adapters/sites/spiders/haidan.ts` | `3b156a5` | `315f775c3413` | tag | +| `app/modules/indexer/spider/hddolby.py` | `adapters/sites/spiders/hddolby.ts` | `3b156a5` | `f0f6e9467f4a` | tag | +| `app/modules/indexer/spider/milkie.py` | `adapters/sites/spiders/milkie.ts` | `3b156a5` | `8e967235574f` | tag | +| `app/modules/indexer/spider/mtorrent.py` | `adapters/sites/mteam/adapter.ts`
`adapters/sites/mteam/client.ts`
`adapters/sites/mteam/definition.ts`
`adapters/sites/mteam/spider.ts`
`adapters/sites/mteam/subtitles.ts`
`adapters/sites/spiders/mtorrent.ts` | `3b156a5` | `6a07b5d161e9` | manual+tag | +| `app/modules/indexer/spider/rousi.py` | `adapters/sites/spiders/rousi.ts` | `3b156a5` | `20f046e4be62` | tag | +| `app/modules/indexer/spider/sunnypt.py` | `adapters/sites/spiders/sunnypt.ts` | `3b156a5` | `3b8c98086932` | tag | +| `app/modules/indexer/spider/tnode.py` | `adapters/sites/spiders/tnode.ts` | `3b156a5` | `fe0641b3c6bc` | tag | +| `app/modules/indexer/spider/torrentleech.py` | `adapters/sites/spiders/torrentleech.ts` | `3b156a5` | `b6c794453473` | tag | +| `app/modules/indexer/spider/yema.py` | `adapters/sites/spiders/yema.ts` | `3b156a5` | `4ed0234b9201` | tag | | `app/modules/qbittorrent/qbittorrent.py` | `adapters/qbittorrent/client.ts`
`adapters/qbittorrent/state.ts` | `e6820a8` | `ab02e97e986f` | manual | | `app/modules/themoviedb/tmdbapi.py` | `adapters/tmdb/client.ts`
`adapters/tmdb/match.ts` | `e6820a8` | `787553a6f57d` | manual | -| `app/schemas/site.py` | `adapters/sites/user-parsers/types.ts` | `e6820a8` | `e7952ed40b7b` | tag | +| `app/schemas/site.py` | `adapters/sites/user-parsers/types.ts` | `3b156a5` | `78739bcb7d79` | tag | | `tests/cases/meta.py` | `packages/domain/test/differential/meta-mp.json`
`packages/domain/test/differential/meta-samples.json` | `e6820a8` | `3af375cf2c43` | manual | | `tests/test_rust_accel.py` | `packages/adapters/test/differential/indexer.json`
`scripts/mp-sync/gen-differential.py` | `e6820a8` | `0259f245d278` | manual | diff --git a/docs/runbook/browser.md b/docs/runbook/browser.md index 3367291a..ad78e9f9 100644 --- a/docs/runbook/browser.md +++ b/docs/runbook/browser.md @@ -27,7 +27,7 @@ docker compose --env-file .env.deploy ps camofox # healthy ``` 上游不发布镜像。compose 缺省用的是**浏览器桌面镜像** `umt-browser-desktop:1.18.0`(`docker/browser-desktop`,§8), -当前默认版本为 1.18.0,构建补丁、锁文件和 Playwright 1.60 协议兼容例外见 [镜像说明](../../docker/camofox/README.md)。以下旧日期的验收表保留历史版本。 +当前默认源码版本为 1.18.0,构建补丁、锁文件和 Playwright peer 上限见 [镜像说明](../../docker/camofox/README.md)。以下旧日期的验收表保留历史版本。 它从纯截图镜像 `camofox-browser:1.18.0` 复制 camofox,所以**先构建基础镜像**:`docker/camofox/Dockerfile` 从 git tag `v1.18.0` 构建(修了上游 Dockerfile 的两个问题,见文件头)。首次构建要从 GitHub 拉源码与约 630 MB 的 Camoufox 浏览器包: @@ -691,7 +691,7 @@ camofox 按 **API 调用**判断活动,有三道空闲回收(`docker-compose.yml ## 16 2026-10-01 的 1.18 升级复测 -当前默认桌面镜像为 `umt-browser-desktop:1.18.0`,基础镜像 `camofox-browser:1.18.0` 锁定上游 `f94c81babf4bfd44cdd4bd46e1f2e1fb1d91c753`。Node 24.21.0、Camoufox 152.0.4-beta.30、camoufox-js 0.12.0;Playwright 保持官方协议可兼容的 1.60.0,原因和独立锁见 [构建说明](../../docker/camofox/README.md)。 +当前默认桌面镜像为 `umt-browser-desktop:1.18.0`,基础镜像 `camofox-browser:1.18.0` 锁定上游 `f94c81babf4bfd44cdd4bd46e1f2e1fb1d91c753`。当前依赖锁使用 Node 24.21.0、Camoufox 152.0.4-beta.30、camoufox-js 0.13.0 和 Playwright 1.62.1,原因和独立锁见 [构建说明](../../docker/camofox/README.md)。2026-11-05 在独立桌面容器和全新 profile 卷验证了当前组合,重启前 12 项、正常重启后 13 项通过,包括身份保持、Cookie 落盘、快照、PNG、可信点击和 profile 隔离;测试容器与卷已清理。下述 2026-10-01 的现场验证使用升级前的依赖组合,其中真实站点相关结论不自动扩展到当前组合。 用生产 profile 与桌面配置的独立副本验证了鉴权、原身份保持、Cookie 正常重启落盘、上下文/页面、evaluate、快照、PNG、选择器与可信坐标点击、profile 隔离。现有真实 BrowserService/Agent 能力核验脚本 `scripts/browser-verify/verify.ts` 的 setup/a/b/idle/c/d 共 25 项严格通过,包含登录、脱敏、提取、下载文件哈希及资源关联、正常重启与空闲回收自动重连、旧引用拒绝、SIGKILL 未落盘登录丢失识别。适配器能力表现已登记 1.18.0 的原十项能力,未知版本不继承核验结果。 diff --git a/docs/runbook/deploy.md b/docs/runbook/deploy.md index 1d2b488e..696a404d 100644 --- a/docs/runbook/deploy.md +++ b/docs/runbook/deploy.md @@ -13,7 +13,7 @@ docker build -t uhd-media-tool:latest . # 或 docker com APP_VERSION=v0.3.11 docker compose --env-file .env.deploy build app # 示例:发布时填写对应版本,写进 web 的 version.json ``` -- 多阶段:`oven/bun:1.4.3`(装依赖 → web 构建 → server 编译)→ `debian:trixie-slim` 运行镜像。 +- 多阶段:`oven/bun:1.4.4`(装依赖 → web 构建 → server 编译)→ `debian:trixie-slim` 运行镜像。 - 运行镜像额外装 `gosu`(降权)及 PG 客户端动态库,从固定的 `postgres:18.6-trixie` 官方镜像复制 pg_dump/pg_restore/psql 与 libpq 18, 外加 `/usr/local/bin/ffprobe`:从 `mwader/static-ffmpeg:9.0.2` 拷单个静态二进制(约 135 MB;Debian `ffmpeg` 包要 200+ 个依赖约 450 MB)。 换来源用 `--build-arg FFPROBE_IMAGE=<含 /ffprobe 的镜像>`。 diff --git a/docs/runbook/dev.md b/docs/runbook/dev.md index c035c8b3..9ea7a699 100644 --- a/docs/runbook/dev.md +++ b/docs/runbook/dev.md @@ -2,7 +2,7 @@ ## 前置 -- Bun 1.4.3(锁定)、Docker;Linux 解析沙箱构建需要 C 编译器和 Node-API 头文件(Debian: `build-essential libnode-dev`)。不需要单独安装 Node.js / pnpm。 +- Bun 1.4.4(锁定)、Docker;Linux 解析沙箱构建需要 C 编译器和 Node-API 头文件(Debian: `build-essential libnode-dev`)。不需要单独安装 Node.js / pnpm。 - 端口:web 5180、API 4390、PG 55433。4380 留给本机部署版(Caddy),本机 3000/5175/5176/55432 已被其他项目占用,不要改用。 ## 首次启动 @@ -16,6 +16,10 @@ bun run dev # 同时起 server(4390)与 web(5180) 单独起:`bun run dev:server`(`bun --watch`,默认 `--role all`,API + worker 同进程)、`bun run dev:web`。 +前端使用 Vite 8 + TanStack Router,路由表在 `apps/web/app/router.tsx`,构建入口为 `apps/web/index.html` 与 `app/entry.client.tsx`。`VITE_API_TARGET` 可覆盖开发代理目标,默认 `http://localhost:4390`;生产产物仍输出到 `apps/web/build/client`,由后端按 SPA 路由回退到 `index.html`。 + +server 测试按 `unit`、`pg`、`serial` 分组运行。`unit` 文件并行,其余两组各自文件串行;OpenWorkflow 0.11 的并发索引迁移会等待其它 schema 的长事务快照,因此不同 schema 的数据库测试也不能并行跑。测试连接开发库,不使用部署库。 + ## 验证 ```sh @@ -105,6 +109,8 @@ curl -H "Authorization: Bearer umt-dev-admin-key" "localhost:4390/api/v1/events? 身份重识别的幂等判断同时覆盖季集、日期、发布组和识别词命中;作品候选相同不代表解析事实相同,不能跳过事实变更后的季集重算。 +整理身份事项按决策 ID 使用稳定恢复幂等键,在原记录的重新规划锁内复用有效后继。人工与智能体同时确认同一作品只入队一次;并发选择不同作品时返回冲突,不能让同一来源生成两份执行计划。 + 季集计算和识别词归 sources-identity,整理请求、文件操作与入库状态归 acquisition-organize。整理把真实失败文件和同批正常文件样本交给映射事项,模拟与写入共用同一判断;映射重算后以 `media.episode_mapping_changed` 通知整理模块,只有季集已经确定的未完成记录重新规划。恢复预检与规划复用 `planFileEpisodes`,保留原资源的发布时间等解析上下文。Agent 的调查运行、预算和授权沿用现有机制,不另外创建恢复框架;身份补齐、重新排队和实际入库完成分别按业务事实核实。 维护时先验证真实的成功、失败和恢复路径,尤其是授权、文件替换、取消与外部副作用;删除空壳或无使用者的机制时同步删除其专属测试与文档。不要为了目录对称增加转发层,也不要把不同阶段的状态合并成一个枚举。 @@ -203,7 +209,7 @@ docker compose up -d --build # 服务:app(4380)+ postgres(数据卷 umt-p curl localhost:4380/api/v1/health ``` -- 镜像:`oven/bun:1.4.3` 构建 web 与 `bun build --compile` 出二进制,并构建 `sandbox-policy.node`,运行时 `debian:trixie-slim` + gosu + postgresql-client + 静态 ffprobe,root 启动修 `/data` 属主后降权到 PUID(缺省 10001)。 +- 镜像:`oven/bun:1.4.4` 构建 web 与 `bun build --compile` 出二进制,并构建 `sandbox-policy.node`,运行时 `debian:trixie-slim` + gosu + postgresql-client + 静态 ffprobe,root 启动修 `/data` 属主后降权到 PUID(缺省 10001)。 - 入口 `docker/entrypoint.sh`:迁移 → 自检 → 同时起 api、worker、worker:agent 三个进程;任一退出则停掉其余并以非 0 退出,由 `restart: unless-stopped` 拉起。 - 主进程使用非 root 用户和只读的 `/app` 产物;外部字节解析另起子进程,由原生 seccomp 策略限制文件、网络与进程访问。镜像带静态 ffprobe;运行期写入落在 `DATA_DIR`(容器内 `/data`)。完整部署说明见 [deploy.md](deploy.md)。 - 构建版本号:`APP_VERSION=v0.1.0 docker compose build`(写入 web 的 `version.json`)。 diff --git a/docs/runbook/sites.md b/docs/runbook/sites.md index 9e396c39..a35e2c74 100644 --- a/docs/runbook/sites.md +++ b/docs/runbook/sites.md @@ -2,6 +2,10 @@ 规格:`docs/design/site-center.md`。实现:`apps/server/src/modules/sources-identity/service/account.ts`(凭证、刷新登录、用户数据、健康检测、删除、定时任务)、`routes/sources.ts`、`runtime.ts`(凭证注入、站点默认、代理);账密登录 `packages/adapters/src/sites/login/nexusphp.ts`;迁移 `0031_site_center`。 +2026-11-06 按 MoviePilot v3.2.4 同步公开站点框架,具体范围与偏离见 [移植账本](../porting/mp-sync-ledger.md)。Milkie 作为手写 API 定义登记,需要 API Key,缺省未启用;未进行真实账号验收的站点不会因此自动启用。 + +搜索、列表、详情、种子下载与 M-Team 字幕共用站点出口、账号 UA 和 `site:<数据库 ID>` 的 PG 限流桶。远端明确频控保留原因并顺延同一桶;不会把限流记作凭据失效。M-Team 的 `config.apiBaseUrl` 优先于定义默认 API 基址;用户数据解析也读取该配置及定义的 `api_url`,`config.userdataPath` 可指定用户信息页。账号校验保留只使用声明 schema 的行为,公用用户数据刷新入口只在 Gazelle、NexusPHP、Unit3D 间回退。 + ## 站点类型与获取方式(IA-2,迁移 `0055_site_kind`) - `site.kind`:`pt`(PT 站,从站点定义库选)/ `public_bt`(公开 BT 站,不登录)/ `rss_feed`(自定义 RSS 源,只有 RSS)/ `torznab`(Jackett、Prowlarr 等的 Torznab 接口,**只定义结构**:`TorznabSiteConfig{apiUrl, apiKeySecretRef, indexerId, categories}`,不能添加、不参与调度)。定义在 `packages/contracts/src/sources.ts` 的 `SITE_KINDS`。 diff --git a/docs/runbook/storage.md b/docs/runbook/storage.md index 7d2f27c3..bafd47b3 100644 --- a/docs/runbook/storage.md +++ b/docs/runbook/storage.md @@ -108,7 +108,7 @@ PATCH /api/v1/settings {"storage":{"retentionDays":{"observation":60}}} | `scheduled_job_run` | 不按天数:每个任务只留最近 50 次(`trimRuns`,原有) | — | | `cache_stat` | 不按天数:写入统计时顺带删 30 天前的行(原有) | — | -引擎语义核对(openworkflow 0.10.1):run 幂等键只在 24 小时内去重(`DEFAULT_RUN_IDEMPOTENCY_PERIOD_MS`), +引擎语义核对(openworkflow 0.11.0):run 幂等键只在 24 小时内去重(`DEFAULT_RUN_IDEMPOTENCY_PERIOD_MS`), 认领 / 恢复只看 pending / running / sleeping,父 run 在 step 里等子 run 时读子 run 状态。 所以删 14 天前的终态 run 不影响去重与恢复;父 run 未结束的子 run 保留。业务侧去重看 `task` 表(`kind + idempotency_key`) 与 `enqueueSweep` 的时间槽,不依赖引擎旧 run。 diff --git a/scripts/mp-sync/ledger.json b/scripts/mp-sync/ledger.json index 10f6b82c..193f6573 100644 --- a/scripts/mp-sync/ledger.json +++ b/scripts/mp-sync/ledger.json @@ -10,8 +10,8 @@ "entries": [ { "upstream": "app/adapters/network/cloudflare.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "a23a2c44da051cd36f5dca87fe834a9518fee2019a1088aa2eecd88b3cafcca4", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "c50274ad9e691eab7eafa6ef61f94609fedb27d095f4267572f99053dc73d799", "ours": ["packages/adapters/src/sites/user-parsers/common.ts"], "source": "tag" }, @@ -35,8 +35,8 @@ }, { "upstream": "app/application/site/sites.pyi", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "8fea9a239354f2b7eebeaf912a569ce0fa161e18b9483c49a802d828c5b10c51", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "b3f0ded8022739b361b730c7dcc47b949c28f0be466d846c15b396937f8b86f0", "ours": [ "packages/adapters/src/sites/definitions/schema.ts", "packages/adapters/src/sites/registry.ts", @@ -44,6 +44,20 @@ ], "source": "manual" }, + { + "upstream": "app/application/site/throttle.py", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "6ed524a1c2b4006b7519e7f4035090202b113062e75d1871f898fbbd4eb0efed", + "ours": ["apps/server/src/modules/acquisition-organize/ports.ts", "packages/adapters/src/sites/rate-limit.ts"], + "source": "manual" + }, + { + "upstream": "app/application/torrent/download.py", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "11affe81a5af4d70d03be94a91b691fbb3f7a36a0b40c9fa4b34b6b29d9d2081", + "ours": ["packages/adapters/src/sites/http.ts"], + "source": "manual" + }, { "upstream": "app/chain/download/submission.py", "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", @@ -57,36 +71,36 @@ }, { "upstream": "app/domain/site.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "650c795a4def0a11317572c9210aa111c51e564973928d3af3fce8ce2da66107", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "5e6b8ebcce64d16fe536323a9e53924efb3e2b486cef06ea57ab33a809d0dcf3", "ours": ["packages/adapters/src/sites/user-parsers/common.ts"], "source": "tag" }, { "upstream": "app/foundation/dom.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "6d6accf2cef132e572aabb15da071ee355a3a6b02957fcb9c2e0c4082f863f4e", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "38b63b12ee03be90ae6fd9ecafc26636b615dde2c4f3ad5eb86ac55e06863ce6", "ours": ["packages/adapters/src/sites/user-parsers/dom.ts"], "source": "tag" }, { "upstream": "app/foundation/size.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "7b0b8f50588cbf9b827be2cfbe00044f493ea329730de8b61526cf159b5a1600", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "ee3289e2cbdd595823ff21615912aea6ce91f436c97c43628932871279167835", "ours": ["packages/adapters/src/sites/spider/spider.ts"], "source": "manual" }, { "upstream": "app/foundation/temporal.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "74941f460e4069f45cae85636ded26366f80f9869438b2ff0ca092ef5dddb892", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "237d9f55de13b2247e55bca987a06e7285f0cda03d7d314c2a78bf66e32c24e3", "ours": ["packages/adapters/src/sites/spiders/common.ts", "packages/adapters/src/sites/user-parsers/common.ts"], "source": "tag" }, { "upstream": "app/foundation/text.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "a88c75dc843131cb4a71e041b59386bea1defd332c77087978be854b56177961", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "8d8c5ed7aa81e36d4d20dc035a768bd176a41cc906ffa192eec928f820645487", "ours": ["packages/adapters/src/sites/user-parsers/common.ts"], "source": "tag" }, @@ -127,8 +141,8 @@ }, { "upstream": "app/modules/indexer/__init__.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "e9c5d15fdaa53205e15210d1f19019b9b82716affe075f6b268f6a044d676641", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "4366034bdc25375eadea7bb8e48ccc5d6f190f9c16c2041b55a88522f2dff51a", "ours": [ "packages/adapters/src/sites/registry.ts", "packages/adapters/src/sites/spider/adapter.ts", @@ -142,221 +156,230 @@ }, { "upstream": "app/modules/indexer/parser/__init__.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "0e6c4215b713493a4efdedfb7f42ff9418ec297adc2567b4f596ebaec8120ea3", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "c6d7e45058cf4dce2a5f9dd3a31e6a5f603a8d8e4a1c5f449df16eb54b3a5cff", "ours": ["packages/adapters/src/sites/user-parsers/base.ts", "packages/adapters/src/sites/user-parsers/types.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/bitpt.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "871d25e570f1fadcd8f214d8ae9f3b07653a053337a5e14a487a96c99a8da3a0", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "daccec01e01dee95a3fa88abe1dc06cd883f12e9ca9cece0a9f14d79c11403d9", "ours": ["packages/adapters/src/sites/user-parsers/bitpt.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/discuz.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "12277ba955ce7263c4b1862749443878c0900350ea8d5eb970bb57e1249c279d", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "f056cb1f36910872001a88706443782394034b6e3259a55255ffc581b558d0fc", "ours": ["packages/adapters/src/sites/user-parsers/discuz.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/file_list.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "f92e65ed73278c8e68fdd86927cb307e418564580daf8fded3e7a467d7bea7ac", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "1e101fe651f547416cf48dd127937a4e66504dab485d68580a3052773c2282dc", "ours": ["packages/adapters/src/sites/user-parsers/file-list.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/gazelle.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "c10e5246724b51391fff60f90e17022c0fd5fa2a4151ae69d6c4ee9ad1c5f1c6", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "37d6b2017708f1b3360aca413632d242b28a3b7606a0f3f43c04ed2cdb012db2", "ours": ["packages/adapters/src/sites/user-parsers/gazelle.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/hddolby.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "a4336d92d603c06953bb957d166f354d7a847eda75d9eaf2bcc96212f6e58c56", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "1a9f885333c39bbbe01ad1aa292a47a9f1c2b1805a17200ff33f85d1444867da", "ours": ["packages/adapters/src/sites/user-parsers/hddolby.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/ipt.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "2b9d04d76606329f32d7f726b5ae478997cdb7ee55f74ff89ab881c641e58bc0", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "fffccd341aceb64f35f5ee33ee46e668acde95b5121ac7b819ddc9525b3de017", "ours": ["packages/adapters/src/sites/user-parsers/ipt.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/mtorrent.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "c5d5a3c6f9f5fa5a60e3d65426bdc6620e94e861e0cf27f718595afc61b86165", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "1f35d41add01e120a6def3893eb158a99dee8994b083901cd9d2203d1d0206d0", "ours": ["packages/adapters/src/sites/user-parsers/mtorrent.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/nexus_audiences.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "1a2d104db7699c8dffbc79546a63dd40108e8fa171f1e7380e886521b0c3698f", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "7b53f3c76c5e5a9974ef8e79702cabc799d44ba3c7f95b37c51d70ef43d5d209", "ours": ["packages/adapters/src/sites/user-parsers/nexus-audiences.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/nexus_hhanclub.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "8850619069d4b9baaac2ef53fbdf5bac88b365b2e3009455ad40593c963c5ccb", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "320c0979c192755236639f54b131a70f33b3f3d43737fd35327710e73e2ab978", "ours": ["packages/adapters/src/sites/user-parsers/nexus-hhanclub.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/nexus_php.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "39303ae96d9cd198d0145e10539357ac17186be0127eaa92ada36ee3746296d2", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "8581be9deb8f21d412b1a2a52fd876da2f2995f7fd286a0b665d437032da9485", "ours": ["packages/adapters/src/sites/user-parsers/nexus-php.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/nexus_project.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "2b5508cfb2f89efa343515c5b91ae7af95fb9adf4624da6686c4f3b0004f07b0", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "5bf6662060dfb916deabd25bb64bce282904babd8c81d2ee878411449de306c6", "ours": ["packages/adapters/src/sites/user-parsers/nexus-project.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/nexus_rabbit.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "09149b89a7c76b9880d34deedc7335aba3051a81910d333df237614a0a55a98f", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "3d543474228229b72640323a071627230d2831cf39d757f88ea8b849978144c4", "ours": ["packages/adapters/src/sites/user-parsers/nexus-rabbit.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/rousi.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "1b1ea93de084c20b7001fc22994d87b4f99d5c04773ee25cb1482966fefca205", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "f1ee95dfef78332eb68c7367255678cc2490d8dcefeb371cfe199afde611a2e6", "ours": ["packages/adapters/src/sites/user-parsers/rousi.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/small_horse.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "b73b5de97737ca60e46bfecfc580bad6d9c7ee573d01bf6002f0282bcdf229ab", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "adb5efa1f74e404d06834d8c4a5502d8b62c5375cf9ddcec763e5b37f1742936", "ours": ["packages/adapters/src/sites/user-parsers/small-horse.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/sunnypt.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "937f8453fa2b73d06a690ff91e8939cd1478009b4b731f622d6585d9cdbc28cf", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "c5449afb49305e5a3e83df228cf39135e0b20d92d79607b39e5c77dafa830044", "ours": ["packages/adapters/src/sites/user-parsers/sunnypt.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/tnode.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "c9cb809fc4bffc161f2595cbb6db97b919c83d437e3bdc29374e46e3c46ba74e", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "6990614700450da17888e014e37ddab53f991f1bf38188739b4612654e2cb8cc", "ours": ["packages/adapters/src/sites/user-parsers/tnode.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/torrent_leech.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "72d1473a3bba5398d1a283995b646f7c93683992c4fa9f6a0d5eb80272754b0b", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "cf0debbfacd9d00366dda5737ac0453cf16f24b12d6ba4ab1ae788f64cea27a0", "ours": ["packages/adapters/src/sites/user-parsers/torrent-leech.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/unit3d.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "9a4d579deca593b445e80e560ed79c265e9b95cb631bef91e4dca859d020b912", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "cfa271284dcb0d62caf6422daae7d368308136da1410b5f36eedee008a2f083e", "ours": ["packages/adapters/src/sites/user-parsers/unit3d.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/yema.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "0f98fd580bad06edebaf81730deb5f11ad7c55ac5ae0d9ebc76ae50c5f828017", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "da2a09e2d8556d2f6a4a20eb7ece21fde4fd1f8febc88a5c42a8117be76d483f", "ours": ["packages/adapters/src/sites/user-parsers/yema.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/parser/zhixing.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "90c8954e73876d9437220bf3a26c8f863d6b4773b54d97ae07321ee0c1b1abc5", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "00de9540751f048761b696eecb9667dd360ba00dacd83359f72fb883074bbbef", "ours": ["packages/adapters/src/sites/user-parsers/zhixing.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/spider/__init__.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "2513d962fa437f9bf2ca06f5f908b7de8b9bb7b032f80eff283611af38a8b487", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "2456e822939b3b564179f92cc61c287774b93d10c2a29bba4c50bc228d248d9f", "ours": [ "packages/adapters/src/sites/spider/map.ts", "packages/adapters/src/sites/spider/schema.ts", "packages/adapters/src/sites/spider/spider.ts", "packages/adapters/src/sites/spider/template.ts" ], - "source": "manual" + "source": "manual+tag" }, { "upstream": "app/modules/indexer/spider/haidan.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "75eb802f19f02cf64ed2b7b981026e47369a2d815c6a666c5a9fd70aa86a598e", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "315f775c34132fed600e543f80dc8087b83cbd9a9a9406030a3bb4705303418b", "ours": ["packages/adapters/src/sites/spiders/haidan.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/spider/hddolby.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "46a50bd8d49f83a21f59b6273b0378d2342feb78e7f2f5b953eb9d1311c38804", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "f0f6e9467f4a27fc5df1ad923d04ca4233fb9bb3678cadf79a280c260c72da2d", "ours": ["packages/adapters/src/sites/spiders/hddolby.ts"], "source": "tag" }, + { + "upstream": "app/modules/indexer/spider/milkie.py", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "8e967235574f38a422375e5c878f3a6ec894e3caedc523313f18385348903af4", + "ours": ["packages/adapters/src/sites/spiders/milkie.ts"], + "source": "tag" + }, { "upstream": "app/modules/indexer/spider/mtorrent.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "a8ab2b822cb24a22c16a8db01e0ef68985435a25b5332d741992f9ff99c10de5", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "6a07b5d161e91a90bf493b0be33030f685bfbba2bb4f33fea82a01b4ddb6469d", "ours": [ "packages/adapters/src/sites/mteam/adapter.ts", "packages/adapters/src/sites/mteam/client.ts", + "packages/adapters/src/sites/mteam/definition.ts", "packages/adapters/src/sites/mteam/spider.ts", + "packages/adapters/src/sites/mteam/subtitles.ts", "packages/adapters/src/sites/spiders/mtorrent.ts" ], "source": "manual+tag" }, { "upstream": "app/modules/indexer/spider/rousi.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "aa13c386a560a97125ac3879b56187c50564d544e925ad9f221bbbefc15e712a", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "20f046e4be62aee9ba26daba6aedefef0b6636b57ae419ceceb6366a5ac07aa0", "ours": ["packages/adapters/src/sites/spiders/rousi.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/spider/sunnypt.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "735b0d721e65ec4006d1ea309b1815635caa7e69e1897643faaa6ca8b84897b8", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "3b8c98086932211bc316ad466487a0d07780f0133c72a1eb0d4f0670ef574183", "ours": ["packages/adapters/src/sites/spiders/sunnypt.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/spider/tnode.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "a3634a268dbad6cfe45a5c70606b94a4c2fe8f9b351d9b7d6c6499f4a8107fe3", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "fe0641b3c6bc9f1f75262b232ff6bb5d133914f22f596f097f65e1bc9592f31e", "ours": ["packages/adapters/src/sites/spiders/tnode.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/spider/torrentleech.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "c965f2bcaea4a50335bcb6e670a9ddb3a96a3f62f59bf3f93deff9cd9e7e28e5", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "b6c794453473473f2121689c2f34c5e755f584b94153554f03acb411d645dffb", "ours": ["packages/adapters/src/sites/spiders/torrentleech.ts"], "source": "tag" }, { "upstream": "app/modules/indexer/spider/yema.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "560b62c1ff102d3efa47ad9dc31ba1a5794eae804fc320dbb06f08b72a004f3f", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "4ed0234b920167aeec45854f5334e5ca7bc34e202d13177660de9d478635b9f7", "ours": ["packages/adapters/src/sites/spiders/yema.ts"], "source": "tag" }, @@ -376,8 +399,8 @@ }, { "upstream": "app/schemas/site.py", - "commit": "e6820a83ab05e8a00eb0e14adf0f357faa965c7e", - "sha256": "e7952ed40b7b5a66d95fe1b07e1d3915601b47c2936ff10ec6880c8a030d0b8f", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "sha256": "78739bcb7d798fcc0c2e16e515814b4f0b0d27f94a2d84ba07c3a8ce93ee7ca9", "ours": ["packages/adapters/src/sites/user-parsers/types.ts"], "source": "tag" }, diff --git a/scripts/mp-sync/ledger.manual.json b/scripts/mp-sync/ledger.manual.json index 5de52206..a22ebdf4 100644 --- a/scripts/mp-sync/ledger.manual.json +++ b/scripts/mp-sync/ledger.manual.json @@ -8,7 +8,8 @@ "packages/adapters/src/sites/spider/schema.ts", "packages/adapters/src/sites/spider/map.ts", "packages/adapters/src/sites/spider/template.ts" - ] + ], + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57" }, { "upstream": "app/modules/indexer/spider/mtorrent.py", @@ -16,11 +17,13 @@ "packages/adapters/src/sites/mteam/spider.ts", "packages/adapters/src/sites/mteam/adapter.ts", "packages/adapters/src/sites/mteam/client.ts" - ] + ], + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57" }, { "upstream": "app/modules/indexer/__init__.py", - "ours": ["packages/adapters/src/sites/spider/adapter.ts", "packages/adapters/src/sites/registry.ts"] + "ours": ["packages/adapters/src/sites/spider/adapter.ts", "packages/adapters/src/sites/registry.ts"], + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57" }, { "upstream": "app/chain/download/submission.py", @@ -32,9 +35,14 @@ "packages/adapters/src/sites/spider/schema.ts", "packages/adapters/src/sites/definitions/schema.ts", "packages/adapters/src/sites/registry.ts" - ] + ], + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57" + }, + { + "upstream": "app/foundation/size.py", + "ours": ["packages/adapters/src/sites/spider/spider.ts"], + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57" }, - { "upstream": "app/foundation/size.py", "ours": ["packages/adapters/src/sites/spider/spider.ts"] }, { "upstream": "app/application/rss.py", "ours": [ @@ -47,8 +55,14 @@ "upstream": "app/modules/qbittorrent/qbittorrent.py", "ours": ["packages/adapters/src/qbittorrent/client.ts", "packages/adapters/src/qbittorrent/state.ts"] }, - { "upstream": "app/modules/_base/downloader.py", "ours": ["packages/adapters/src/qbittorrent/torrent.ts"] }, - { "upstream": "app/modules/__init__.py", "ours": ["packages/adapters/src/qbittorrent/paths.ts"] }, + { + "upstream": "app/modules/_base/downloader.py", + "ours": ["packages/adapters/src/qbittorrent/torrent.ts"] + }, + { + "upstream": "app/modules/__init__.py", + "ours": ["packages/adapters/src/qbittorrent/paths.ts"] + }, { "upstream": "app/modules/themoviedb/tmdbapi.py", "ours": ["packages/adapters/src/tmdb/client.ts", "packages/adapters/src/tmdb/match.ts"] @@ -57,9 +71,18 @@ "upstream": "app/modules/filemanager/storages/local.py", "ours": ["packages/adapters/src/fs/copy.ts", "packages/adapters/src/fs/move.ts"] }, - { "upstream": "app/modules/filemanager/transhandler.py", "ours": ["packages/adapters/src/fs/move.ts"] }, - { "upstream": "app/modules/emby/emby.py", "ours": ["packages/adapters/src/emby/client.ts"] }, - { "upstream": "app/application/mediaserver.py", "ours": ["packages/adapters/src/emby/client.ts"] }, + { + "upstream": "app/modules/filemanager/transhandler.py", + "ours": ["packages/adapters/src/fs/move.ts"] + }, + { + "upstream": "app/modules/emby/emby.py", + "ours": ["packages/adapters/src/emby/client.ts"] + }, + { + "upstream": "app/application/mediaserver.py", + "ours": ["packages/adapters/src/emby/client.ts"] + }, { "upstream": "tests/cases/meta.py", "ours": ["packages/domain/test/differential/meta-samples.json", "packages/domain/test/differential/meta-mp.json"], @@ -69,6 +92,18 @@ "upstream": "tests/test_rust_accel.py", "ours": ["scripts/mp-sync/gen-differential.py", "packages/adapters/test/differential/indexer.json"], "note": "字幕页 HTML 与字幕字段配置取自 test_rust_subtitle_parser_is_several_times_faster_than_python,作 hhanclub-subtitles@inline 差分样本" + }, + { + "upstream": "app/application/site/throttle.py", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "ours": ["packages/adapters/src/sites/rate-limit.ts", "apps/server/src/modules/acquisition-organize/ports.ts"], + "note": "共享站点额度按既有PG桶实现,不复制上游进程内流控结构" + }, + { + "upstream": "app/application/torrent/download.py", + "commit": "3b156a545f14fea0285cd3ed3ea4aa49d97faa57", + "ours": ["packages/adapters/src/sites/http.ts"], + "note": "仅对照远端频控原因与冷却规则;下载编排由本项目任务服务维护" } ] } diff --git a/scripts/mp-sync/lib.mjs b/scripts/mp-sync/lib.mjs index bac6d2f7..c03e0fd6 100644 --- a/scripts/mp-sync/lib.mjs +++ b/scripts/mp-sync/lib.mjs @@ -1,4 +1,4 @@ -// mp-sync 公共函数(Deno 2 运行,无第三方依赖) +// mp-sync 公共函数(Bun 运行,无第三方依赖) import { execFileSync } from "node:child_process"; import { createHash } from "node:crypto"; import { existsSync, readdirSync, readFileSync, statSync } from "node:fs";