120 lines
13 KiB
Markdown
120 lines
13 KiB
Markdown
# MoviePilot 跟随移植账本(mp-sync)
|
|
|
|
> **许可结论(引自 `license-and-provenance.md`)**:MoviePilot 为 GPLv3;本目录登记的站点框架文件是用户 2026-09-27 明确要求"照抄 MP v3 实现迁移"的数据来源适配部分,每个移植文件头保留 `SPDX-License-Identifier: GPL-3.0-only` 与来源声明;MP 的闭源/加密部分(`sites.*.so`、`user.sites.v3.bin`)**不解密、不参考**,站点定义改由 MovieClaw **v0.27.0(MIT,`copied-mit`,见 `packages/adapters/src/sites/definitions/NOTICE-MovieClaw.md`)** 转换 + 手写覆盖。MovieClaw v0.27.0 之后改为附条件许可,**不跟随其后续提交**。
|
|
|
|
## 1. 机制
|
|
|
|
- **登记来源**:每个移植文件头写 `@mp-source <上游路径> @ <提交>`(一个文件可多行);sites 目录外或 W1-3a/b 早期文件在 `scripts/mp-sync/ledger.manual.json` 人工登记。
|
|
- **ledger**:`scripts/mp-sync/ledger.json`(`build-ledger.mjs` 生成)按上游文件聚合:`upstream`、移植时 `commit`、该文件在该提交的内容 `sha256`、本项目文件 `ours[]`。
|
|
- **监视目录**:`app/modules/indexer/spider/`、`app/modules/indexer/parser/`,出现 ledger 未收录的新文件即提示(MP 新增站点 spider/解析器)。
|
|
|
|
## 2. 用法
|
|
|
|
MP 本地仓库默认 `/opt/moviepilot-plus/upstream/MoviePilot`(早期浅克隆,`e6820a8`)。用仓库锁定的 Bun 运行脚本;核查其他 checkout 时显式传 `--mp <路径>`,每个条目的已同步提交由下表单独记录。
|
|
|
|
```bash
|
|
# 1) 取上游新提交(浅克隆)
|
|
git -C /opt/moviepilot-plus/upstream/MoviePilot fetch --depth=1 origin <新提交或 v3>
|
|
# 2) 查差异:已变更 / 已删除 / 监视目录新增,附逐文件 git diff 命令;有变化退出码 1
|
|
bun scripts/mp-sync/check.mjs --to <新提交> # 或 --mp <路径> --json
|
|
# 3) 按 diff 增量移植 TS 文件,把文件头 @mp-source 的提交号改为新提交(新文件加标签)
|
|
# 4) 重建 ledger 与下表
|
|
bun scripts/mp-sync/build-ledger.mjs --commit <新提交>
|
|
# 5) 跑适配器测试
|
|
cd packages/adapters && bun x --bun vitest run src/sites
|
|
```
|
|
|
|
MovieClaw 站点定义(与 MP 无关,但同属站点框架跟随工具):
|
|
|
|
```bash
|
|
bun scripts/mp-sync/movieclaw-import.mjs # 从 v0.27.0 重新转换 23 个站点定义(覆盖层:scripts/mp-sync/movieclaw-overrides/)
|
|
bun scripts/mp-sync/movieclaw-import.mjs --check # 只比较,有差异退出码 1
|
|
```
|
|
|
|
## 3. 当前对照总表
|
|
|
|
2026-11-06 核对 GitHub 最新稳定发布 [v3.2.4](https://github.com/jxxghp/MoviePilot/releases/tag/v3.2.4),固定提交 `3b156a545f14fea0285cd3ed3ea4aa49d97faa57`。本轮同步站点索引、用户解析与共享频控涉及的 42 个来源文件;其余 12 个媒体服务器、RSS、下载编排、文件管理、TMDB 与通用语料条目保留原基线,不据此宣称全应用已追平。
|
|
|
|
| 范围 | 本次同步 |
|
|
| --- | --- |
|
|
| 资源搜索 | 新增 Milkie;M-Team、HDDolby、Yema 分类透传与 API 地址覆盖;TNode 会话隔离;通用 XML/RSS、空 Feed 与无服务端分页的关键词窗口 |
|
|
| 用户数据 | NexusPHP UUID、专用做种入口与流量回填;Audiences 顶栏用户识别;Discuz 分享率、Unit3D 用户名 ID 与积分入口;M-Team 等级名、空做种页停止;HDDolby API 前缀及受限 schema 回退 |
|
|
| 公共请求 | 复用既有 `site:<数据库 ID>` 的 PG 桶;种子下载使用同一出口、账号 UA 和有效凭据;远端频控保留原因与冷却,错误页面标题脱敏 |
|
|
| M-Team 字幕 | API 搜索、按种子列字幕和下载接入现有补字幕流程;来源枚举与迁移 `0096`,解析、写入、撤销沿用现有服务 |
|
|
|
|
保留项目边界:站点定义继续来自已核实的 MovieClaw MIT 快照与手写定义,不使用 MP 加密资源;MoviePilot 的采集器迁到 [MoviePilot-Site-Collector](https://github.com/jxxghp/MoviePilot-Site-Collector),该工具的迁移不代表站点配置变为可直接导入。Milkie 默认未启用。API Key、Cookie 和下载签名不进入搜索结果或历史;M-Team 字幕搜索只返回稳定 ID,下载时现场换签。无 Cookie 的签名下载不复用浏览器 profile,CDN 跳转也不转发账号凭据。
|
|
|
|
Audiences 的 compact 指标使用局部 XPath,比上游仍可能回到整文档的绝对 XPath 更严格,避免列表卡片污染当前用户数据。分页上限、ID 字符串化、缺失数值归一及确定性绝对日期解析延续既有合同。Python 类型整理、httpx 连接池恢复和 dateparser 语言加载属于上游实现细节,不另造对应 TS 机制。
|
|
|
|
已用 21 个自编 HTML/JSON 案例动态运行固定上游解析器做差分,按合同归一后 21/21 一致;自动化站点与字幕回归使用模拟请求及隔离数据库。真实 PT 账号、站点挑战和端到端下载仍需各自现场验收,未计入这些自动化结论。
|
|
|
|
<!-- mp-sync:table:begin -->
|
|
共 54 个上游文件(`adapters/` = `packages/adapters/src/`)。
|
|
|
|
| 上游文件 | 本项目文件 | 移植提交 | sha256(前 12 位) | 来源 |
|
|
| --- | --- | --- | --- | --- |
|
|
| `app/adapters/network/cloudflare.py` | `adapters/sites/user-parsers/common.ts` | `3b156a5` | `c50274ad9e69` | tag |
|
|
| `app/application/mediaserver.py` | `adapters/emby/client.ts` | `e6820a8` | `f034f0be8ba6` | manual |
|
|
| `app/application/rss.py` | `adapters/rss/fetch.ts`<br>`adapters/rss/parse.ts`<br>`adapters/rss/rss-link.ts` | `e6820a8` | `8ea70f712512` | manual |
|
|
| `app/application/site/sites.pyi` | `adapters/sites/definitions/schema.ts`<br>`adapters/sites/registry.ts`<br>`adapters/sites/spider/schema.ts` | `3b156a5` | `b3f0ded80227` | manual |
|
|
| `app/application/site/throttle.py` | `apps/server/src/modules/acquisition-organize/ports.ts`<br>`adapters/sites/rate-limit.ts` | `3b156a5` | `6ed524a1c2b4` | manual |
|
|
| `app/application/torrent/download.py` | `adapters/sites/http.ts` | `3b156a5` | `11affe81a5af` | manual |
|
|
| `app/chain/download/submission.py` | `adapters/sites/mteam/adapter.ts`<br>`adapters/sites/spider/adapter.ts`<br>`adapters/sites/spiders/common.ts` | `e6820a8` | `f7cb870e47d6` | manual+tag |
|
|
| `app/domain/site.py` | `adapters/sites/user-parsers/common.ts` | `3b156a5` | `5e6b8ebcce64` | tag |
|
|
| `app/foundation/dom.py` | `adapters/sites/user-parsers/dom.ts` | `3b156a5` | `38b63b12ee03` | tag |
|
|
| `app/foundation/size.py` | `adapters/sites/spider/spider.ts` | `3b156a5` | `ee3289e2cbdd` | manual |
|
|
| `app/foundation/temporal.py` | `adapters/sites/spiders/common.ts`<br>`adapters/sites/user-parsers/common.ts` | `3b156a5` | `237d9f55de13` | tag |
|
|
| `app/foundation/text.py` | `adapters/sites/user-parsers/common.ts` | `3b156a5` | `8d8c5ed7aa81` | tag |
|
|
| `app/modules/__init__.py` | `adapters/qbittorrent/paths.ts` | `e6820a8` | `2edada0ea591` | manual |
|
|
| `app/modules/_base/downloader.py` | `adapters/qbittorrent/torrent.ts` | `e6820a8` | `84bd8946c8b2` | manual |
|
|
| `app/modules/emby/emby.py` | `adapters/emby/client.ts` | `e6820a8` | `e6468ea34ef2` | manual |
|
|
| `app/modules/filemanager/storages/local.py` | `adapters/fs/copy.ts`<br>`adapters/fs/move.ts` | `e6820a8` | `806a1970ca8e` | manual |
|
|
| `app/modules/filemanager/transhandler.py` | `adapters/fs/move.ts` | `e6820a8` | `13e2bba491ac` | manual |
|
|
| `app/modules/indexer/__init__.py` | `adapters/sites/registry.ts`<br>`adapters/sites/spider/adapter.ts`<br>`adapters/sites/spiders/adapter.ts`<br>`adapters/sites/spiders/index.ts`<br>`adapters/sites/spiders/map.ts`<br>`adapters/sites/spiders/types.ts`<br>`adapters/sites/user-parsers/registry.ts` | `3b156a5` | `4366034bdc25` | manual+tag |
|
|
| `app/modules/indexer/parser/__init__.py` | `adapters/sites/user-parsers/base.ts`<br>`adapters/sites/user-parsers/types.ts` | `3b156a5` | `c6d7e45058cf` | tag |
|
|
| `app/modules/indexer/parser/bitpt.py` | `adapters/sites/user-parsers/bitpt.ts` | `3b156a5` | `daccec01e01d` | tag |
|
|
| `app/modules/indexer/parser/discuz.py` | `adapters/sites/user-parsers/discuz.ts` | `3b156a5` | `f056cb1f3691` | tag |
|
|
| `app/modules/indexer/parser/file_list.py` | `adapters/sites/user-parsers/file-list.ts` | `3b156a5` | `1e101fe651f5` | tag |
|
|
| `app/modules/indexer/parser/gazelle.py` | `adapters/sites/user-parsers/gazelle.ts` | `3b156a5` | `37d6b2017708` | tag |
|
|
| `app/modules/indexer/parser/hddolby.py` | `adapters/sites/user-parsers/hddolby.ts` | `3b156a5` | `1a9f885333c3` | tag |
|
|
| `app/modules/indexer/parser/ipt.py` | `adapters/sites/user-parsers/ipt.ts` | `3b156a5` | `fffccd341ace` | tag |
|
|
| `app/modules/indexer/parser/mtorrent.py` | `adapters/sites/user-parsers/mtorrent.ts` | `3b156a5` | `1f35d41add01` | tag |
|
|
| `app/modules/indexer/parser/nexus_audiences.py` | `adapters/sites/user-parsers/nexus-audiences.ts` | `3b156a5` | `7b53f3c76c5e` | tag |
|
|
| `app/modules/indexer/parser/nexus_hhanclub.py` | `adapters/sites/user-parsers/nexus-hhanclub.ts` | `3b156a5` | `320c0979c192` | tag |
|
|
| `app/modules/indexer/parser/nexus_php.py` | `adapters/sites/user-parsers/nexus-php.ts` | `3b156a5` | `8581be9deb8f` | tag |
|
|
| `app/modules/indexer/parser/nexus_project.py` | `adapters/sites/user-parsers/nexus-project.ts` | `3b156a5` | `5bf6662060df` | tag |
|
|
| `app/modules/indexer/parser/nexus_rabbit.py` | `adapters/sites/user-parsers/nexus-rabbit.ts` | `3b156a5` | `3d5434742282` | tag |
|
|
| `app/modules/indexer/parser/rousi.py` | `adapters/sites/user-parsers/rousi.ts` | `3b156a5` | `f1ee95dfef78` | tag |
|
|
| `app/modules/indexer/parser/small_horse.py` | `adapters/sites/user-parsers/small-horse.ts` | `3b156a5` | `adb5efa1f74e` | tag |
|
|
| `app/modules/indexer/parser/sunnypt.py` | `adapters/sites/user-parsers/sunnypt.ts` | `3b156a5` | `c5449afb4930` | tag |
|
|
| `app/modules/indexer/parser/tnode.py` | `adapters/sites/user-parsers/tnode.ts` | `3b156a5` | `699061470045` | tag |
|
|
| `app/modules/indexer/parser/torrent_leech.py` | `adapters/sites/user-parsers/torrent-leech.ts` | `3b156a5` | `cf0debbfacd9` | tag |
|
|
| `app/modules/indexer/parser/unit3d.py` | `adapters/sites/user-parsers/unit3d.ts` | `3b156a5` | `cfa271284dcb` | tag |
|
|
| `app/modules/indexer/parser/yema.py` | `adapters/sites/user-parsers/yema.ts` | `3b156a5` | `da2a09e2d855` | tag |
|
|
| `app/modules/indexer/parser/zhixing.py` | `adapters/sites/user-parsers/zhixing.ts` | `3b156a5` | `00de9540751f` | tag |
|
|
| `app/modules/indexer/spider/__init__.py` | `adapters/sites/spider/map.ts`<br>`adapters/sites/spider/schema.ts`<br>`adapters/sites/spider/spider.ts`<br>`adapters/sites/spider/template.ts` | `3b156a5` | `2456e822939b` | manual+tag |
|
|
| `app/modules/indexer/spider/haidan.py` | `adapters/sites/spiders/haidan.ts` | `3b156a5` | `315f775c3413` | tag |
|
|
| `app/modules/indexer/spider/hddolby.py` | `adapters/sites/spiders/hddolby.ts` | `3b156a5` | `f0f6e9467f4a` | tag |
|
|
| `app/modules/indexer/spider/milkie.py` | `adapters/sites/spiders/milkie.ts` | `3b156a5` | `8e967235574f` | tag |
|
|
| `app/modules/indexer/spider/mtorrent.py` | `adapters/sites/mteam/adapter.ts`<br>`adapters/sites/mteam/client.ts`<br>`adapters/sites/mteam/definition.ts`<br>`adapters/sites/mteam/spider.ts`<br>`adapters/sites/mteam/subtitles.ts`<br>`adapters/sites/spiders/mtorrent.ts` | `3b156a5` | `6a07b5d161e9` | manual+tag |
|
|
| `app/modules/indexer/spider/rousi.py` | `adapters/sites/spiders/rousi.ts` | `3b156a5` | `20f046e4be62` | tag |
|
|
| `app/modules/indexer/spider/sunnypt.py` | `adapters/sites/spiders/sunnypt.ts` | `3b156a5` | `3b8c98086932` | tag |
|
|
| `app/modules/indexer/spider/tnode.py` | `adapters/sites/spiders/tnode.ts` | `3b156a5` | `fe0641b3c6bc` | tag |
|
|
| `app/modules/indexer/spider/torrentleech.py` | `adapters/sites/spiders/torrentleech.ts` | `3b156a5` | `b6c794453473` | tag |
|
|
| `app/modules/indexer/spider/yema.py` | `adapters/sites/spiders/yema.ts` | `3b156a5` | `4ed0234b9201` | tag |
|
|
| `app/modules/qbittorrent/qbittorrent.py` | `adapters/qbittorrent/client.ts`<br>`adapters/qbittorrent/state.ts` | `e6820a8` | `ab02e97e986f` | manual |
|
|
| `app/modules/themoviedb/tmdbapi.py` | `adapters/tmdb/client.ts`<br>`adapters/tmdb/match.ts` | `e6820a8` | `787553a6f57d` | manual |
|
|
| `app/schemas/site.py` | `adapters/sites/user-parsers/types.ts` | `3b156a5` | `78739bcb7d79` | tag |
|
|
| `tests/cases/meta.py` | `packages/domain/test/differential/meta-mp.json`<br>`packages/domain/test/differential/meta-samples.json` | `e6820a8` | `3af375cf2c43` | manual |
|
|
| `tests/test_rust_accel.py` | `packages/adapters/test/differential/indexer.json`<br>`scripts/mp-sync/gen-differential.py` | `e6820a8` | `0259f245d278` | manual |
|
|
<!-- mp-sync:table:end -->
|
|
|
|
## 4. 移植时的统一偏离(每个文件头另有细节)
|
|
|
|
1. **凭据不进候选/结果**:MP 两段式 enclosure(`[base64(含 apikey 的请求配置)]url`)不生成,下载时由 adapter 现场换直链;用户信息 `raw` 不含 cookie/apikey/token。
|
|
2. **错误显式化**:MP spider 的 `(is_error, [])` 改为抛 `SiteError{kind}`;用户信息解析保留 MP 的 `err_msg` 语义,另记脱敏的 `http_errors`。
|
|
3. **翻页上限**:做种/消息翻页最多 50 页(MP 无上限)。
|
|
4. **XPath**:lxml → `xpath` + `@xmldom/xmldom`(经 htmlparser2 解析,不插入 tbody),见 `user-parsers/dom.ts`。
|
|
5. **dateparser**:只实现绝对时间格式,相对时间原样返回(待核验)。
|
|
6. **写操作门控**:读取站内信后标记已读(MTorrent/Rousi/NexusRabbit)只在 `fetchMessages=true` 时发生(MP 由全局 `SITE_MESSAGE` 控制)。
|