feat(adapters): port MoviePilot open site framework (7 spiders, 20 user parsers) + MovieClaw v0.27.0 site definitions + mp-sync tools (W1-3c)
- sites/spiders: Haidan/HDDolby/RousiPro/SunnyPT/TNode/TorrentLeech/Yema (+mTorrent wrapper), specialized SiteAdapter, indirect download resolved at download time - sites/user-parsers: SiteParserBase + all 20 MP SiteSchema parsers (lxml XPath via xpath+xmldom), refreshSiteUserData with schema fallback - sites/definitions: zod SiteDefinition schemas, 23 MovieClaw v0.27.0 (MIT) definitions via scripts/mp-sync/movieclaw-import.mjs, registry getSiteDefinition/listSiteDefinitions, NOTICE - scripts/mp-sync: ledger.json (upstream sha256), check.mjs, build-ledger.mjs; docs/porting/mp-sync-ledger.md - license-and-provenance: add copied-mit kind, re-anchor MovieClaw rows to v0.27.0 Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
rootandClaude Opus 5.5 committed
1 parent
cf85481d95
commit
d9f2af2f3c
166 files changed
+16573
-182
No files matched your search
@@ -40,8 +40,9 @@
|
||||
| 引用事实性常量：协议字段名（qB WebUI API、M-Team API、TMDB API）、站点公开的 HTML 结构、行业通用词表（`WEB-DL`、`REMUX`、`DDP5.1`） | 照搬上游**整理过的数据表**：`RELEASE_GROUPS` 站点到组名的正则表（`app/domain/meta/releasegroup.py:26`）、`StreamingPlatforms` 平台码表（`app/domain/meta/streamingplatform.py`，约 270 条）、`_name_nostring_re` 干扰词表。这类表要**自行编制**，并在 ledger 里写明来源（例如公开的 scene/P2P 命名规范、站点官方规则页） |
|
||||
| 上游测试里的**标题字符串**（事实数据）可以作为输入语料 | 整份拷贝 `tests/cases/meta.py` 这类测试文件；期望值必须由我们自己按 REQ 判定，不是照抄上游的 target |
|
||||
| MovieClaw：阅读 YAML 的**字段结构**，参考其字段设计 | 在没有书面授权时拷贝 MovieClaw main 的配置或代码（附条件许可）；把 main 的内容当作 MIT 使用 |
|
||||
| MovieClaw **tag `v0.27.0`（`c79bb1b0`，MIT）** 的资产可以原样复制或转换（`kind=copied-mit`），前提是保留 MIT 全文与 `Copyright (c) 2026 MovieClaw`（`packages/adapters/src/sites/definitions/NOTICE-MovieClaw.md`），每个产物写明来源提交与路径（站点定义 JSON 的 `provenance` 字段），且只经 `git show v0.27.0:<path>` 读取（用户 2026-09-27 确认，依据 `movieclaw-assessment.md` §1） | 复制 `v0.27.0` 之后（`251f1988` 起附条件许可）的任何内容；复制 MC 仓库内自带独立许可的第三方子组件（`skill-creator` Apache-2.0、`liquid-glass`）而不遵守其各自许可 |
|
||||
|
||||
**对每个 PR 的要求**：只要实现参考了上游，PR 描述里就必须写"参考：`docs/porting/<文件>#<节>`"，并在 §3 ledger 里补一行，登记状态为 `behavior-ref`。只有真的复制了代码或数据时，才登记为 `copied`；按目前的边界，**不应出现 `copied`**。
|
||||
**对每个 PR 的要求**：只要实现参考了上游，PR 描述里就必须写"参考：`docs/porting/<文件>#<节>`"，并在 §3 ledger 里补一行，登记状态为 `behavior-ref`。只有真的复制了代码或数据时，才登记为 `copied`；按目前的边界，**不应出现 `copied`**。例外是 `copied-mit`：复制 MIT 许可的资产（目前仅 MovieClaw v0.27.0），必须 `notice_required=是` 并附 NOTICE。
|
||||
|
||||
### 2.1 分发时的注意事项（首期不分发，先写在这里）
|
||||
|
||||
@@ -59,7 +60,7 @@
|
||||
| --- | --- |
|
||||
| `id` | `CL-<序号>` |
|
||||
| `our_module` | 本项目的文件或函数，例如 `packages/domain/src/title/parse.ts#parseReleaseTitle` |
|
||||
| `kind` | `behavior-ref`（只参考行为）/ `data-ref`（参考了数据结构，内容自行编制）/ `copied`（复制，原则上禁止）/ `dependency`（npm 依赖） |
|
||||
| `kind` | `behavior-ref`（只参考行为）/ `data-ref`（参考了数据结构，内容自行编制）/ `copied`（复制，原则上禁止）/ `copied-mit`（复制 MIT 资产，须保留 NOTICE，见 §2）/ `dependency`（npm 依赖） |
|
||||
| `upstream` | 仓库 URL |
|
||||
| `commit_or_version` | 精确提交或 tag，不能写 `main` |
|
||||
| `upstream_paths` | 文件:函数:行号 |
|
||||
@@ -82,9 +83,11 @@
|
||||
| CL-006 | `packages/domain/src/organize/*`、`packages/adapters/src/fs/*`、`packages/tasks/src/verify-and-import.ts` | behavior-ref | MoviePilot | e6820a8 | 见 `moviepilot-transfer.md` §2 各表 | GPL-3.0 | transfer §2-§4 | 见 transfer §3 的必须偏离表 | 否 | internal-only |
|
||||
| CL-007 | `packages/adapters/src/qbittorrent/*` | behavior-ref | MoviePilot | e6820a8 | `app/modules/qbittorrent/__init__.py`、`qbittorrent.py`（行号见 downloader-and-sites 文档） | GPL-3.0 | downloader-and-sites §qB | 标签接管，删数据前做引用检查 | 否 | internal-only |
|
||||
| CL-008 | `packages/adapters/src/sites/nexusphp/*` | behavior-ref | MoviePilot | e6820a8 | `app/modules/indexer/spider/*`、`app/modules/indexer/parser/nexus_php.py`、`nexus_audiences.py` | GPL-3.0 | downloader-and-sites §站点 | 站点定义自行编制；不使用闭源 SitesHelper | 否 | internal-only |
|
||||
| CL-009 | `packages/adapters/src/sites/nexusphp/configs/audiences.ts` | data-ref | MovieClaw | fc9eb72 | `src/movieclaw_tracker/sites/configs/audiences.yaml` | MovieClaw License（Apache-2.0 加非商业条件） | downloader-and-sites §MovieClaw | **只参考字段结构**；选择器值以我们自己抓取的观众页面为准重新编写 | 否（未复制） | internal-only |
|
||||
| CL-010 | `packages/adapters/src/sites/mteam/*` | behavior-ref + data-ref | MovieClaw + MoviePilot | fc9eb72 / e6820a8 | MC：`sites/configs/mteam.yaml`、`sites/custom/mteam.py`；MP：`app/modules/indexer/parser/mtorrent.py` | MC 附条件 / GPL-3.0 | downloader-and-sites §M-Team | API 字段以 M-Team 公开 API 为准，按 API Key 模式实现 | 否 | internal-only |
|
||||
| CL-009（retired → CL-012） | `packages/adapters/src/sites/nexusphp/configs/audiences.ts` | data-ref | MovieClaw | fc9eb72 | `src/movieclaw_tracker/sites/configs/audiences.yaml` | MovieClaw License（Apache-2.0 加非商业条件） | downloader-and-sites §MovieClaw | **只参考字段结构**；选择器值以我们自己抓取的观众页面为准重新编写 | 否（未复制） | internal-only |
|
||||
| CL-010（retired → CL-013） | `packages/adapters/src/sites/mteam/*` | behavior-ref + data-ref | MovieClaw + MoviePilot | fc9eb72 / e6820a8 | MC：`sites/configs/mteam.yaml`、`sites/custom/mteam.py`；MP：`app/modules/indexer/parser/mtorrent.py` | MC 附条件 / GPL-3.0 | downloader-and-sites §M-Team | API 字段以 M-Team 公开 API 为准，按 API Key 模式实现 | 否 | internal-only |
|
||||
| CL-011 | `packages/testkit/fixtures/title-parity.json` | data-ref | MoviePilot | e6820a8 | 输入语料参考 `tests/cases/meta.py`；**期望值自行判定** | GPL-3.0（原测试文件） | metadata-recognition §9 | 只取标题字符串，必要时脱敏 | 否 | internal-only |
|
||||
| CL-012 | `packages/adapters/src/sites/definitions/*.json`（23 站）、`definitions/manifest.generated.ts`、`spider/mc-selector.ts`、`scripts/mp-sync/movieclaw-import.mjs` | copied-mit | MovieClaw | **v0.27.0**（`c79bb1b0`） | `src/movieclaw_tracker/sites/configs/*.yaml`；`selectors.py` `NexusPHPSelectors` 默认值；`frameworks/nexusphp.py` `_css_text` | MIT | downloader-and-sites §W1-3c | YAML 转为 MP indexer schema + `umt`；观众/馒头/HHClub/CHD 由 `scripts/mp-sync/movieclaw-overrides/*.json` 覆盖；除 4 主力站外 `enabled:false` | 是（`definitions/NOTICE-MovieClaw.md`） | internal-only |
|
||||
| CL-013 | `packages/adapters/src/sites/mteam/*` | behavior-ref + copied-mit | MovieClaw + MoviePilot | **v0.27.0**（`c79bb1b0`）/ e6820a8 | MC：`sites/configs/mteam.yaml`（→ `definitions/mteam.json`）、`sites/custom/mteam.py`（字段对照）；MP：`app/modules/indexer/spider/mtorrent.py`、`parser/mtorrent.py` | MIT / GPL-3.0 | downloader-and-sites §M-Team | 定义由 MC 转换 + W1-3b 值覆盖；API 行为按 MP | 是（MC 部分） | internal-only |
|
||||
|
||||
> 新增 npm 依赖（例如 anitopy 的等价实现、`re2`、`fast-xml-parser`）按 `kind=dependency` 登记，填锁定版本和 SPDX。
|
||||
|
||||
@@ -92,7 +95,7 @@
|
||||
|
||||
1. ledger 以本文件 §3.2 为唯一登记处，按 id 追加，不删除旧行；撤销时把行标记为 `retired`。
|
||||
2. 上游提交升级时，**新增一行**，写新的提交号，不覆盖旧行。对照文档里的行号也要按新提交重新核对。
|
||||
3. `copied` 行出现时，CI 必须失败。建议在 `scripts/` 里加一个检查：ledger 中 `kind=copied` 的行数必须为 0。
|
||||
3. `copied` 行出现时，CI 必须失败。建议在 `scripts/` 里加一个检查：ledger 中 `kind=copied` 的行数必须为 0。`copied-mit` 不在此列，但必须有 NOTICE 文件且 `notice=是`。
|
||||
4. 审核人检查三件事：PR 里有没有"参考："链接；ledger 有没有对应行；diff 中有没有大段和上游高度相似的代码（可以用 `jscpd` 这类工具，把上游目录作为外部语料做一次抽样检查）。
|
||||
|
||||
---
|
||||
|
||||
@@ -662,3 +662,36 @@ MC 注册表 `MC:src/movieclaw_tracker/registry.py:172` `_load_site_yaml` 读取
|
||||
### 10.4 许可提示
|
||||
|
||||
本节模块是对 MP(GPL-3.0)实现的逐函数 TS 移植,性质上比 `behavior-ref` 更接近衍生作品。项目目前为 internal-only 使用;若将来对外分发,需按 `license-and-provenance.md` §2 重新评估(GPL-3.0 同许可与源码提供义务)。
|
||||
|
||||
## 11. W1-3c 实际迁移对照(MP 开源站点框架全量 + MovieClaw 站点定义)
|
||||
|
||||
逐文件对照总表与跟随更新流程见 `mp-sync-ledger.md`(`scripts/mp-sync/ledger.json` 由源码 `@mp-source` 标签生成)。本节只记结构性决定。
|
||||
|
||||
### 11.1 专用 spider(`packages/adapters/src/sites/spiders/`)
|
||||
|
||||
| MP | 本项目 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `IndexerModule.SPIDER_PARSER_CLASSES` / `_SPECIALIZED_SEARCH_ARGUMENTS` | `spiders/index.ts` `SPECIALIZED_SPIDERS`、`SpecializedSpider.search(ctx, {keyword, mtype, cat, page})` | 键名与 MP 相同(Haidan/HDDolby/RousiPro/SunnyPT/TNodeSpider/TorrentLeech/Yema/mTorrent) |
|
||||
| 各 spider `__get_params` / `__parse_result` / `__process_response` / 系数函数 | `spiders/<name>.ts` 同名纯函数 | 常量表逐字保留;结果为 MP 字段(`SpiderTorrent`)+ `category_id`、`site_release_id`、`*_utc` 扩展 |
|
||||
| `(True, [])` 错误返回 | `SiteError` | 401/403 → `auth_invalid`,缺凭据 → `auth_missing`,业务错误按文案分类 |
|
||||
| `[base64 请求配置]url` enclosure + `submission._resolve_indirect_download_url` | `spiders/common.ts` `resolveIndirectDownloadUrl` / `downloadTorrentFile` | 下载时现场换直链,结果约束到可信 API 基址,不带凭据下载并校验 .torrent |
|
||||
| `TNodeSpider.__get_token`(`@cached` 24h)、`SunnyPTSpider._category_cache`(1h) | 适配器实例内 TTL 缓存 | |
|
||||
|
||||
### 11.2 站点用户信息解析器(`packages/adapters/src/sites/user-parsers/`)
|
||||
|
||||
| MP | 本项目 |
|
||||
| --- | --- |
|
||||
| `parser/__init__.py` `SiteSchema` / `SiteParserBase` | `types.ts` `SiteSchema` / `base.ts` `SiteParserBase`(`parse` 流程、`_get_page_content` 请求模式、`_parse_logged_in`、`to_dict`) |
|
||||
| `parser/<name>.py` 20 个类 | `user-parsers/<kebab-name>.ts` 同名类 |
|
||||
| `IndexerModule.refresh_userdata`(schema 回退) | `registry.ts` `refreshSiteUserData` |
|
||||
| `SiteUserData` | `SiteUserInfo`(camelCase,`raw` = `to_dict()`,不含凭据) |
|
||||
| lxml XPath / BeautifulSoup | `dom.ts`(xpath + xmldom)/ cheerio |
|
||||
| `SiteUtils.is_logged_in`、`under_challenge`、`parse_int/parse_float`、`normalize_datetime`、`format_timestamp`、`extract_domain` | `common.ts` 同名函数 |
|
||||
|
||||
偏离:翻页上限 50;网络/HTTP 失败另记 `http_errors`;写操作(标记已读)受 `fetchMessages` 门控;`normalize_datetime` 只处理绝对时间。
|
||||
|
||||
### 11.3 站点定义
|
||||
|
||||
- MP 的站点定义不可得(加密),定义 schema 按 `sites.pyi` 与 spider 读取的键整理为 zod(`definitions/schema.ts`),加载即校验。
|
||||
- 选择器值:用户 2026-09-27 确认复制 MovieClaw **v0.27.0**(MIT)资产,23 个站点经 `scripts/mp-sync/movieclaw-import.mjs` 转换(ledger CL-012/CL-013,`copied-mit`,NOTICE 见 `definitions/NOTICE-MovieClaw.md`)。映射规则按 `movieclaw-assessment.md` §4;`page_offset -1 → browse.start 0`;`promo_*_rules → case` 表(补 `"*": 1`);`X::attr(a) → {selector, attribute}`;分类取 `\d+`;MovieClaw 的 documentary/anime/game/av/other 只进 `categoryNames`(原始 ID 保留)。
|
||||
- 观众/馒头保留 W1-3b 手写值(覆盖层),HHClub/CHD 覆盖层补站点分类名;4 个主力站启用,其余 19 站 `enabled:false`(未实测)。
|
||||
@@ -0,0 +1,99 @@
|
||||
# MoviePilot 跟随移植账本(mp-sync)
|
||||
|
||||
> **许可结论(引自 `license-and-provenance.md`)**:MoviePilot 为 GPLv3;本目录登记的站点框架文件是用户 2026-09-27 明确要求"照抄 MP v3 实现迁移"的数据来源适配部分,每个移植文件头保留 `SPDX-License-Identifier: GPL-3.0-only` 与来源声明;MP 的闭源/加密部分(`sites.*.so`、`user.sites.v3.bin`)**不解密、不参考**,站点定义改由 MovieClaw **v0.27.0(MIT,`copied-mit`,见 `packages/adapters/src/sites/definitions/NOTICE-MovieClaw.md`)** 转换 + 手写覆盖。MovieClaw v0.27.0 之后改为附条件许可,**不跟随其后续提交**。
|
||||
|
||||
## 1. 机制
|
||||
|
||||
- **登记来源**:每个移植文件头写 `@mp-source <上游路径> @ <提交>`(一个文件可多行);sites 目录外或 W1-3a/b 早期文件在 `scripts/mp-sync/ledger.manual.json` 人工登记。
|
||||
- **ledger**:`scripts/mp-sync/ledger.json`(`build-ledger.mjs` 生成)按上游文件聚合:`upstream`、移植时 `commit`、该文件在该提交的内容 `sha256`、本项目文件 `ours[]`。
|
||||
- **监视目录**:`app/modules/indexer/spider/`、`app/modules/indexer/parser/`,出现 ledger 未收录的新文件即提示(MP 新增站点 spider/解析器)。
|
||||
|
||||
## 2. 用法
|
||||
|
||||
MP 本地仓库默认 `/opt/moviepilot-plus/upstream/MoviePilot`(浅克隆,当前 `e6820a8`)。以下命令 Node 24 或 `deno run -A` 均可(不依赖根 package.json / deno.json 任务)。
|
||||
|
||||
```bash
|
||||
# 1) 取上游新提交(浅克隆)
|
||||
git -C /opt/moviepilot-plus/upstream/MoviePilot fetch --depth=1 origin <新提交或 v3>
|
||||
# 2) 查差异:已变更 / 已删除 / 监视目录新增,附逐文件 git diff 命令;有变化退出码 1
|
||||
node scripts/mp-sync/check.mjs --to <新提交> # 或 --mp <路径> --json
|
||||
# 3) 按 diff 增量移植 TS 文件,把文件头 @mp-source 的提交号改为新提交(新文件加标签)
|
||||
# 4) 重建 ledger 与下表
|
||||
node scripts/mp-sync/build-ledger.mjs --commit <新提交>
|
||||
# 5) 跑适配器测试
|
||||
cd packages/adapters && deno run -A npm:vitest run src/sites
|
||||
```
|
||||
|
||||
MovieClaw 站点定义(与 MP 无关,但同属站点框架跟随工具):
|
||||
|
||||
```bash
|
||||
node scripts/mp-sync/movieclaw-import.mjs # 从 v0.27.0 重新转换 23 个站点定义(覆盖层:scripts/mp-sync/movieclaw-overrides/)
|
||||
node scripts/mp-sync/movieclaw-import.mjs --check # 只比较,有差异退出码 1
|
||||
```
|
||||
|
||||
## 3. 当前对照总表
|
||||
|
||||
<!-- mp-sync:table:begin -->
|
||||
共 49 个上游文件(`adapters/` = `packages/adapters/src/`)。
|
||||
|
||||
| 上游文件 | 本项目文件 | 移植提交 | sha256(前 12 位) | 来源 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `app/adapters/network/cloudflare.py` | `adapters/sites/user-parsers/common.ts` | `e6820a8` | `a23a2c44da05` | tag |
|
||||
| `app/application/mediaserver.py` | `adapters/emby/client.ts` | `e6820a8` | `f034f0be8ba6` | manual |
|
||||
| `app/application/rss.py` | `adapters/rss/fetch.ts`<br>`adapters/rss/parse.ts`<br>`adapters/rss/rss-link.ts` | `e6820a8` | `8ea70f712512` | manual |
|
||||
| `app/application/site/sites.pyi` | `adapters/sites/definitions/schema.ts`<br>`adapters/sites/registry.ts`<br>`adapters/sites/spider/schema.ts` | `e6820a8` | `8fea9a239354` | manual |
|
||||
| `app/chain/download/submission.py` | `adapters/sites/mteam/adapter.ts`<br>`adapters/sites/spider/adapter.ts`<br>`adapters/sites/spiders/common.ts` | `e6820a8` | `f7cb870e47d6` | manual+tag |
|
||||
| `app/domain/site.py` | `adapters/sites/user-parsers/common.ts` | `e6820a8` | `650c795a4def` | tag |
|
||||
| `app/foundation/dom.py` | `adapters/sites/user-parsers/dom.ts` | `e6820a8` | `6d6accf2cef1` | tag |
|
||||
| `app/foundation/size.py` | `adapters/sites/spider/spider.ts` | `e6820a8` | `7b0b8f50588c` | manual |
|
||||
| `app/foundation/temporal.py` | `adapters/sites/spiders/common.ts`<br>`adapters/sites/user-parsers/common.ts` | `e6820a8` | `74941f460e40` | tag |
|
||||
| `app/foundation/text.py` | `adapters/sites/user-parsers/common.ts` | `e6820a8` | `a88c75dc8431` | tag |
|
||||
| `app/modules/__init__.py` | `adapters/qbittorrent/paths.ts` | `e6820a8` | `2edada0ea591` | manual |
|
||||
| `app/modules/_base/downloader.py` | `adapters/qbittorrent/torrent.ts` | `e6820a8` | `84bd8946c8b2` | manual |
|
||||
| `app/modules/emby/emby.py` | `adapters/emby/client.ts` | `e6820a8` | `e6468ea34ef2` | manual |
|
||||
| `app/modules/filemanager/storages/local.py` | `adapters/fs/copy.ts`<br>`adapters/fs/move.ts` | `e6820a8` | `806a1970ca8e` | manual |
|
||||
| `app/modules/filemanager/transhandler.py` | `adapters/fs/move.ts` | `e6820a8` | `13e2bba491ac` | manual |
|
||||
| `app/modules/indexer/__init__.py` | `adapters/sites/registry.ts`<br>`adapters/sites/spider/adapter.ts`<br>`adapters/sites/spiders/adapter.ts`<br>`adapters/sites/spiders/index.ts`<br>`adapters/sites/spiders/map.ts`<br>`adapters/sites/spiders/types.ts`<br>`adapters/sites/user-parsers/registry.ts` | `e6820a8` | `e9c5d15fdaa5` | manual+tag |
|
||||
| `app/modules/indexer/parser/__init__.py` | `adapters/sites/user-parsers/base.ts`<br>`adapters/sites/user-parsers/types.ts` | `e6820a8` | `0e6c4215b713` | tag |
|
||||
| `app/modules/indexer/parser/bitpt.py` | `adapters/sites/user-parsers/bitpt.ts` | `e6820a8` | `871d25e570f1` | tag |
|
||||
| `app/modules/indexer/parser/discuz.py` | `adapters/sites/user-parsers/discuz.ts` | `e6820a8` | `12277ba955ce` | tag |
|
||||
| `app/modules/indexer/parser/file_list.py` | `adapters/sites/user-parsers/file-list.ts` | `e6820a8` | `f92e65ed7327` | tag |
|
||||
| `app/modules/indexer/parser/gazelle.py` | `adapters/sites/user-parsers/gazelle.ts` | `e6820a8` | `c10e5246724b` | tag |
|
||||
| `app/modules/indexer/parser/hddolby.py` | `adapters/sites/user-parsers/hddolby.ts` | `e6820a8` | `a4336d92d603` | tag |
|
||||
| `app/modules/indexer/parser/ipt.py` | `adapters/sites/user-parsers/ipt.ts` | `e6820a8` | `2b9d04d76606` | tag |
|
||||
| `app/modules/indexer/parser/mtorrent.py` | `adapters/sites/user-parsers/mtorrent.ts` | `e6820a8` | `c5d5a3c6f9f5` | tag |
|
||||
| `app/modules/indexer/parser/nexus_audiences.py` | `adapters/sites/user-parsers/nexus-audiences.ts` | `e6820a8` | `1a2d104db769` | tag |
|
||||
| `app/modules/indexer/parser/nexus_hhanclub.py` | `adapters/sites/user-parsers/nexus-hhanclub.ts` | `e6820a8` | `8850619069d4` | tag |
|
||||
| `app/modules/indexer/parser/nexus_php.py` | `adapters/sites/user-parsers/nexus-php.ts` | `e6820a8` | `39303ae96d9c` | tag |
|
||||
| `app/modules/indexer/parser/nexus_project.py` | `adapters/sites/user-parsers/nexus-project.ts` | `e6820a8` | `2b5508cfb2f8` | tag |
|
||||
| `app/modules/indexer/parser/nexus_rabbit.py` | `adapters/sites/user-parsers/nexus-rabbit.ts` | `e6820a8` | `09149b89a7c7` | tag |
|
||||
| `app/modules/indexer/parser/rousi.py` | `adapters/sites/user-parsers/rousi.ts` | `e6820a8` | `1b1ea93de084` | tag |
|
||||
| `app/modules/indexer/parser/small_horse.py` | `adapters/sites/user-parsers/small-horse.ts` | `e6820a8` | `b73b5de97737` | tag |
|
||||
| `app/modules/indexer/parser/sunnypt.py` | `adapters/sites/user-parsers/sunnypt.ts` | `e6820a8` | `937f8453fa2b` | tag |
|
||||
| `app/modules/indexer/parser/tnode.py` | `adapters/sites/user-parsers/tnode.ts` | `e6820a8` | `c9cb809fc4bf` | tag |
|
||||
| `app/modules/indexer/parser/torrent_leech.py` | `adapters/sites/user-parsers/torrent-leech.ts` | `e6820a8` | `72d1473a3bba` | tag |
|
||||
| `app/modules/indexer/parser/unit3d.py` | `adapters/sites/user-parsers/unit3d.ts` | `e6820a8` | `9a4d579deca5` | tag |
|
||||
| `app/modules/indexer/parser/yema.py` | `adapters/sites/user-parsers/yema.ts` | `e6820a8` | `0f98fd580bad` | tag |
|
||||
| `app/modules/indexer/parser/zhixing.py` | `adapters/sites/user-parsers/zhixing.ts` | `e6820a8` | `90c8954e7387` | tag |
|
||||
| `app/modules/indexer/spider/__init__.py` | `adapters/sites/spider/map.ts`<br>`adapters/sites/spider/schema.ts`<br>`adapters/sites/spider/spider.ts`<br>`adapters/sites/spider/template.ts` | `e6820a8` | `2513d962fa43` | manual |
|
||||
| `app/modules/indexer/spider/haidan.py` | `adapters/sites/spiders/haidan.ts` | `e6820a8` | `75eb802f19f0` | tag |
|
||||
| `app/modules/indexer/spider/hddolby.py` | `adapters/sites/spiders/hddolby.ts` | `e6820a8` | `46a50bd8d49f` | tag |
|
||||
| `app/modules/indexer/spider/mtorrent.py` | `adapters/sites/mteam/adapter.ts`<br>`adapters/sites/mteam/client.ts`<br>`adapters/sites/mteam/spider.ts`<br>`adapters/sites/spiders/mtorrent.ts` | `e6820a8` | `a8ab2b822cb2` | manual+tag |
|
||||
| `app/modules/indexer/spider/rousi.py` | `adapters/sites/spiders/rousi.ts` | `e6820a8` | `aa13c386a560` | tag |
|
||||
| `app/modules/indexer/spider/sunnypt.py` | `adapters/sites/spiders/sunnypt.ts` | `e6820a8` | `735b0d721e65` | tag |
|
||||
| `app/modules/indexer/spider/tnode.py` | `adapters/sites/spiders/tnode.ts` | `e6820a8` | `a3634a268dba` | tag |
|
||||
| `app/modules/indexer/spider/torrentleech.py` | `adapters/sites/spiders/torrentleech.ts` | `e6820a8` | `c965f2bcaea4` | tag |
|
||||
| `app/modules/indexer/spider/yema.py` | `adapters/sites/spiders/yema.ts` | `e6820a8` | `560b62c1ff10` | tag |
|
||||
| `app/modules/qbittorrent/qbittorrent.py` | `adapters/qbittorrent/client.ts`<br>`adapters/qbittorrent/state.ts` | `e6820a8` | `ab02e97e986f` | manual |
|
||||
| `app/modules/themoviedb/tmdbapi.py` | `adapters/tmdb/client.ts`<br>`adapters/tmdb/match.ts` | `e6820a8` | `787553a6f57d` | manual |
|
||||
| `app/schemas/site.py` | `adapters/sites/user-parsers/types.ts` | `e6820a8` | `e7952ed40b7b` | tag |
|
||||
<!-- mp-sync:table:end -->
|
||||
|
||||
## 4. 移植时的统一偏离(每个文件头另有细节)
|
||||
|
||||
1. **凭据不进候选/结果**:MP 两段式 enclosure(`[base64(含 apikey 的请求配置)]url`)不生成,下载时由 adapter 现场换直链;用户信息 `raw` 不含 cookie/apikey/token。
|
||||
2. **错误显式化**:MP spider 的 `(is_error, [])` 改为抛 `SiteError{kind}`;用户信息解析保留 MP 的 `err_msg` 语义,另记脱敏的 `http_errors`。
|
||||
3. **翻页上限**:做种/消息翻页最多 50 页(MP 无上限)。
|
||||
4. **XPath**:lxml → `xpath` + `@xmldom/xmldom`(经 htmlparser2 解析,不插入 tbody),见 `user-parsers/dom.ts`。
|
||||
5. **dateparser**:只实现绝对时间格式,相对时间原样返回(待核验)。
|
||||
6. **写操作门控**:读取站内信后标记已读(MTorrent/Rousi/NexusRabbit)只在 `fetchMessages=true` 时发生(MP 由全局 `SITE_MESSAGE` 控制)。
|
||||
@@ -52,9 +52,51 @@ PT 站点适配器,按 MoviePilot v3(提交 e6820a8)迁移:通用配置驱动 sp
|
||||
9. `categoryNames` 为自行编制的粗分类名;默认网页域 `tp.m-team.cc` 需按账号实际域名配置。
|
||||
10. H&R:API 未提供,恒为 `unknown`。
|
||||
|
||||
## M2 预留:站点用户数据解析器(首版未实现)
|
||||
## W1-3c:MP 开源站点框架全量移植 + MovieClaw 站点定义
|
||||
|
||||
- `app/modules/indexer/parser/nexus_php.py`(NexusPHP 通用:上传/下载/分享率/做种)
|
||||
- `app/modules/indexer/parser/nexus_audiences.py`(观众新版 userbar;未读私信首页不能带 page 参数)
|
||||
- `app/modules/indexer/parser/nexus_hhanclub.py`(HHClub)
|
||||
- `app/modules/indexer/parser/mtorrent.py`(M-Team:`api/member/profile`、`api/tracker/myPeerStatus` 等)
|
||||
跟随更新工具与对照总表见 `docs/porting/mp-sync-ledger.md`(`scripts/mp-sync/`)。每个移植文件头有 `SPDX-License-Identifier: GPL-3.0-only`、`@mp-source <上游路径> @ e6820a8` 和函数对照。
|
||||
|
||||
### 专用 spider(`spiders/`,MP `app/modules/indexer/spider/*.py`)
|
||||
|
||||
| parser 键(MP `SPIDER_PARSER_CLASSES`) | 文件 | 凭据 | 下载 |
|
||||
| --- | --- | --- | --- |
|
||||
| `Haidan` | `spiders/haidan.ts` | cookie | 列表直链(仅内存),raw 脱敏 |
|
||||
| `HDDolby` | `spiders/hddolby.ts` | apiKey(+cookie) | `download.php?id=&downhash=`(仅内存) |
|
||||
| `RousiPro` | `spiders/rousi.ts` | apiKey(Bearer) | 现场请求详情接口取 `data.download_url` |
|
||||
| `SunnyPT` | `spiders/sunnypt.ts` | apiKey | 现场 `download-token` 换直链 |
|
||||
| `TNodeSpider` | `spiders/tnode.ts` | cookie + 首页 CSRF(实例内缓存 24h) | 直链 |
|
||||
| `TorrentLeech` | `spiders/torrentleech.ts` | cookie | 直链;中文关键词按 MP 返回空 |
|
||||
| `Yema` | `spiders/yema.ts` | apiKey(Authorization) | 现场 `generateDownloadKey` 换 token |
|
||||
| `mTorrent` | `spiders/mtorrent.ts`(包装 `mteam/*`) | apiKey | genDlToken |
|
||||
|
||||
`createSpecializedAdapterFor(def, deps)` 实现统一 `SiteAdapter`(list = 空关键词搜索;`detail` 为 `unsupported`);定义 `umt.kind = "api"`,schema 见 `definitions/schema.ts` `SpecializedSiteDefinitionSchema`。MP 的 `[base64(含 apikey 的请求配置)]url` 两段式 enclosure 不生成,下载时按同一配置现场换直链并校验 .torrent。`mteam/spider.ts` 已与 `mtorrent.py` 逐条核对(修正:纯数字时间戳按 +08:00 转换)。
|
||||
|
||||
### 站点用户信息解析器(`user-parsers/`,MP `app/modules/indexer/parser/*.py`)
|
||||
|
||||
20 个 `SiteSchema` 全部移植:NexusPhp、NexusAudiences、NexusHhanclub、NexusProject、NexusRabbit、Gazelle、Unit3d、DiscuzX、IPTorrents、FileList、Small Horse、Zhixing、Bitpt、TNode、TorrentLeech、HDDolby、RousiPro、SunnyPT、Yema、MTorrent。
|
||||
|
||||
- 基类 `SiteParserBase`(`base.ts`)与 MP 相同的获取顺序:首页登录判定 → `_parseSitePage` → 基础信息 → 详情 → 未读消息(`fetchMessages`)→ 流量 → 做种分页(上限 50 页);cookie/apikey 两种请求模式;200/500/403 视为有内容;Cloudflare 挑战页返回空。
|
||||
- XPath 用 `xpath` + `@xmldom/xmldom`(`dom.ts`,htmlparser2 解析,不插入 tbody),表达式与 MP 逐字一致。
|
||||
- 入口 `refreshSiteUserData({name, url, schema, cookie|apikey|token, fetch?, limiter?})` → `SiteUserInfo{schema, domain, userId, username, userLevel, joinAt, upload, download, ratio, bonus, seeding, seedingSize, leeching, leechingSize, seedingInfo, messageUnread, messageUnreadContents, hrCount?, errMsg, raw}`;声明 schema 取不到 userid 时按 MP 顺序回退到其他 schema。
|
||||
- 写操作:MTorrent / Rousi / NexusRabbit 读消息后标记已读,只在 `fetchMessages: true` 时发生。
|
||||
|
||||
### 站点定义体系(`definitions/`)
|
||||
|
||||
- 23 个站点 JSON 由 `scripts/mp-sync/movieclaw-import.mjs` 从 **MovieClaw v0.27.0(`c79bb1b0`,MIT)** 转换(见 `definitions/NOTICE-MovieClaw.md`),每个带 `provenance`;手工覆盖层 `scripts/mp-sync/movieclaw-overrides/*.json`(观众、馒头沿用 W1-3b 值,HHClub/CHD 补分类名)。**不要直接改 JSON**,改覆盖层后重跑脚本。
|
||||
- 加载时 zod 校验(`definitions/schema.ts`:`IndexerDefinitionSchema` / `MteamDefinitionSchema` / `SpecializedSiteDefinitionSchema`,按 `umt.kind` 分派);注册表 `getSiteDefinition(id)`、`listSiteDefinitions({enabledOnly})`、`createSiteAdapter(def, deps)`。
|
||||
- 新增字段:`enabled`、`schema`(MP 用户信息解析器)、`provenance`、`umt.detail.fields`(MovieClaw 逐字段详情选择器,支持 HHClub div 详情页)、`umt.profile`(MovieClaw 资料选择器,备用)、`umt.login`、`umt.authSupported`、`umt.customClass`。选择器扩展语法 `::attr(x)` / `::text` / `::next_sibling_text` 由 `spider/mc-selector.ts` 求值。
|
||||
|
||||
| 状态 | 站点 |
|
||||
| --- | --- |
|
||||
| 启用(主力,仍须真实账号核验) | audiences 观众、hhanclub 憨憨、chdbits 彩虹岛、mteam 馒头 |
|
||||
| `enabled:false`,**未实测** | agsvpt、hdarea、hddolby、hdfans、hdhome、hdsky、hdtime、hdvideo、keepfrds、nicept、ourbits、piggo、pter、pthome、pttime、soulvoice、ssd、tjupt、ttg |
|
||||
|
||||
未实测站点只经 schema 校验,选择器是 MovieClaw 作者在 2026-08/09 的实测值,未经本项目核验。TTG 需要自定义适配器(中文分类、`/t/<id>` 路径、做种/下载同列),通用 spider 结果不可靠;OurBits 的 `cat[]` 分类参数已由定义表达;hddolby 在 MovieClaw 中按 HTML 站点配置,MP 则用 API spider(`HDDolby`),两条路径待账号确认后二选一。
|
||||
|
||||
### 待真实站点核验(W1-3c 新增)
|
||||
|
||||
1. HHClub:`div.torrent-table-sub-info` 卡片结构、`?cat[]=` 分类链接(若 URL 编码为 `cat%5B%5D=` 则 `\d+` 会取错,需改 querystring 过滤)、`promotion-tag-*` 新旧类名、`div.torrent-info-hr`、详情页 Tailwind 选择器、分页 0 起。
|
||||
2. CHDBits:域名 `ptchdbits.co`、`font.subtitle`、`td.embedded > span[title]` 促销截止、`a[href='logout.php']`。
|
||||
3. 专用 spider 的 JSON 字段均按 MP 读取的键编写 fixture,真实响应待核;HDDolby `hr` 字段语义、`page_number` 起点;TorrentLeech 时区;Yema `tagList` 类型;直链缺失时的回退地址(`TODO(verify)`)。
|
||||
4. 用户信息解析器 fixture 均为最小自编 HTML/JSON;`normalizeDatetime` 不处理相对时间(MP 用 dateparser)。
|
||||
5. 馒头 `mode=adult` 独立索引(MovieClaw custom/mteam.py)未实现。
|
||||
@@ -0,0 +1,38 @@
|
||||
# 第三方声明:MovieClaw(MIT)
|
||||
|
||||
本目录下的站点定义 JSON(`provenance.source = "movieclaw"`)由 `scripts/mp-sync/movieclaw-import.mjs`
|
||||
从 MovieClaw **tag `v0.27.0`(提交 `c79bb1b065c826823fc1c15bcb99b3c282c3fce0`)** 的
|
||||
`src/movieclaw_tracker/sites/configs/*.yaml` 与 `src/movieclaw_tracker/selectors.py`(`NexusPHPSelectors` 默认值)
|
||||
转换而来;`packages/adapters/src/sites/spider/mc-selector.ts` 的选择器语义对照 `frameworks/nexusphp.py` `_css_text`。
|
||||
该版本以 MIT 许可发布(MovieClaw 自 `251f1988` 起改为附条件许可,**新许可下的内容未被使用**)。
|
||||
|
||||
- 上游仓库:https://github.com/movieclaw/movieclaw
|
||||
- 许可类别:`copied-mit`(见 `docs/porting/license-and-provenance.md`)
|
||||
- 本项目的修改:字段改写为 MoviePilot indexer schema + `umt` 扩展;默认值展开;分类名与观众/馒头定义由
|
||||
`scripts/mp-sync/movieclaw-overrides/*.json` 覆盖。
|
||||
|
||||
以下为 MovieClaw v0.27.0 `LICENSE` 全文(原样):
|
||||
|
||||
```
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 MovieClaw
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
```
|
||||
@@ -0,0 +1,252 @@
|
||||
{
|
||||
"id": "agsvpt",
|
||||
"name": "AGSVPT",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/agsvpt.yaml"
|
||||
},
|
||||
"domain": "https://www.agsvpt.com/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "407",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "419",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "411",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded > span"
|
||||
},
|
||||
"category": {
|
||||
"selector": "a[href^='?cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "AGSVPT",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "剧集",
|
||||
"408": "音乐",
|
||||
"411": "音乐",
|
||||
"413": "游戏",
|
||||
"415": "其他",
|
||||
"419": "剧集"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='User_Name']::attr(href)",
|
||||
"username_css": "a[class*='User_Name']",
|
||||
"class_css": "a[class*='User_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "td.rowhead:contains('魔力') + td::text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img[alt='Torrents seeding']::next_sibling_text",
|
||||
"leeching_css": "img[alt='Torrents leeching']::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,16 @@
|
||||
{
|
||||
"_provenance": "按 MoviePilot SiteSpider indexer schema 手写;字段结构参考 MovieClaw audiences.yaml(data-ref,未复制);选择器值按 NexusPHP 公开页面惯例编写,仅对 testkit mock 验证,待真实页面核验。见 ../README.md",
|
||||
"id": "audiences",
|
||||
"name": "观众",
|
||||
"enabled": true,
|
||||
"schema": "NexusAudiences",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/audiences.yaml",
|
||||
"overrides": "scripts/mp-sync/movieclaw-overrides/audiences.json"
|
||||
},
|
||||
"domain": "https://audiences.me/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
@@ -10,14 +19,45 @@
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [{ "path": "torrents.php", "type": "all" }],
|
||||
"params": { "search": "{keyword}", "search_area": 4 }
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}",
|
||||
"search_area": 4
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"batch": { "delimiter": " ", "space_replace": "_" },
|
||||
"browse": { "path": "torrents.php?page={page}", "start": 0 },
|
||||
"category": {
|
||||
"movie": [{ "id": "401", "cat": "Movies", "desc": "电影" }],
|
||||
"tv": [{ "id": "402", "cat": "TV Series", "desc": "剧集" }, { "id": "403", "cat": "TV Shows", "desc": "综艺" }],
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"cat": "Movies",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"cat": "TV Series",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"cat": "TV Shows",
|
||||
"desc": "综艺"
|
||||
}
|
||||
],
|
||||
"music": []
|
||||
},
|
||||
"torrents": {
|
||||
@@ -25,28 +65,62 @@
|
||||
"selector": "table.torrents > tbody > tr:has(table.torrentname), table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": { "selector": "a[href*=\"details.php?id=\"]" },
|
||||
"title_optional": { "selector": "a[title][href*=\"details.php?id=\"]", "attribute": "title" },
|
||||
"title_default": {
|
||||
"selector": "a[href*=\"details.php?id=\"]"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[title][href*=\"details.php?id=\"]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": { "selector": "a[href*=\"details.php?id=\"]", "attribute": "href" },
|
||||
"download": { "selector": "a[href*=\"download.php?id=\"]", "attribute": "href" },
|
||||
"details": {
|
||||
"selector": "a[href*=\"details.php?id=\"]",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*=\"download.php?id=\"]",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [{ "name": "re_search", "args": ["tt\\d+", 0] }]
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4) > span"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date_elapsed": { "selector": "td:nth-child(4) > span" },
|
||||
"date_added": { "selector": "td:nth-child(4) > span", "attribute": "title" },
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [{ "name": "dateparse", "args": "%Y-%m-%d %H:%M:%S" }]
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"size": { "selector": "td:nth-child(5)" },
|
||||
"seeders": { "selector": "td:nth-child(6)" },
|
||||
"leechers": { "selector": "td:nth-child(7)" },
|
||||
"grabs": { "selector": "td:nth-child(8)" },
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
@@ -65,17 +139,31 @@
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"freedate": {
|
||||
"selector": "img[class^=\"pro_\"]",
|
||||
"attribute": "title",
|
||||
"filters": [{ "name": "re_search", "args": ["\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}", 0] }]
|
||||
"description": {
|
||||
"selector": "span.torrent-subtitle-line"
|
||||
},
|
||||
"description": { "selector": "span.torrent-subtitle-line" },
|
||||
"labels": { "selector": "span.tags" },
|
||||
"category": {
|
||||
"selector": "a[href*=\"?cat=\"]",
|
||||
"attribute": "href",
|
||||
"filters": [{ "name": "querystring", "args": "cat" }]
|
||||
"filters": [
|
||||
{
|
||||
"name": "querystring",
|
||||
"args": "cat"
|
||||
}
|
||||
]
|
||||
},
|
||||
"freedate": {
|
||||
"selector": "img[class^=\"pro_\"]",
|
||||
"attribute": "title",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"labels": {
|
||||
"selector": "span.tags"
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -84,14 +172,47 @@
|
||||
"kind": "nexusphp",
|
||||
"displayName": "观众",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": { "minIntervalMs": 3000, "burst": 1 },
|
||||
"capabilities": { "list": true, "search": true, "detail": true, "download": true, "rss": true },
|
||||
"categoryNames": { "401": "电影", "402": "剧集", "403": "综艺", "406": "纪录片" },
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": true
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "综艺",
|
||||
"406": "纪录片"
|
||||
},
|
||||
"loginSuccess": "a[href*=\"logout\"]",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[href*='userdetails.php?id=']::attr(href)",
|
||||
"username_css": "a[href*='userdetails.php?id=']",
|
||||
"class_css": "td.rowhead:contains('等级') + td::text",
|
||||
"uploaded_css": "span.site-userbar__compact-metric--uploaded",
|
||||
"downloaded_css": "span.site-userbar__compact-metric--downloaded",
|
||||
"ratio_css": "td.rowhead:contains('分享率') + td::text",
|
||||
"bonus_css": "td.rowhead:contains('魔力') + td::text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "td.rowhead:contains('当前做种') + td::text",
|
||||
"leeching_css": "td.rowhead:contains('当前下载') + td::text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
|
||||
@@ -0,0 +1,252 @@
|
||||
{
|
||||
"id": "chdbits",
|
||||
"name": "彩虹岛",
|
||||
"enabled": true,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/chdbits.yaml",
|
||||
"overrides": "scripts/mp-sync/movieclaw-overrides/chdbits.json"
|
||||
},
|
||||
"domain": "https://ptchdbits.co/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "综艺"
|
||||
},
|
||||
{
|
||||
"id": "407",
|
||||
"desc": "体育"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "MV"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "HQ Audio"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "font.subtitle"
|
||||
},
|
||||
"category": {
|
||||
"selector": "a[href^='?cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"freedate": {
|
||||
"selector": "td.embedded > span[title]",
|
||||
"attribute": "title",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "彩虹岛",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "综艺",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "MV",
|
||||
"407": "体育",
|
||||
"408": "HQ Audio",
|
||||
"409": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href='logout.php']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"username_css": "a[href^='userdetails.php?id=']",
|
||||
"class_css": "td.rowhead:contains('等级') + td::text",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "td.rowhead:contains('魔力') + td::text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img[alt='Torrents seeding']::next_sibling_text",
|
||||
"leeching_css": "img[alt='Torrents leeching']::next_sibling_text",
|
||||
"vip_css": "a[class^='VIP']"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
import { CHDBITS_LIST_PAGE, CHDBITS_LOGIN_PAGE, HHANCLUB_DETAIL_PAGE, HHANCLUB_LIST_PAGE } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { getSiteDefinition, listSiteDefinitions } from "../registry.ts";
|
||||
import { parseDetailPage } from "../spider/detail.ts";
|
||||
import { spiderTorrentToRelease } from "../spider/map.ts";
|
||||
import type { IndexerDefinition } from "../spider/schema.ts";
|
||||
import { SiteSpider } from "../spider/spider.ts";
|
||||
import {
|
||||
BUILTIN_SITE_DEFINITIONS,
|
||||
chdbitsDefinition,
|
||||
hhanclubDefinition,
|
||||
loadSiteDefinition,
|
||||
parseIndexerDefinition,
|
||||
} from "./index.ts";
|
||||
import { MOVIECLAW_DEFINITION_JSON } from "./manifest.generated.ts";
|
||||
|
||||
const OBSERVED = "2026-09-27T00:00:00.000Z";
|
||||
const PRIMARY = ["audiences", "chdbits", "hhanclub", "mteam"];
|
||||
|
||||
describe("站点定义:加载时 zod 校验", () => {
|
||||
const ids = Object.keys(MOVIECLAW_DEFINITION_JSON).sort();
|
||||
|
||||
it("MovieClaw v0.27.0 全部 23 站转换结果均在注册表中", () => {
|
||||
expect(ids).toHaveLength(23);
|
||||
expect(listSiteDefinitions().map((d) => d.id)).toEqual(ids);
|
||||
});
|
||||
|
||||
it.each(ids)("%s:schema 校验通过,provenance 指向 MovieClaw v0.27.0 MIT", (id) => {
|
||||
const def = loadSiteDefinition(MOVIECLAW_DEFINITION_JSON[id]);
|
||||
expect(def.id).toBe(id);
|
||||
expect(def.provenance).toMatchObject({ source: "movieclaw", ref: "v0.27.0", commit: "c79bb1b0", license: "MIT" });
|
||||
expect(def.provenance?.file).toBe(`src/movieclaw_tracker/sites/configs/${id}.yaml`);
|
||||
expect(def.enabled).toBe(PRIMARY.includes(id));
|
||||
expect(getSiteDefinition(id)).toBe(BUILTIN_SITE_DEFINITIONS[id]);
|
||||
});
|
||||
|
||||
it("enabledOnly 只列 4 个主力站", () => {
|
||||
expect(listSiteDefinitions({ enabledOnly: true }).map((d) => d.id)).toEqual(PRIMARY);
|
||||
});
|
||||
|
||||
it("非法定义被拒绝并指出字段路径", () => {
|
||||
const bad = structuredClone(MOVIECLAW_DEFINITION_JSON.chdbits) as Record<string, unknown>;
|
||||
(bad as { domain: string }).domain = "https://ptchdbits.co"; // 缺结尾 /
|
||||
delete (bad as { umt: Record<string, unknown> }).umt.timezone;
|
||||
expect(() => parseIndexerDefinition(bad)).toThrow(/chdbits.*domain.*umt\.timezone|chdbits.*umt\.timezone.*domain/);
|
||||
expect(() => loadSiteDefinition({ id: "x", umt: { kind: "gopher" } })).toThrow(/umt\.kind 未知/);
|
||||
const typo = structuredClone(MOVIECLAW_DEFINITION_JSON.hhanclub) as {
|
||||
torrents: { fields: Record<string, unknown> };
|
||||
};
|
||||
typo.torrents.fields.size = { selectr: "div" };
|
||||
expect(() => parseIndexerDefinition(typo)).toThrow(/torrents\.fields\.size/);
|
||||
});
|
||||
|
||||
it("观众:W1-3b 手写值优先(覆盖层),MovieClaw 值补空", () => {
|
||||
const a = getSiteDefinition("audiences") as IndexerDefinition;
|
||||
expect(a.search?.params).toEqual({ search: "{keyword}", search_area: 4 }); // W1-3b
|
||||
expect(a.torrents.fields.category?.filters).toEqual([{ name: "querystring", args: "cat" }]); // W1-3b
|
||||
expect(a.umt.profile?.uploaded_css).toBe("span.site-userbar__compact-metric--uploaded"); // MovieClaw 补空
|
||||
expect(a.schema).toBe("NexusAudiences");
|
||||
expect(a.provenance?.overrides).toBe("scripts/mp-sync/movieclaw-overrides/audiences.json");
|
||||
});
|
||||
|
||||
it("OurBits 的 cat[] 分类参数由定义表达;TTG 需自定义适配器 → 禁用", () => {
|
||||
const ob = getSiteDefinition("ourbits") as IndexerDefinition;
|
||||
expect(ob.category?.param).toBe("cat[]");
|
||||
const url = new URL(new SiteSpider(ob, { keyword: "x", mtype: "movie" }).getSearchUrl());
|
||||
expect(url.searchParams.getAll("cat[]").length).toBeGreaterThan(0);
|
||||
const ttg = getSiteDefinition("ttg") as IndexerDefinition;
|
||||
expect(ttg.umt.customClass).toContain("TTGSite");
|
||||
expect(ttg.enabled).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("HHClub 定义(div 卡片模板)", () => {
|
||||
const parse = (html: string) => {
|
||||
const r = new SiteSpider(hhanclubDefinition, {}).parse(html);
|
||||
return r.torrents.map((t) => spiderTorrentToRelease(t, hhanclubDefinition, OBSERVED)!);
|
||||
};
|
||||
|
||||
it("列表:标题/副标题/综艺原始分类 403/大小/人数/发布时间/免费 + H&R", () => {
|
||||
const [a, b] = parse(HHANCLUB_LIST_PAGE);
|
||||
expect(a).toMatchObject({
|
||||
siteId: "hhanclub",
|
||||
siteReleaseId: "90001",
|
||||
title: "Go.Fighting.S10E01.2026.2160p.WEB-DL.H265.AAC-HHWEB",
|
||||
subtitle: "极限挑战 第十季 第1期",
|
||||
category: { id: "403", name: "综艺" },
|
||||
sizeBytes: Math.round(6.12 * 1024 ** 3),
|
||||
seeders: 88,
|
||||
leechers: 5,
|
||||
snatched: 1024,
|
||||
promotion: { type: "free", downloadFactor: 0, uploadFactor: 1 },
|
||||
hr: true,
|
||||
uploadedAt: "2026-09-20T12:30:00.000Z",
|
||||
detailUrl: "https://hhanclub.net/details.php?id=90001&hit=1",
|
||||
});
|
||||
expect(b).toMatchObject({
|
||||
siteReleaseId: "90002",
|
||||
category: { id: "401", name: "电影" },
|
||||
promotion: { type: "2x50%", downloadFactor: 0.5, uploadFactor: 2 },
|
||||
hr: false,
|
||||
});
|
||||
});
|
||||
|
||||
it("浏览从 page=0 起(MovieClaw page_offset -1);mtype=tv 选 402/403/407/412", () => {
|
||||
expect(new SiteSpider(hhanclubDefinition, { page: 0 }).getSearchUrl()).toBe(
|
||||
"https://hhanclub.net/torrents.php?page=0",
|
||||
);
|
||||
const u = new URL(new SiteSpider(hhanclubDefinition, { keyword: "极限挑战", mtype: "tv" }).getSearchUrl());
|
||||
expect(["cat402", "cat403", "cat407", "cat412"].every((k) => u.searchParams.get(k) === "1")).toBe(true);
|
||||
expect(u.searchParams.get("search")).toBe("极限挑战");
|
||||
});
|
||||
|
||||
it("详情页:逐字段选择器(Tailwind div 结构)取标题/副标题/大小/IMDb/豆瓣", () => {
|
||||
const { kind, detail } = parseDetailPage(HHANCLUB_DETAIL_PAGE, hhanclubDefinition, "90002", OBSERVED);
|
||||
expect(kind).toBe("ok");
|
||||
expect(detail).toMatchObject({
|
||||
title: "Dune.Part.Two.2024.2160p.UHD.BluRay.REMUX-HHWEB",
|
||||
subtitle: "沙丘2 | 中字",
|
||||
sizeBytes: Math.round(78.3 * 1024 ** 3),
|
||||
imdbId: "tt15239678",
|
||||
doubanId: "35575567",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("CHDBits 定义(标准 NexusPHP)", () => {
|
||||
it("列表:title 属性全名、font.subtitle、2x 免费 + 截止时间、cat 链接分类", () => {
|
||||
const r = new SiteSpider(chdbitsDefinition, {}).parse(CHDBITS_LIST_PAGE);
|
||||
const [a, b] = r.torrents.map((t) => spiderTorrentToRelease(t, chdbitsDefinition, OBSERVED)!);
|
||||
expect(a).toMatchObject({
|
||||
siteReleaseId: "555001",
|
||||
title: "The.Last.of.Us.S02.2025.1080p.BluRay.x264.DTS-HD.MA.5.1-CHD",
|
||||
category: { id: "402", name: "剧集" },
|
||||
sizeBytes: Math.round(45.2 * 1024 ** 3),
|
||||
seeders: 42,
|
||||
leechers: 3,
|
||||
snatched: 100,
|
||||
promotion: { type: "2xfree", downloadFactor: 0, uploadFactor: 2, expiresAt: "2026-09-30T04:00:00.000Z" },
|
||||
hr: "unknown",
|
||||
uploadedAt: "2026-09-25T00:00:00.000Z",
|
||||
});
|
||||
expect(a!.subtitle).toContain("最后生还者 第二季");
|
||||
expect(b).toMatchObject({
|
||||
siteReleaseId: "555002",
|
||||
category: { id: "401", name: "电影" },
|
||||
promotion: { type: "none", downloadFactor: 1, uploadFactor: 1, expiresAt: null },
|
||||
});
|
||||
});
|
||||
|
||||
it("登录页识别为登录/权限页,不产生结果", () => {
|
||||
const r = new SiteSpider(chdbitsDefinition, {}).parse(CHDBITS_LOGIN_PAGE);
|
||||
expect(r.loginPage).toBe(true);
|
||||
expect(r.torrents).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,279 @@
|
||||
{
|
||||
"id": "hdarea",
|
||||
"name": "高清视界 HDArea",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/hdarea.yaml"
|
||||
},
|
||||
"domain": "https://www.hdarea.club/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "300",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "410",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "411",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "412",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "413",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "414",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "415",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "416",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "417",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr, table.torrents > tbody > tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(5) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(9)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded > div[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "高清视界 HDArea",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 60000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"300": "电影",
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "其他",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"410": "电影",
|
||||
"411": "电影",
|
||||
"412": "电影",
|
||||
"413": "电影",
|
||||
"414": "电影",
|
||||
"415": "电影",
|
||||
"416": "电影",
|
||||
"417": "电影"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,236 @@
|
||||
{
|
||||
"id": "hddolby",
|
||||
"name": "高清杜比 HDDolby",
|
||||
"enabled": false,
|
||||
"schema": "HDDolby",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/hddolby.yaml"
|
||||
},
|
||||
"domain": "https://www.hddolby.com/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr, table.torrents > tbody > tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded span[style*='float:left']"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "高清杜比 HDDolby",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 5000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "其他",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"410": "游戏",
|
||||
"411": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,245 @@
|
||||
{
|
||||
"id": "hdfans",
|
||||
"name": "HDFans",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/hdfans.yaml"
|
||||
},
|
||||
"domain": "https://hdfans.org/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "416",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "407",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr, table.torrents > tbody > tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded br"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "HDFans",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 5000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "纪录片",
|
||||
"404": "其他",
|
||||
"405": "其他",
|
||||
"406": "音乐",
|
||||
"407": "音乐",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"410": "其他",
|
||||
"416": "剧集",
|
||||
"417": "动漫",
|
||||
"418": "其他",
|
||||
"419": "其他",
|
||||
"421": "游戏",
|
||||
"423": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,434 @@
|
||||
{
|
||||
"id": "hdhome",
|
||||
"name": "高清家园",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/hdhome.yaml"
|
||||
},
|
||||
"domain": "https://hdhome.org/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "411",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "412",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "413",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "414",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "415",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "416",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "450",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "499",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "505",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "506",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "518",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "425",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "426",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "427",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "428",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "429",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "430",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "431",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "432",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "433",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "434",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "435",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "436",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "437",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "438",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "442",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "443",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "452",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "453",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "502",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "504",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "511",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "523",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "526",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "439",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "440",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "441",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "503",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "423",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "424",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded > span:not([class])"
|
||||
},
|
||||
"category": {
|
||||
"selector": "a[href^='?cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"hr": {
|
||||
"selector": "img.hitandrun"
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "高清家园",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"409": "其他",
|
||||
"411": "电影",
|
||||
"412": "电影",
|
||||
"413": "电影",
|
||||
"414": "电影",
|
||||
"415": "电影",
|
||||
"416": "电影",
|
||||
"417": "纪录片",
|
||||
"418": "纪录片",
|
||||
"419": "纪录片",
|
||||
"420": "纪录片",
|
||||
"421": "纪录片",
|
||||
"422": "纪录片",
|
||||
"423": "音乐",
|
||||
"424": "音乐",
|
||||
"425": "剧集",
|
||||
"426": "剧集",
|
||||
"427": "剧集",
|
||||
"428": "剧集",
|
||||
"429": "剧集",
|
||||
"430": "剧集",
|
||||
"431": "剧集",
|
||||
"432": "剧集",
|
||||
"433": "剧集",
|
||||
"434": "剧集",
|
||||
"435": "剧集",
|
||||
"436": "剧集",
|
||||
"437": "剧集",
|
||||
"438": "剧集",
|
||||
"439": "音乐",
|
||||
"440": "音乐",
|
||||
"441": "音乐",
|
||||
"442": "剧集",
|
||||
"443": "剧集",
|
||||
"444": "动漫",
|
||||
"445": "动漫",
|
||||
"446": "动漫",
|
||||
"447": "动漫",
|
||||
"448": "动漫",
|
||||
"449": "动漫",
|
||||
"450": "电影",
|
||||
"451": "纪录片",
|
||||
"452": "剧集",
|
||||
"453": "剧集",
|
||||
"454": "动漫",
|
||||
"499": "电影",
|
||||
"500": "纪录片",
|
||||
"501": "动漫",
|
||||
"502": "剧集",
|
||||
"503": "音乐",
|
||||
"504": "剧集",
|
||||
"505": "电影",
|
||||
"506": "电影",
|
||||
"507": "纪录片",
|
||||
"508": "纪录片",
|
||||
"509": "动漫",
|
||||
"510": "动漫",
|
||||
"511": "剧集",
|
||||
"518": "电影",
|
||||
"523": "剧集",
|
||||
"526": "剧集",
|
||||
"529": "纪录片",
|
||||
"531": "动漫"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[href*='userdetails.php?id=']::attr(href)",
|
||||
"username_css": "a[href*='userdetails.php?id=']",
|
||||
"class_css": "a[href*='userdetails.php?id=']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus::next_sibling_text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img[alt='Torrents seeding'] + *",
|
||||
"leeching_css": "img[alt='Torrents leeching'] + *"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,276 @@
|
||||
{
|
||||
"id": "hdsky",
|
||||
"name": "HDSky",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/hdsky.yaml"
|
||||
},
|
||||
"domain": "https://hdsky.me/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "410",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "407",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "411",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "412",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "413",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "414",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "415",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "416",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents.progresstable tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a[href*='details.php'] b span"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a[href*='details.php'] b span[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "table.torrentname a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "form[action*='download.php']",
|
||||
"attribute": "action"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "a[href*='dllist=1#seeders']"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "a[href*='dllist=1#leechers']"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "a[href*='viewsnatches.php'] b, td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_2up": 2,
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "table.torrentname br + span"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:first-child a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "HDSky",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "剧集",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"410": "电影",
|
||||
"411": "剧集",
|
||||
"412": "剧集",
|
||||
"413": "剧集",
|
||||
"414": "剧集",
|
||||
"415": "剧集",
|
||||
"416": "剧集"
|
||||
},
|
||||
"loginSuccess": "a[href='logout.php']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td.rowfollow",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow",
|
||||
"description": "#kdescr",
|
||||
"download": "form[action*='download.php']::attr(action)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,239 @@
|
||||
{
|
||||
"id": "hdtime",
|
||||
"name": "HDTime",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/hdtime.yaml"
|
||||
},
|
||||
"domain": "https://hdtime.org/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "424",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr, table.torrents > tbody > tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded br"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "HDTime",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 5000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "其他",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"410": "游戏",
|
||||
"411": "其他",
|
||||
"414": "其他",
|
||||
"424": "电影"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"id": "hdvideo",
|
||||
"name": "HDVIDEO",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/hdvideo.yaml"
|
||||
},
|
||||
"domain": "https://hdvideo.top/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "407",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "span.tt-name a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "span.tt-name a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "span.tt-name a",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(5) span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "a[href*='dllist=1#seeders']"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "a[href*='dllist=1#leechers'], td:nth-child(8)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "a[href*='viewsnatches.php'] b"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_2up": 2,
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "span.tt-sub"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(2) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "HDVIDEO",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "剧集",
|
||||
"408": "音乐"
|
||||
},
|
||||
"loginSuccess": "a[href='logout.php']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td.rowfollow",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,254 @@
|
||||
{
|
||||
"id": "hhanclub",
|
||||
"name": "憨憨",
|
||||
"enabled": true,
|
||||
"schema": "NexusHhanclub",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/hhanclub.yaml",
|
||||
"overrides": "scripts/mp-sync/movieclaw-overrides/hhanclub.json"
|
||||
},
|
||||
"domain": "https://hhanclub.net/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "电视剧"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "综艺"
|
||||
},
|
||||
{
|
||||
"id": "407",
|
||||
"desc": "体育"
|
||||
},
|
||||
{
|
||||
"id": "412",
|
||||
"desc": "短剧"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "410",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "411",
|
||||
"desc": "MV"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "div.torrent-table-sub-info"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "div.torrent-title a.torrent-info-text-name"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "div.torrent-title a.torrent-info-text-name[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "div.torrent-title a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "div.torrent-manage a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "div.torrent-info-text-added"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "div.torrent-info-text-added > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "div.torrent-info-text-size"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "div.torrent-info-text-seeders"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "div.torrent-info-text-leechers"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "div.torrent-info-text-finished"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"span.promotion-tag-free": 0,
|
||||
"span.promotion-tag-free2up": 0,
|
||||
"span.promotion-tag-2xfree": 0,
|
||||
"span.promotion-tag-50": 0.5,
|
||||
"span.promotion-tag-50pctdown": 0.5,
|
||||
"span.promotion-tag-50pctdown2up": 0.5,
|
||||
"span.promotion-tag-30": 0.3,
|
||||
"span.promotion-tag-30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"span.promotion-tag-free2up": 2,
|
||||
"span.promotion-tag-50pctdown2up": 2,
|
||||
"span.promotion-tag-2up": 2,
|
||||
"span.promotion-tag-2xfree": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "div.torrent-info-text-small_name"
|
||||
},
|
||||
"category": {
|
||||
"selector": "div.torrent-cat a",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"hr": {
|
||||
"selector": "div.torrent-info-hr"
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "憨憨",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "电视剧",
|
||||
"403": "综艺",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"407": "体育",
|
||||
"409": "其他",
|
||||
"410": "音乐",
|
||||
"411": "MV",
|
||||
"412": "短剧"
|
||||
},
|
||||
"loginSuccess": "a[href='logout.php']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name']",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "img[alt='上传']::next_sibling_text",
|
||||
"downloaded_css": "img[alt='下载']::next_sibling_text",
|
||||
"ratio_css": "img[alt='分享率'] + div",
|
||||
"bonus_css": "img[alt='憨豆'] + a div",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img[alt='做种数']::next_sibling_text",
|
||||
"leeching_css": "img[alt='下载数']::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "div.font-bold.leading-6:contains('标题'):not(:contains('副标题')) + div.font-bold.leading-6",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "div.font-bold.leading-6:contains('标题'):not(:contains('副标题')) + div.font-bold.leading-6",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "div.font-bold.leading-6:contains('标题'):not(:contains('副标题')) + div.font-bold.leading-6",
|
||||
"subtitle": "div.font-bold.leading-6:contains('副标题') + div.font-bold.leading-6",
|
||||
"size": "span.font-bold:contains('大小') + span",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,16 +1,48 @@
|
||||
import type { IndexerDefinition } from "../spider/schema";
|
||||
import audiencesJson from "./audiences.json";
|
||||
/**
|
||||
* 站点定义加载与校验。
|
||||
* - `*.json`:由 scripts/mp-sync/movieclaw-import.mjs 从 MovieClaw v0.27.0(MIT,见 NOTICE-MovieClaw.md)转换,
|
||||
* 手工覆盖层在 scripts/mp-sync/movieclaw-overrides/;不要直接手改 JSON(重跑脚本会覆盖)。
|
||||
* - 加载时一律经 zod schema(./schema.ts)校验,失败即抛错。
|
||||
*/
|
||||
import type { MteamDefinition } from "../mteam/definition.ts";
|
||||
import type { IndexerDefinition } from "../spider/schema.ts";
|
||||
import type { SpecializedSiteDefinition } from "../spiders/types.ts";
|
||||
import { MOVIECLAW_DEFINITION_JSON } from "./manifest.generated.ts";
|
||||
import { parseIndexerDefinition, parseMteamDefinition, parseSpecializedSiteDefinition } from "./schema.ts";
|
||||
|
||||
/** 站点定义最小校验(随产品发布的定义文件;加站只需新增 JSON 并在此登记)。 */
|
||||
export type SiteDefinition = IndexerDefinition | MteamDefinition | SpecializedSiteDefinition;
|
||||
|
||||
/** 按 `umt.kind` 分派校验 */
|
||||
export function loadSiteDefinition(json: unknown): SiteDefinition {
|
||||
const kind = (json as { umt?: { kind?: unknown } } | null)?.umt?.kind;
|
||||
if (kind === "mteam") return parseMteamDefinition(json);
|
||||
if (kind === "nexusphp") return parseIndexerDefinition(json);
|
||||
if (kind === "api") return parseSpecializedSiteDefinition(json);
|
||||
throw new Error(`站点定义 ${(json as { id?: unknown })?.id ?? "<unknown>"} 的 umt.kind 未知:${String(kind)}`);
|
||||
}
|
||||
|
||||
/** 兼容 W1-3b 导出名:NexusPHP 定义校验 */
|
||||
export function loadIndexerDefinition(json: unknown): IndexerDefinition {
|
||||
const d = json as Partial<IndexerDefinition>;
|
||||
if (!d || typeof d !== "object") throw new Error("站点定义必须是对象");
|
||||
for (const k of ["id", "name", "domain"] as const) {
|
||||
if (typeof d[k] !== "string" || !d[k]) throw new Error(`站点定义缺少 ${k}`);
|
||||
}
|
||||
if (!d.torrents?.list?.selector || !d.torrents.fields) throw new Error(`站点定义 ${d.id} 缺少 torrents.list/fields`);
|
||||
if (!d.umt?.version || !d.umt.timezone || !d.umt.rateLimit) throw new Error(`站点定义 ${d.id} 缺少 umt 扩展`);
|
||||
return d as IndexerDefinition;
|
||||
return parseIndexerDefinition(json);
|
||||
}
|
||||
|
||||
export const audiencesDefinition: IndexerDefinition = loadIndexerDefinition(audiencesJson);
|
||||
/** 随产品发布的全部定义(已校验),键为站点 id */
|
||||
export const BUILTIN_SITE_DEFINITIONS: Readonly<Record<string, SiteDefinition>> = Object.freeze(
|
||||
Object.fromEntries(Object.entries(MOVIECLAW_DEFINITION_JSON).map(([id, json]) => [id, loadSiteDefinition(json)])),
|
||||
);
|
||||
|
||||
export const audiencesDefinition = BUILTIN_SITE_DEFINITIONS.audiences as IndexerDefinition;
|
||||
export const hhanclubDefinition = BUILTIN_SITE_DEFINITIONS.hhanclub as IndexerDefinition;
|
||||
export const chdbitsDefinition = BUILTIN_SITE_DEFINITIONS.chdbits as IndexerDefinition;
|
||||
|
||||
export {
|
||||
DefinitionProvenanceSchema,
|
||||
IndexerDefinitionSchema,
|
||||
MteamDefinitionSchema,
|
||||
parseIndexerDefinition,
|
||||
parseMteamDefinition,
|
||||
parseSpecializedSiteDefinition,
|
||||
SpecializedSiteDefinitionSchema,
|
||||
SpiderFieldSelectorSchema,
|
||||
SpiderFilterSchema,
|
||||
} from "./schema.ts";
|
||||
@@ -0,0 +1,261 @@
|
||||
{
|
||||
"id": "keepfrds",
|
||||
"name": "朋友",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/keepfrds.yaml"
|
||||
},
|
||||
"domain": "https://pt.keepfrds.com/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "407",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "420",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "421",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_nl": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_nl": 0,
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded > br"
|
||||
},
|
||||
"category": {
|
||||
"selector": "a[href^='?cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "朋友",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"300": "其他",
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "剧集",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"414": "其他",
|
||||
"415": "其他",
|
||||
"419": "成人",
|
||||
"420": "音乐",
|
||||
"421": "音乐",
|
||||
"422": "其他",
|
||||
"423": "游戏",
|
||||
"424": "其他",
|
||||
"425": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[href*='userdetails.php?id=']::attr(href)",
|
||||
"username_css": "a[href*='userdetails.php?id=']",
|
||||
"class_css": "a[href*='userdetails.php?id=']::attr(class)",
|
||||
"uploaded_css": "span.uploaded",
|
||||
"downloaded_css": "span.downloaded",
|
||||
"ratio_css": "td.rowhead:contains('分享率') + td::text",
|
||||
"bonus_css": "td.rowhead:contains('魔力') + td::text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "td.rowhead:contains('当前做种') + td::text",
|
||||
"leeching_css": "td.rowhead:contains('当前下载') + td::text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
// 由 scripts/mp-sync/movieclaw-import.mjs 生成,请勿手改(MovieClaw v0.27.0 c79bb1b0,MIT)
|
||||
import agsvptJson from "./agsvpt.json" with { type: "json" };
|
||||
import audiencesJson from "./audiences.json" with { type: "json" };
|
||||
import chdbitsJson from "./chdbits.json" with { type: "json" };
|
||||
import hdareaJson from "./hdarea.json" with { type: "json" };
|
||||
import hddolbyJson from "./hddolby.json" with { type: "json" };
|
||||
import hdfansJson from "./hdfans.json" with { type: "json" };
|
||||
import hdhomeJson from "./hdhome.json" with { type: "json" };
|
||||
import hdskyJson from "./hdsky.json" with { type: "json" };
|
||||
import hdtimeJson from "./hdtime.json" with { type: "json" };
|
||||
import hdvideoJson from "./hdvideo.json" with { type: "json" };
|
||||
import hhanclubJson from "./hhanclub.json" with { type: "json" };
|
||||
import keepfrdsJson from "./keepfrds.json" with { type: "json" };
|
||||
import mteamJson from "./mteam.json" with { type: "json" };
|
||||
import niceptJson from "./nicept.json" with { type: "json" };
|
||||
import ourbitsJson from "./ourbits.json" with { type: "json" };
|
||||
import piggoJson from "./piggo.json" with { type: "json" };
|
||||
import pterJson from "./pter.json" with { type: "json" };
|
||||
import pthomeJson from "./pthome.json" with { type: "json" };
|
||||
import pttimeJson from "./pttime.json" with { type: "json" };
|
||||
import soulvoiceJson from "./soulvoice.json" with { type: "json" };
|
||||
import ssdJson from "./ssd.json" with { type: "json" };
|
||||
import tjuptJson from "./tjupt.json" with { type: "json" };
|
||||
import ttgJson from "./ttg.json" with { type: "json" };
|
||||
|
||||
/** MovieClaw 转换来的站点定义原始 JSON(未校验;经 loadSiteDefinition 校验后使用) */
|
||||
export const MOVIECLAW_DEFINITION_JSON: Record<string, unknown> = {
|
||||
agsvpt: agsvptJson,
|
||||
audiences: audiencesJson,
|
||||
chdbits: chdbitsJson,
|
||||
hdarea: hdareaJson,
|
||||
hddolby: hddolbyJson,
|
||||
hdfans: hdfansJson,
|
||||
hdhome: hdhomeJson,
|
||||
hdsky: hdskyJson,
|
||||
hdtime: hdtimeJson,
|
||||
hdvideo: hdvideoJson,
|
||||
hhanclub: hhanclubJson,
|
||||
keepfrds: keepfrdsJson,
|
||||
mteam: mteamJson,
|
||||
nicept: niceptJson,
|
||||
ourbits: ourbitsJson,
|
||||
piggo: piggoJson,
|
||||
pter: pterJson,
|
||||
pthome: pthomeJson,
|
||||
pttime: pttimeJson,
|
||||
soulvoice: soulvoiceJson,
|
||||
ssd: ssdJson,
|
||||
tjupt: tjuptJson,
|
||||
ttg: ttgJson,
|
||||
};
|
||||
@@ -0,0 +1,72 @@
|
||||
{
|
||||
"id": "mteam",
|
||||
"name": "馒头",
|
||||
"enabled": true,
|
||||
"schema": "MTorrent",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/mteam.yaml",
|
||||
"overrides": "scripts/mp-sync/movieclaw-overrides/mteam.json"
|
||||
},
|
||||
"domain": "https://tp.m-team.cc/",
|
||||
"parser": "mTorrent",
|
||||
"timeout": 15,
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "mteam",
|
||||
"displayName": "馒头",
|
||||
"apiBaseUrl": "https://api.m-team.cc",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 1000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "综艺",
|
||||
"404": "纪录",
|
||||
"405": "动画",
|
||||
"406": "音乐",
|
||||
"407": "体育",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"410": "成人",
|
||||
"411": "成人",
|
||||
"412": "成人",
|
||||
"413": "成人",
|
||||
"419": "电影",
|
||||
"420": "电影",
|
||||
"421": "电影",
|
||||
"422": "其他",
|
||||
"423": "游戏",
|
||||
"424": "成人",
|
||||
"425": "成人",
|
||||
"426": "成人",
|
||||
"427": "其他",
|
||||
"429": "成人",
|
||||
"430": "成人",
|
||||
"431": "成人",
|
||||
"432": "成人",
|
||||
"433": "成人",
|
||||
"434": "音乐",
|
||||
"435": "剧集",
|
||||
"436": "成人",
|
||||
"437": "成人",
|
||||
"438": "剧集",
|
||||
"439": "电影"
|
||||
},
|
||||
"authSupported": ["apikey"],
|
||||
"customClass": "movieclaw_tracker.sites.custom.mteam.MTeamSite"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,215 @@
|
||||
{
|
||||
"id": "nicept",
|
||||
"name": "NicePT",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/nicept.yaml"
|
||||
},
|
||||
"domain": "https://www.nicept.net/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [],
|
||||
"tv": [],
|
||||
"music": []
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr, table.torrents > tbody > tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded br"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"hr": {
|
||||
"selector": "img.hitandrun"
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "NicePT",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 5000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "成人",
|
||||
"402": "成人",
|
||||
"403": "成人",
|
||||
"404": "成人",
|
||||
"500": "成人",
|
||||
"501": "成人",
|
||||
"503": "成人",
|
||||
"504": "成人"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副標題') + td::text",
|
||||
"size": "td.rowhead:contains('基本資訊') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,261 @@
|
||||
{
|
||||
"id": "ourbits",
|
||||
"name": "OurBits",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/ourbits.yaml"
|
||||
},
|
||||
"domain": "https://ourbits.club/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "405",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "412",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "413",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "415",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "416",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "414",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "419",
|
||||
"desc": "音乐"
|
||||
}
|
||||
],
|
||||
"param": "cat[]"
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded > div"
|
||||
},
|
||||
"category": {
|
||||
"selector": "a[href^='?cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"hr": {
|
||||
"selector": "img.hitandrun"
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "OurBits",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "电影",
|
||||
"405": "剧集",
|
||||
"410": "纪录片",
|
||||
"411": "动漫",
|
||||
"412": "剧集",
|
||||
"413": "剧集",
|
||||
"414": "音乐",
|
||||
"415": "剧集",
|
||||
"416": "音乐",
|
||||
"419": "音乐"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[href*='userdetails.php?id=']::attr(href)",
|
||||
"username_css": "a[href*='userdetails.php?id=']",
|
||||
"class_css": "a[href*='userdetails.php?id=']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "td.rowhead:contains('分享率') + td::text",
|
||||
"bonus_css": "td.rowhead:contains('魔力') + td::text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img[alt='Torrents seeding'] + *",
|
||||
"leeching_css": "img[alt='Torrents leeching'] + *"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
},
|
||||
"customClass": "movieclaw_tracker.sites.custom.ourbits.OurBitsSite"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,233 @@
|
||||
{
|
||||
"id": "piggo",
|
||||
"name": "猪猪网 Piggo",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/piggo.yaml"
|
||||
},
|
||||
"domain": "https://piggo.me/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr, table.torrents > tbody > tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded br"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "猪猪网 Piggo",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 8000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "其他",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"912": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,233 @@
|
||||
{
|
||||
"id": "pter",
|
||||
"name": "猫站",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/pter.yaml"
|
||||
},
|
||||
"domain": "https://pterclub.net/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "404",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "405",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "418",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"category": {
|
||||
"selector": "a[href^='?cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "猫站",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "纪录片",
|
||||
"403": "动漫",
|
||||
"404": "剧集",
|
||||
"405": "剧集",
|
||||
"406": "音乐",
|
||||
"409": "游戏",
|
||||
"418": "音乐"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[href*='userdetails.php?id=']::attr(href)",
|
||||
"username_css": "a[href*='userdetails.php?id=']",
|
||||
"class_css": "a[href*='userdetails.php?id=']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "td.rowhead:contains('魔力') + td::text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img[alt='Torrents seeding'] + *",
|
||||
"leeching_css": "img[alt='Torrents leeching'] + *"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,239 @@
|
||||
{
|
||||
"id": "pthome",
|
||||
"name": "铂金家",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/pthome.yaml"
|
||||
},
|
||||
"domain": "https://pthome.net/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "407",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded > span:not([class])"
|
||||
},
|
||||
"category": {
|
||||
"selector": "a[href^='?cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "铂金家",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"407": "剧集",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"410": "游戏",
|
||||
"411": "其他",
|
||||
"412": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[href*='userdetails.php?id=']::attr(href)",
|
||||
"username_css": "a[href*='userdetails.php?id=']",
|
||||
"class_css": "a[href*='userdetails.php?id=']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus::next_sibling_text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img[alt='Torrents seeding'] + *",
|
||||
"leeching_css": "img[alt='Torrents leeching'] + *"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"id": "pttime",
|
||||
"name": "PTTime",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/pttime.yaml"
|
||||
},
|
||||
"domain": "https://www.pttime.org/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr, table.torrents > tbody > tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(5) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(9)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"font.promotion.free": 0,
|
||||
"font.promotion.twoupfree": 0,
|
||||
"font.promotion.halfdown": 0.5,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"font.promotion.twoupfree": 2,
|
||||
"font.promotion.twoup": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded font[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "PTTime",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 5000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"100": "其他",
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "其他",
|
||||
"406": "动漫",
|
||||
"407": "其他",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"411": "其他",
|
||||
"412": "其他",
|
||||
"420": "其他",
|
||||
"421": "游戏",
|
||||
"422": "其他",
|
||||
"423": "其他",
|
||||
"430": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "#info_block span.medium a[href*='userdetails.php']::attr(href)",
|
||||
"username_css": "#info_block span.medium a[href*='userdetails.php'] b",
|
||||
"class_css": "td.rowhead:contains('用户等级') + td b::text",
|
||||
"uploaded_css": "#info_block font:contains('上传:')::next_sibling_text",
|
||||
"downloaded_css": "#info_block font:contains('下载:')::next_sibling_text",
|
||||
"ratio_css": "#info_block font:contains('分享率:')::next_sibling_text",
|
||||
"bonus_css": "#info_block a[href='mybonus.php']::next_sibling_text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "#info_block font[title='当前做种']::next_sibling_text",
|
||||
"leeching_css": "#info_block font[title='当前下载']::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,248 @@
|
||||
/**
|
||||
* 站点定义 zod schema(加载时校验)。
|
||||
*
|
||||
* 键名按 MoviePilot e6820a8 `app/application/site/sites.pyi` `SitesHelper.get_indexer(domain) -> dict` 与
|
||||
* `app/modules/indexer/spider/__init__.py` `SiteSpider` / 专用 spider 读取的 indexer dict 键整理
|
||||
* (id、name、domain、encoding、public、proxy、language、parser、result_num、timeout、media_type、search、batch、
|
||||
* browse、category、torrents),MP 的真实定义在加密 user.sites.v3.bin 中不可读;
|
||||
* `enabled` / `schema`(MP 用户信息解析器 SiteSchema)/ `provenance` / `umt` 为本项目扩展。
|
||||
* TS 类型仍以 spider/schema.ts `IndexerDefinition`、mteam/definition.ts `MteamDefinition` 为准,
|
||||
* 本文件保证 JSON 与之一致(见 schema.test.ts)。
|
||||
*/
|
||||
import { z } from "zod";
|
||||
import type { MteamDefinition } from "../mteam/definition.ts";
|
||||
import type { IndexerDefinition } from "../spider/schema.ts";
|
||||
import type { SpecializedSiteDefinition } from "../spiders/types.ts";
|
||||
import { SiteSchema } from "../user-parsers/types.ts";
|
||||
|
||||
const SITE_SCHEMA_VALUES = Object.values(SiteSchema) as [string, ...string[]];
|
||||
|
||||
export const DefinitionProvenanceSchema = z.strictObject({
|
||||
source: z.enum(["movieclaw", "handwritten"]),
|
||||
ref: z.string().optional(),
|
||||
commit: z.string().optional(),
|
||||
license: z.string().optional(),
|
||||
file: z.string().optional(),
|
||||
overrides: z.string().optional(),
|
||||
note: z.string().optional(),
|
||||
});
|
||||
|
||||
export const SpiderFilterSchema = z.discriminatedUnion("name", [
|
||||
z.strictObject({ name: z.literal("re_search"), args: z.tuple([z.string(), z.number().int()]) }),
|
||||
z.strictObject({ name: z.literal("split"), args: z.tuple([z.string(), z.number().int()]) }),
|
||||
z.strictObject({ name: z.literal("replace"), args: z.tuple([z.string(), z.string()]) }),
|
||||
z.strictObject({ name: z.literal("dateparse"), args: z.string() }),
|
||||
z.strictObject({ name: z.literal("strip"), args: z.unknown().optional() }),
|
||||
z.strictObject({ name: z.literal("appendleft"), args: z.string() }),
|
||||
z.strictObject({ name: z.literal("querystring"), args: z.string() }),
|
||||
]);
|
||||
|
||||
export const SpiderFieldSelectorSchema = z.strictObject({
|
||||
selector: z.string().min(1).optional(),
|
||||
selectors: z.string().min(1).optional(),
|
||||
attribute: z.string().min(1).optional(),
|
||||
index: z.number().int().optional(),
|
||||
contents: z.number().int().optional(),
|
||||
remove: z.string().optional(),
|
||||
filters: z.array(SpiderFilterSchema).optional(),
|
||||
text: z.string().optional(),
|
||||
case: z.record(z.string(), z.number()).optional(),
|
||||
});
|
||||
|
||||
const CategoryItemSchema = z.strictObject({
|
||||
id: z.union([z.string(), z.number()]),
|
||||
cat: z.string().optional(),
|
||||
desc: z.string().optional(),
|
||||
param: z.string().optional(),
|
||||
value: z.union([z.string(), z.number()]).optional(),
|
||||
params: z.record(z.string(), z.union([z.string(), z.number()])).optional(),
|
||||
});
|
||||
|
||||
const CategoryConfigSchema = z.strictObject({
|
||||
movie: z.array(CategoryItemSchema).optional(),
|
||||
tv: z.array(CategoryItemSchema).optional(),
|
||||
music: z.array(CategoryItemSchema).optional(),
|
||||
field: z.string().optional(),
|
||||
delimiter: z.string().optional(),
|
||||
param: z.string().optional(),
|
||||
});
|
||||
|
||||
const CapabilitiesSchema = z.strictObject({
|
||||
list: z.boolean(),
|
||||
search: z.boolean(),
|
||||
detail: z.boolean(),
|
||||
download: z.boolean(),
|
||||
rss: z.boolean(),
|
||||
});
|
||||
const RateLimitSchema = z.strictObject({ minIntervalMs: z.number().int().min(0), burst: z.number().int().min(1) });
|
||||
const TimezoneSchema = z.string().regex(/^[+-]\d{2}:\d{2}$/, "时区须为 ±HH:MM 偏移");
|
||||
const PairSchema = z.tuple([z.string(), z.string()]);
|
||||
|
||||
const UmtNexusSchema = z.strictObject({
|
||||
version: z.string().min(1),
|
||||
kind: z.literal("nexusphp"),
|
||||
displayName: z.string().optional(),
|
||||
customClass: z.string().optional(),
|
||||
hrSeedHours: z.number().positive().optional(),
|
||||
authSupported: z.array(z.enum(["cookie", "credential", "apikey"])).optional(),
|
||||
timezone: TimezoneSchema,
|
||||
rateLimit: RateLimitSchema,
|
||||
capabilities: CapabilitiesSchema,
|
||||
categoryNames: z.record(z.string(), z.string()).optional(),
|
||||
loginSuccess: z.string().optional(),
|
||||
emptyMarkers: z.array(z.string()).optional(),
|
||||
pagination: z.string().optional(),
|
||||
login: z
|
||||
.strictObject({
|
||||
path: z.string(),
|
||||
usernameField: z.string(),
|
||||
passwordField: z.string(),
|
||||
errorCss: z.string(),
|
||||
extraFormData: z.array(PairSchema).optional(),
|
||||
selectDefaults: z.array(PairSchema).optional(),
|
||||
})
|
||||
.optional(),
|
||||
profile: z.record(z.string(), z.string()).optional(),
|
||||
detail: z
|
||||
.strictObject({
|
||||
path: z.string(),
|
||||
title: z.string(),
|
||||
rowLabel: z.string(),
|
||||
rowValue: z.string(),
|
||||
labels: z.strictObject({
|
||||
subtitle: z.array(z.string()),
|
||||
basicInfo: z.array(z.string()),
|
||||
peers: z.array(z.string()),
|
||||
}),
|
||||
description: z.string(),
|
||||
headerScope: z.string(),
|
||||
notFoundMarkers: z.array(z.string()),
|
||||
fields: z
|
||||
.strictObject({
|
||||
title: z.string().optional(),
|
||||
subtitle: z.string().optional(),
|
||||
size: z.string().optional(),
|
||||
description: z.string().optional(),
|
||||
download: z.string().optional(),
|
||||
imdb: z.string().optional(),
|
||||
douban: z.string().optional(),
|
||||
fileList: z.string().optional(),
|
||||
})
|
||||
.optional(),
|
||||
})
|
||||
.optional(),
|
||||
downloadPath: z.string().min(1),
|
||||
detailPath: z.string().min(1),
|
||||
});
|
||||
|
||||
const BaseDefinitionShape = {
|
||||
id: z.string().regex(/^[a-z0-9_]+$/, "id 只能是小写字母/数字/下划线"),
|
||||
name: z.string().min(1),
|
||||
enabled: z.boolean().optional(),
|
||||
schema: z.enum(SITE_SCHEMA_VALUES).optional(),
|
||||
provenance: DefinitionProvenanceSchema.optional(),
|
||||
domain: z
|
||||
.string()
|
||||
.url()
|
||||
.refine((s) => s.endsWith("/"), "domain 须以 / 结尾"),
|
||||
timeout: z.number().positive().optional(),
|
||||
proxy: z.boolean().optional(),
|
||||
};
|
||||
|
||||
const FieldsSchema = z.record(z.string(), SpiderFieldSelectorSchema);
|
||||
|
||||
export const IndexerDefinitionSchema = z.strictObject({
|
||||
...BaseDefinitionShape,
|
||||
encoding: z.string().optional(),
|
||||
public: z.boolean().optional(),
|
||||
language: z.string().optional(),
|
||||
parser: z.string().optional(),
|
||||
result_num: z.number().int().positive().optional(),
|
||||
media_type: z.string().optional(),
|
||||
search: z
|
||||
.strictObject({
|
||||
paths: z.array(z.strictObject({ path: z.string(), type: z.enum(["all", "movie", "tv", "music"]).optional() })),
|
||||
params: z.record(z.string(), z.union([z.string(), z.number()])).optional(),
|
||||
imdbid_format: z.string().optional(),
|
||||
result_media_type: z.literal("requested").optional(),
|
||||
})
|
||||
.optional(),
|
||||
batch: z.strictObject({ delimiter: z.string().optional(), space_replace: z.string().optional() }).optional(),
|
||||
browse: z
|
||||
.strictObject({
|
||||
path: z.string(),
|
||||
start: z.number().int().optional(),
|
||||
list: z.strictObject({ selector: z.string() }).optional(),
|
||||
fields: FieldsSchema.optional(),
|
||||
})
|
||||
.optional(),
|
||||
category: CategoryConfigSchema.optional(),
|
||||
torrents: z.strictObject({
|
||||
list: z.strictObject({ selector: z.string().min(1) }),
|
||||
fields: FieldsSchema.refine((f) => Boolean(f.title || f.title_default), "torrents.fields 缺少 title"),
|
||||
}),
|
||||
umt: UmtNexusSchema,
|
||||
});
|
||||
|
||||
export const MteamDefinitionSchema = z.strictObject({
|
||||
...BaseDefinitionShape,
|
||||
parser: z.literal("mTorrent"),
|
||||
timeout: z.number().positive(),
|
||||
umt: z.strictObject({
|
||||
version: z.string().min(1),
|
||||
kind: z.literal("mteam"),
|
||||
displayName: z.string(),
|
||||
apiBaseUrl: z.string().url().optional(),
|
||||
timezone: TimezoneSchema,
|
||||
rateLimit: RateLimitSchema,
|
||||
capabilities: CapabilitiesSchema,
|
||||
categoryNames: z.record(z.string(), z.string()),
|
||||
authSupported: z.array(z.enum(["cookie", "credential", "apikey"])).optional(),
|
||||
customClass: z.string().optional(),
|
||||
}),
|
||||
});
|
||||
|
||||
/** MP 专用 spider(`SPIDER_PARSER_CLASSES` 中除 mTorrent 外的 7 个)站点定义 */
|
||||
export const SpecializedSiteDefinitionSchema = z.strictObject({
|
||||
...BaseDefinitionShape,
|
||||
parser: z.enum(["Haidan", "HDDolby", "RousiPro", "SunnyPT", "TNodeSpider", "TorrentLeech", "Yema"]),
|
||||
public: z.boolean().optional(),
|
||||
category: CategoryConfigSchema.optional(),
|
||||
api_url: z.string().url().optional(),
|
||||
umt: z.strictObject({
|
||||
version: z.string().min(1),
|
||||
kind: z.literal("api"),
|
||||
displayName: z.string(),
|
||||
timezone: TimezoneSchema,
|
||||
rateLimit: RateLimitSchema,
|
||||
capabilities: CapabilitiesSchema,
|
||||
categoryNames: z.record(z.string(), z.string()).optional(),
|
||||
apiBaseUrl: z.string().url().optional(),
|
||||
}),
|
||||
});
|
||||
|
||||
function formatIssues(id: unknown, e: z.ZodError): string {
|
||||
const issues = e.issues.map((i) => `${i.path.join(".") || "<root>"}: ${i.message}`).join("; ");
|
||||
return `站点定义 ${typeof id === "string" ? id : "<unknown>"} 校验失败:${issues}`;
|
||||
}
|
||||
|
||||
/** NexusPHP(通用 SiteSpider)定义校验 */
|
||||
export function parseIndexerDefinition(json: unknown): IndexerDefinition {
|
||||
const r = IndexerDefinitionSchema.safeParse(json);
|
||||
if (!r.success) throw new Error(formatIssues((json as { id?: unknown })?.id, r.error));
|
||||
return r.data as IndexerDefinition;
|
||||
}
|
||||
|
||||
/** 馒头(M-Team API)定义校验 */
|
||||
export function parseMteamDefinition(json: unknown): MteamDefinition {
|
||||
const r = MteamDefinitionSchema.safeParse(json);
|
||||
if (!r.success) throw new Error(formatIssues((json as { id?: unknown })?.id, r.error));
|
||||
return r.data as MteamDefinition;
|
||||
}
|
||||
|
||||
/** 专用 API spider 定义校验 */
|
||||
export function parseSpecializedSiteDefinition(json: unknown): SpecializedSiteDefinition {
|
||||
const r = SpecializedSiteDefinitionSchema.safeParse(json);
|
||||
if (!r.success) throw new Error(formatIssues((json as { id?: unknown })?.id, r.error));
|
||||
return r.data as SpecializedSiteDefinition;
|
||||
}
|
||||
@@ -0,0 +1,238 @@
|
||||
{
|
||||
"id": "soulvoice",
|
||||
"name": "聆音",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/soulvoice.yaml"
|
||||
},
|
||||
"domain": "https://pt.soulvoice.club/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "401",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "424",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "402",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "403",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "406",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "408",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr, table.torrents > tbody > tr"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "table.torrentname a b"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "table.torrentname a b[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td:nth-child(4)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(4) > span",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded br"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) a[href*='cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "聆音",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 5000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"401": "电影",
|
||||
"402": "剧集",
|
||||
"403": "剧集",
|
||||
"404": "纪录片",
|
||||
"405": "动漫",
|
||||
"406": "音乐",
|
||||
"407": "其他",
|
||||
"408": "音乐",
|
||||
"409": "其他",
|
||||
"424": "电影",
|
||||
"499": "其他",
|
||||
"500": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href*='logout']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[class*='_Name']::attr(href)",
|
||||
"username_css": "a[class*='_Name'] b",
|
||||
"class_css": "a[class*='_Name']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "font.color_bonus",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img.arrowup::next_sibling_text",
|
||||
"leeching_css": "img.arrowdown::next_sibling_text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "h1#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "h1#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "h1#top",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('基本信息') + td.rowfollow > b::next_sibling_text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,252 @@
|
||||
{
|
||||
"id": "ssd",
|
||||
"name": "不可说",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/ssd.yaml"
|
||||
},
|
||||
"domain": "https://springsunday.net/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 0
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "501",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "502",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "505",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "506",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "507",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "508",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:has(table.torrentname)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_elapsed": {
|
||||
"selector": "td.rowfollow:nth-last-child(6)"
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td.rowfollow:nth-last-child(6) > span[title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td.rowfollow:nth-last-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td.rowfollow:nth-last-child(4)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td.rowfollow:nth-last-child(3)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td.rowfollow:nth-last-child(2)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"span.torrent-pro-free": 0,
|
||||
"span.torrent-pro-free2up": 0,
|
||||
"span.torrent-pro-50pctdown": 0.5,
|
||||
"span.torrent-pro-50pctdown2up": 0.5,
|
||||
"span.torrent-pro-30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"span.torrent-pro-free2up": 2,
|
||||
"span.torrent-pro-50pctdown2up": 2,
|
||||
"span.torrent-pro-2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "div.torrent-smalldescr > span:not([class])"
|
||||
},
|
||||
"category": {
|
||||
"selector": "a[href^='?cat=']",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"freedate": {
|
||||
"selector": "span.torrent-pro-free ~ span > span[title]",
|
||||
"attribute": "title",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "不可说",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"501": "电影",
|
||||
"502": "剧集",
|
||||
"503": "纪录片",
|
||||
"504": "动漫",
|
||||
"505": "剧集",
|
||||
"506": "剧集",
|
||||
"507": "音乐",
|
||||
"508": "音乐",
|
||||
"509": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href^='logout.php']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a[href^='userdetails.php?id=']::attr(href)",
|
||||
"username_css": "a[href^='userdetails.php?id=']",
|
||||
"class_css": "a[href^='userdetails.php?id=']::attr(class)",
|
||||
"uploaded_css": "font.color_uploaded::next_sibling_text",
|
||||
"downloaded_css": "font.color_downloaded::next_sibling_text",
|
||||
"ratio_css": "font.color_ratio::next_sibling_text",
|
||||
"bonus_css": "td.rowhead:contains('魔力') + td::text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "img[alt='Torrents seeding'] + *",
|
||||
"leeching_css": "img[alt='Torrents leeching'] + *",
|
||||
"vip_css": "a[class^='VIP']"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
{
|
||||
"id": "tjupt",
|
||||
"name": "北洋园",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/tjupt.yaml"
|
||||
},
|
||||
"domain": "https://tjupt.org/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "torrents.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "torrents.php?page={page}",
|
||||
"start": 1
|
||||
},
|
||||
"category": {
|
||||
"movie": [],
|
||||
"tv": [],
|
||||
"music": []
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table.torrents > tr:not(:first-child)"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": ":scope > td:nth-child(2) a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": ":scope > td:nth-child(2) a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_added": {
|
||||
"selector": ":scope > td:nth-child(4)"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": ":scope > td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": ":scope > td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": ":scope > td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": ":scope > td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free": 0,
|
||||
"img.pro_free2up": 0,
|
||||
"img.pro_50pctdown": 0.5,
|
||||
"img.pro_50pctdown2up": 0.5,
|
||||
"img.pro_30pctdown": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img.pro_free2up": 2,
|
||||
"img.pro_50pctdown2up": 2,
|
||||
"img.pro_2up": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded > span"
|
||||
},
|
||||
"category": {
|
||||
"selector": ":scope > td:nth-child(1) a",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "北洋园",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 5000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {},
|
||||
"loginSuccess": "a[href*='logout.php']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"authSupported": ["cookie", "credential"],
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text",
|
||||
"selectDefaults": [["logout", "7"]]
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"uid_css": "a.User_Name::attr(href)",
|
||||
"username_css": "a.User_Name b",
|
||||
"class_css": "td.rowhead:contains('等级') + td img::attr(title)",
|
||||
"uploaded_css": "td.rowhead:contains('上传量') + td::text",
|
||||
"bonus_css": "td.rowhead:contains('魔力值') + td",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,340 @@
|
||||
{
|
||||
"id": "ttg",
|
||||
"name": "听听歌",
|
||||
"enabled": false,
|
||||
"schema": "NexusPhp",
|
||||
"provenance": {
|
||||
"source": "movieclaw",
|
||||
"ref": "v0.27.0",
|
||||
"commit": "c79bb1b0",
|
||||
"license": "MIT",
|
||||
"file": "src/movieclaw_tracker/sites/configs/ttg.yaml"
|
||||
},
|
||||
"domain": "https://totheglory.im/",
|
||||
"encoding": "UTF-8",
|
||||
"public": false,
|
||||
"proxy": false,
|
||||
"language": "zh",
|
||||
"result_num": 100,
|
||||
"timeout": 30,
|
||||
"search": {
|
||||
"paths": [
|
||||
{
|
||||
"path": "browse.php",
|
||||
"type": "all"
|
||||
}
|
||||
],
|
||||
"params": {
|
||||
"search": "{keyword}"
|
||||
}
|
||||
},
|
||||
"batch": {
|
||||
"delimiter": " ",
|
||||
"space_replace": "_"
|
||||
},
|
||||
"browse": {
|
||||
"path": "browse.php?page={page}",
|
||||
"start": 1
|
||||
},
|
||||
"category": {
|
||||
"movie": [
|
||||
{
|
||||
"id": "电影DVDRip",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "电影720p",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "电影1080i/p",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "BluRay原盘",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "影视2160p",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "UHD原盘",
|
||||
"desc": "电影"
|
||||
},
|
||||
{
|
||||
"id": "iPhone/iPad视频",
|
||||
"desc": "电影"
|
||||
}
|
||||
],
|
||||
"tv": [
|
||||
{
|
||||
"id": "欧美剧720p",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "欧美剧1080i/p",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "高清日剧",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "大陆港台剧720p",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "高清韩剧",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "大陆港台剧1080i/p",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "欧美剧包",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "日剧包",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "华语剧包",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "韩剧包",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "高清体育节目",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "韩国综艺",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "日本综艺",
|
||||
"desc": "剧集"
|
||||
},
|
||||
{
|
||||
"id": "高清综艺",
|
||||
"desc": "剧集"
|
||||
}
|
||||
],
|
||||
"music": [
|
||||
{
|
||||
"id": "(电影原声&Game)OST",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "无损音乐FLAC&APE",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "MV&演唱会",
|
||||
"desc": "音乐"
|
||||
},
|
||||
{
|
||||
"id": "补充音轨",
|
||||
"desc": "音乐"
|
||||
}
|
||||
]
|
||||
},
|
||||
"torrents": {
|
||||
"list": {
|
||||
"selector": "table#torrent_table > tr[id]"
|
||||
},
|
||||
"fields": {
|
||||
"title_default": {
|
||||
"selector": "a[href*='details.php']"
|
||||
},
|
||||
"title_optional": {
|
||||
"selector": "a[href*='details.php'][title]",
|
||||
"attribute": "title"
|
||||
},
|
||||
"title": {
|
||||
"text": "{% if fields['title_optional'] %}{{ fields['title_optional'] }}{% else %}{{ fields['title_default'] }}{% endif %}"
|
||||
},
|
||||
"details": {
|
||||
"selector": "a[href*='details.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"download": {
|
||||
"selector": "a[href*='download.php']",
|
||||
"attribute": "href"
|
||||
},
|
||||
"imdbid": {
|
||||
"selector": "a[href*=\"imdb.com/title/tt\"]",
|
||||
"attribute": "href",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["tt\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"date_added": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"date": {
|
||||
"text": "{% if fields['date_elapsed'] or fields['date_added'] %}{{ fields['date_added'] if fields['date_added'] else fields['date_elapsed'] }}{% else %}now{% endif %}",
|
||||
"filters": [
|
||||
{
|
||||
"name": "dateparse",
|
||||
"args": "%Y-%m-%d %H:%M:%S"
|
||||
}
|
||||
]
|
||||
},
|
||||
"size": {
|
||||
"selector": "td:nth-child(5)"
|
||||
},
|
||||
"seeders": {
|
||||
"selector": "td:nth-child(6)"
|
||||
},
|
||||
"leechers": {
|
||||
"selector": "td:nth-child(7)"
|
||||
},
|
||||
"grabs": {
|
||||
"selector": "td:nth-child(8)"
|
||||
},
|
||||
"downloadvolumefactor": {
|
||||
"case": {
|
||||
"img[alt=\"free\"]": 0,
|
||||
"img[alt=\"50%\"]": 0.5,
|
||||
"img[alt=\"30%\"]": 0.3,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"uploadvolumefactor": {
|
||||
"case": {
|
||||
"img[alt=\"200%\"]": 2,
|
||||
"*": 1
|
||||
}
|
||||
},
|
||||
"description": {
|
||||
"selector": "td.embedded > span"
|
||||
},
|
||||
"category": {
|
||||
"selector": "td:nth-child(1) img",
|
||||
"attribute": "class",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+", 0]
|
||||
}
|
||||
]
|
||||
},
|
||||
"freedate": {
|
||||
"selector": "span[onclick]",
|
||||
"attribute": "onclick",
|
||||
"filters": [
|
||||
{
|
||||
"name": "re_search",
|
||||
"args": ["\\d+年\\d+月\\d+日\\d+点\\d+分", 0]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"umt": {
|
||||
"version": "[email protected]",
|
||||
"kind": "nexusphp",
|
||||
"displayName": "听听歌",
|
||||
"timezone": "+08:00",
|
||||
"rateLimit": {
|
||||
"minIntervalMs": 3000,
|
||||
"burst": 1
|
||||
},
|
||||
"capabilities": {
|
||||
"list": true,
|
||||
"search": true,
|
||||
"detail": true,
|
||||
"download": true,
|
||||
"rss": false
|
||||
},
|
||||
"categoryNames": {
|
||||
"电影DVDRip": "电影",
|
||||
"电影720p": "电影",
|
||||
"电影1080i/p": "电影",
|
||||
"BluRay原盘": "电影",
|
||||
"影视2160p": "电影",
|
||||
"UHD原盘": "电影",
|
||||
"iPhone/iPad视频": "电影",
|
||||
"纪录片720p": "纪录片",
|
||||
"纪录片1080i/p": "纪录片",
|
||||
"纪录片BluRay原盘": "纪录片",
|
||||
"欧美剧720p": "剧集",
|
||||
"欧美剧1080i/p": "剧集",
|
||||
"高清日剧": "剧集",
|
||||
"大陆港台剧720p": "剧集",
|
||||
"高清韩剧": "剧集",
|
||||
"大陆港台剧1080i/p": "剧集",
|
||||
"欧美剧包": "剧集",
|
||||
"日剧包": "剧集",
|
||||
"华语剧包": "剧集",
|
||||
"韩剧包": "剧集",
|
||||
"高清体育节目": "剧集",
|
||||
"韩国综艺": "剧集",
|
||||
"日本综艺": "剧集",
|
||||
"高清综艺": "剧集",
|
||||
"(电影原声&Game)OST": "音乐",
|
||||
"无损音乐FLAC&APE": "音乐",
|
||||
"MV&演唱会": "音乐",
|
||||
"补充音轨": "音乐",
|
||||
"高清动漫": "动漫",
|
||||
"MiniVideo": "其他"
|
||||
},
|
||||
"loginSuccess": "a[href='/logout.php']",
|
||||
"emptyMarkers": ["没有种子", "没有结果", "Nothing found"],
|
||||
"pagination": "a[href*=\"page=\"]",
|
||||
"detailPath": "details.php",
|
||||
"downloadPath": "download.php",
|
||||
"login": {
|
||||
"path": "login.php",
|
||||
"usernameField": "username",
|
||||
"passwordField": "password",
|
||||
"errorCss": "td.text"
|
||||
},
|
||||
"profile": {
|
||||
"path": "userdetails.php",
|
||||
"username_css": "h1::text",
|
||||
"class_css": "td.rowhead:contains('等级') + td::text",
|
||||
"uploaded_css": "td.rowhead:contains('上传量') + td::text",
|
||||
"downloaded_css": "td.rowhead:contains('下载量') + td::text",
|
||||
"ratio_css": "td.rowhead:contains('分享率') + td::text",
|
||||
"bonus_css": "td.rowhead:contains('魔力') + td::text",
|
||||
"join_date_css": "td.rowhead:contains('加入日期') + td::text",
|
||||
"seeding_css": "td.rowhead:contains('当前做种') + td::text",
|
||||
"leeching_css": "td.rowhead:contains('当前下载') + td::text"
|
||||
},
|
||||
"detail": {
|
||||
"path": "details.php",
|
||||
"title": "#top",
|
||||
"rowLabel": "td.rowhead",
|
||||
"rowValue": "td.rowfollow",
|
||||
"labels": {
|
||||
"subtitle": ["副标题", "Small Description"],
|
||||
"basicInfo": ["基本信息", "Basic Info"],
|
||||
"peers": ["同伴", "Peers"]
|
||||
},
|
||||
"description": "#kdescr",
|
||||
"headerScope": "#top",
|
||||
"notFoundMarkers": ["没有该ID的种子", "No torrent with ID"],
|
||||
"fields": {
|
||||
"title": "#top::text",
|
||||
"subtitle": "td.rowhead:contains('副标题') + td::text",
|
||||
"size": "td.rowhead:contains('大小') + td::text",
|
||||
"description": "#kdescr",
|
||||
"download": "a[href*='download.php']::attr(href)",
|
||||
"imdb": "a[href*='imdb.com']::attr(href)",
|
||||
"douban": "a[href*='douban.com']::attr(href)",
|
||||
"fileList": "td.rowfollow > ul > li::text"
|
||||
}
|
||||
},
|
||||
"customClass": "movieclaw_tracker.sites.custom.ttg.TTGSite"
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { SiteHttpContext } from "./http";
|
||||
import { createSiteRateLimiter, type SiteRateLimiter } from "./rate-limit";
|
||||
import type { SecretResolver } from "./types";
|
||||
import type { SiteHttpContext } from "./http.ts";
|
||||
import { createSiteRateLimiter, type SiteRateLimiter } from "./rate-limit.ts";
|
||||
import type { SecretResolver } from "./types.ts";
|
||||
|
||||
export interface SiteAdapterDeps {
|
||||
secrets: SecretResolver;
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { redactText } from "../rss/redact";
|
||||
import type { SiteRateLimiter } from "./rate-limit";
|
||||
import { SiteError } from "./types";
|
||||
import { redactText } from "../rss/redact.ts";
|
||||
import type { SiteRateLimiter } from "./rate-limit.ts";
|
||||
import { SiteError } from "./types.ts";
|
||||
|
||||
export interface SiteHttpContext {
|
||||
siteId: string;
|
||||
|
||||
@@ -1,8 +1,24 @@
|
||||
export { audiencesDefinition, loadIndexerDefinition } from "./definitions";
|
||||
export type { SiteAdapterDeps } from "./deps";
|
||||
export { createMteamAdapter, mteamTorrentToRelease } from "./mteam/adapter";
|
||||
export { type MteamApiPath, mteamPost } from "./mteam/client";
|
||||
export { type MteamDefinition, mteamApiBase, mteamDefinition } from "./mteam/definition";
|
||||
export {
|
||||
audiencesDefinition,
|
||||
BUILTIN_SITE_DEFINITIONS,
|
||||
chdbitsDefinition,
|
||||
DefinitionProvenanceSchema,
|
||||
hhanclubDefinition,
|
||||
IndexerDefinitionSchema,
|
||||
loadIndexerDefinition,
|
||||
loadSiteDefinition,
|
||||
MteamDefinitionSchema,
|
||||
parseIndexerDefinition,
|
||||
parseMteamDefinition,
|
||||
parseSpecializedSiteDefinition,
|
||||
SpecializedSiteDefinitionSchema,
|
||||
SpiderFieldSelectorSchema,
|
||||
SpiderFilterSchema,
|
||||
} from "./definitions/index.ts";
|
||||
export type { SiteAdapterDeps } from "./deps.ts";
|
||||
export { createMteamAdapter, mteamTorrentToRelease } from "./mteam/adapter.ts";
|
||||
export { type MteamApiPath, mteamPost } from "./mteam/client.ts";
|
||||
export { type MteamDefinition, mteamApiBase, mteamDefinition } from "./mteam/definition.ts";
|
||||
export {
|
||||
findImdbid,
|
||||
getDownloadVolumeFactor,
|
||||
@@ -10,13 +26,22 @@ export {
|
||||
type MteamTorrentJson,
|
||||
mteamSearchParams,
|
||||
parseMteamResult,
|
||||
} from "./mteam/spider";
|
||||
export { createSiteRateLimiter, type RateLimiterOptions, type SiteRateLimiter } from "./rate-limit";
|
||||
export { createSiteAdapter, getSiteAdapter, SITE_DEFINITIONS, type SiteDefinition } from "./registry";
|
||||
export { clearSearchText, createSpiderSiteAdapter } from "./spider/adapter";
|
||||
export { parseDetailPage } from "./spider/detail";
|
||||
export { promotionFromFactors, spiderTorrentToRelease } from "./spider/map";
|
||||
} from "./mteam/spider.ts";
|
||||
export { createSiteRateLimiter, type RateLimiterOptions, type SiteRateLimiter } from "./rate-limit.ts";
|
||||
export {
|
||||
createSiteAdapter,
|
||||
getSiteAdapter,
|
||||
getSiteDefinition,
|
||||
listSiteDefinitions,
|
||||
SITE_DEFINITIONS,
|
||||
type SiteDefinition,
|
||||
} from "./registry.ts";
|
||||
export { clearSearchText, createSpiderSiteAdapter } from "./spider/adapter.ts";
|
||||
export { parseDetailPage } from "./spider/detail.ts";
|
||||
export { promotionFromFactors, spiderTorrentToRelease } from "./spider/map.ts";
|
||||
export { mcSelect } from "./spider/mc-selector.ts";
|
||||
export type {
|
||||
DefinitionProvenance,
|
||||
IndexerDefinition,
|
||||
SpiderCategoryConfig,
|
||||
SpiderCategoryItem,
|
||||
@@ -24,7 +49,7 @@ export type {
|
||||
SpiderFields,
|
||||
SpiderFilter,
|
||||
UmtSiteExtension,
|
||||
} from "./spider/schema";
|
||||
} from "./spider/schema.ts";
|
||||
export {
|
||||
combineUrl,
|
||||
type MediaKind,
|
||||
@@ -35,7 +60,8 @@ export {
|
||||
type SiteSpiderQuery,
|
||||
type SpiderTorrent,
|
||||
selectMediaCategories,
|
||||
} from "./spider/spider";
|
||||
} from "./spider/spider.ts";
|
||||
export * from "./spiders/index.ts";
|
||||
export {
|
||||
type DownloadTarget,
|
||||
type HitAndRun,
|
||||
@@ -57,5 +83,6 @@ export {
|
||||
type SiteRelease,
|
||||
type SiteReleaseDetail,
|
||||
staticSecretResolver,
|
||||
} from "./types";
|
||||
export { parseCount, parseSiteTime } from "./util";
|
||||
} from "./types.ts";
|
||||
export * from "./user-parsers/index.ts";
|
||||
export { parseCount, parseSiteTime } from "./util.ts";
|
||||
@@ -9,12 +9,12 @@
|
||||
* - `app/modules/indexer/parser/mtorrent.py`(api/member/profile)→ `healthCheck`
|
||||
* 详情 `/api/torrent/detail` 为本项目扩展(MP spider 无详情调用,字段结构参考 MovieClaw mteam.py,data-ref)。
|
||||
*/
|
||||
import { redactUrl } from "../../rss/redact";
|
||||
import { buildHttpContext, type SiteAdapterDeps } from "../deps";
|
||||
import { readBody, siteFetch } from "../http";
|
||||
import { healthFromError } from "../spider/adapter";
|
||||
import { promotionFromFactors } from "../spider/map";
|
||||
import type { SpiderTorrent } from "../spider/spider";
|
||||
import { redactUrl } from "../../rss/redact.ts";
|
||||
import { buildHttpContext, type SiteAdapterDeps } from "../deps.ts";
|
||||
import { readBody, siteFetch } from "../http.ts";
|
||||
import { healthFromError } from "../spider/adapter.ts";
|
||||
import { promotionFromFactors } from "../spider/map.ts";
|
||||
import type { SpiderTorrent } from "../spider/spider.ts";
|
||||
import {
|
||||
type DownloadTarget,
|
||||
type SiteAdapter,
|
||||
@@ -24,11 +24,11 @@ import {
|
||||
type SitePage,
|
||||
type SiteRelease,
|
||||
type SiteReleaseDetail,
|
||||
} from "../types";
|
||||
import { extractDoubanId, inspectTorrent, parseSiteTime } from "../util";
|
||||
import { mteamPost } from "./client";
|
||||
import { type MteamDefinition, mteamApiBase } from "./definition";
|
||||
import { type MteamTorrentJson, mteamSearchParams, parseMteamResult } from "./spider";
|
||||
} from "../types.ts";
|
||||
import { extractDoubanId, inspectTorrent, parseSiteTime } from "../util.ts";
|
||||
import { mteamPost } from "./client.ts";
|
||||
import { type MteamDefinition, mteamApiBase } from "./definition.ts";
|
||||
import { type MteamTorrentJson, mteamSearchParams, parseMteamResult } from "./spider.ts";
|
||||
|
||||
interface MteamSearchData {
|
||||
data?: MteamTorrentJson[] | null;
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
* M-Team API 调用。对应 MP `MTorrentSpider.search/__process_response`(MP 只看 HTTP 200);
|
||||
* 业务 code/message 检查与凭据/解析错误分层参考 MovieClaw `_post_api`(behavior-ref,迁移账本 MT2)。
|
||||
*/
|
||||
import { redactText } from "../../rss/redact";
|
||||
import { readText, type SiteHttpContext, siteFetch } from "../http";
|
||||
import { SiteError } from "../types";
|
||||
import { redactText } from "../../rss/redact.ts";
|
||||
import { readText, type SiteHttpContext, siteFetch } from "../http.ts";
|
||||
import { SiteError } from "../types.ts";
|
||||
|
||||
export type MteamApiPath =
|
||||
| "/api/torrent/search"
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
import type { SiteCapabilities } from "../types";
|
||||
import mteamJson from "../definitions/mteam.json" with { type: "json" };
|
||||
import { parseMteamDefinition } from "../definitions/schema.ts";
|
||||
import type { DefinitionProvenance } from "../spider/schema.ts";
|
||||
import type { SiteCapabilities } from "../types.ts";
|
||||
|
||||
/**
|
||||
* 馒头站点定义:键名沿用 MP indexer dict(id/name/domain/parser/timeout),
|
||||
@@ -7,6 +10,10 @@ import type { SiteCapabilities } from "../types";
|
||||
export interface MteamDefinition {
|
||||
id: string;
|
||||
name: string;
|
||||
enabled?: boolean;
|
||||
/** MP 用户信息解析器 schema(MTorrent) */
|
||||
schema?: string;
|
||||
provenance?: DefinitionProvenance;
|
||||
/** 网页域(MP `_url`,详情页 = `${domain}detail/${id}`) */
|
||||
domain: string;
|
||||
parser: "mTorrent";
|
||||
@@ -22,41 +29,16 @@ export interface MteamDefinition {
|
||||
capabilities: SiteCapabilities;
|
||||
/** 原始分类 ID → 名称(自行编制,待核验) */
|
||||
categoryNames: Record<string, string>;
|
||||
authSupported?: string[];
|
||||
customClass?: string;
|
||||
};
|
||||
}
|
||||
|
||||
export const mteamDefinition: MteamDefinition = {
|
||||
id: "mteam",
|
||||
name: "馒头",
|
||||
domain: "https://tp.m-team.cc/",
|
||||
parser: "mTorrent",
|
||||
timeout: 15,
|
||||
umt: {
|
||||
version: "[email protected]",
|
||||
kind: "mteam",
|
||||
displayName: "馒头",
|
||||
timezone: "+08:00",
|
||||
rateLimit: { minIntervalMs: 1000, burst: 1 },
|
||||
capabilities: { list: true, search: true, detail: true, download: true, rss: false },
|
||||
categoryNames: {
|
||||
"401": "电影",
|
||||
"419": "电影",
|
||||
"420": "电影",
|
||||
"421": "电影",
|
||||
"439": "电影",
|
||||
"402": "剧集",
|
||||
"403": "综艺",
|
||||
"435": "剧集",
|
||||
"438": "剧集",
|
||||
"404": "纪录",
|
||||
"405": "动画",
|
||||
"407": "体育",
|
||||
"406": "音乐",
|
||||
"408": "音乐",
|
||||
"434": "音乐",
|
||||
},
|
||||
},
|
||||
};
|
||||
/**
|
||||
* 馒头定义:definitions/mteam.json(由 scripts/mp-sync/movieclaw-import.mjs 从 MovieClaw v0.27.0 mteam.yaml 转换,
|
||||
* W1-3b 原值经 movieclaw-overrides/mteam.json 覆盖),加载时 zod 校验。
|
||||
*/
|
||||
export const mteamDefinition: MteamDefinition = parseMteamDefinition(mteamJson);
|
||||
|
||||
/** MP `site_rules.extract_domain` 的简化:取主机名最后两级。 */
|
||||
export function rootDomain(url: string): string {
|
||||
|
||||
@@ -9,7 +9,7 @@ import {
|
||||
SiteError,
|
||||
type SiteRateLimiter,
|
||||
staticSecretResolver,
|
||||
} from "../index";
|
||||
} from "../index.ts";
|
||||
|
||||
let api: MockMteamApi | undefined;
|
||||
afterEach(async () => {
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
* 2. 额外输出 `category_id`(原始分类 ID),`category` 仍是 MP 的媒体类型投影。
|
||||
* 3. 额外保留 `status` 原文到 raw,便于核验。
|
||||
*/
|
||||
import type { MediaKind, MpMediaType, SpiderTorrent } from "../spider/spider";
|
||||
import type { MediaKind, MpMediaType, SpiderTorrent } from "../spider/spider.ts";
|
||||
|
||||
export const MOVIE_CATEGORY = ["401", "419", "420", "421", "439", "405", "404"];
|
||||
export const TV_CATEGORY = ["403", "402", "435", "438", "404", "405"];
|
||||
@@ -87,10 +87,18 @@ export function getUploadVolumeFactor(discount: string | null | undefined): numb
|
||||
return discount ? (map[discount] ?? 1) : 1;
|
||||
}
|
||||
|
||||
/** MP `time_tools.format_timestamp`:非数字字符串原样返回,Unix 秒转本地时间(此处按站点时区交给调用方)。 */
|
||||
function formatTimestamp(v: string | null | undefined): string | undefined {
|
||||
/**
|
||||
* MP `time_tools.format_timestamp`:非纯数字字符串原样返回;Unix 秒 → 本地墙钟 "YYYY-MM-DD HH:MM:SS"。
|
||||
* MP 用进程本地时区(容器默认 Asia/Shanghai);这里固定 +08:00,与 adapter 按 `umt.timezone`(+08:00)转 UTC 配套。
|
||||
*/
|
||||
function formatTimestamp(v: string | number | null | undefined): string | undefined {
|
||||
if (v == null) return undefined;
|
||||
return String(v);
|
||||
if (typeof v === "string" && !/^\d+$/.test(v)) return v;
|
||||
const n = Number(v);
|
||||
if (!Number.isFinite(n)) return String(v);
|
||||
const d = new Date(Math.trunc(n) * 1000 + 8 * 3600_000);
|
||||
const p = (x: number) => String(x).padStart(2, "0");
|
||||
return `${d.getUTCFullYear()}-${p(d.getUTCMonth() + 1)}-${p(d.getUTCDate())} ${p(d.getUTCHours())}:${p(d.getUTCMinutes())}:${p(d.getUTCSeconds())}`;
|
||||
}
|
||||
|
||||
function toInt(v: unknown): number {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { createSiteRateLimiter } from "./rate-limit";
|
||||
import { parseSizeMp as parseSize } from "./spider/spider";
|
||||
import { parseCount, parseSiteTime } from "./util";
|
||||
import { createSiteRateLimiter } from "./rate-limit.ts";
|
||||
import { parseSizeMp as parseSize } from "./spider/spider.ts";
|
||||
import { parseCount, parseSiteTime } from "./util.ts";
|
||||
|
||||
describe("站点令牌桶", () => {
|
||||
it("burst 内不等待,之后按间隔补充;penalize 顺延;并发按顺序排队", async () => {
|
||||
|
||||
@@ -2,30 +2,49 @@
|
||||
* 随产品发布的站点注册表(迁移账本 SA1)。
|
||||
* 对应 MP `SitesHelper.get_indexers()/get_indexer(domain)`(闭源,sites.pyi)与
|
||||
* `IndexerModule` 按 `indexer.parser` 分派专用解析器(`mTorrent` → MTorrentSpider,其余 → SiteSpider)。
|
||||
* 加站:新增 definitions/<site>.json 并在此登记。
|
||||
* 加站:MovieClaw 来源重跑 scripts/mp-sync/movieclaw-import.mjs;其他来源新增 JSON 并登记到 definitions/。
|
||||
*/
|
||||
|
||||
import { audiencesDefinition } from "./definitions";
|
||||
import type { SiteAdapterDeps } from "./deps";
|
||||
import { createMteamAdapter } from "./mteam/adapter";
|
||||
import { type MteamDefinition, mteamDefinition } from "./mteam/definition";
|
||||
import { createSpiderSiteAdapter } from "./spider/adapter";
|
||||
import type { IndexerDefinition } from "./spider/schema";
|
||||
import type { SiteAdapter } from "./types";
|
||||
import { BUILTIN_SITE_DEFINITIONS, type SiteDefinition } from "./definitions/index.ts";
|
||||
import type { SiteAdapterDeps } from "./deps.ts";
|
||||
import { createMteamAdapter } from "./mteam/adapter.ts";
|
||||
import type { MteamDefinition } from "./mteam/definition.ts";
|
||||
import { createSpiderSiteAdapter } from "./spider/adapter.ts";
|
||||
import type { IndexerDefinition } from "./spider/schema.ts";
|
||||
import { createSpecializedAdapterFor } from "./spiders/index.ts";
|
||||
import type { SpecializedSiteDefinition } from "./spiders/types.ts";
|
||||
import type { SiteAdapter } from "./types.ts";
|
||||
|
||||
export type SiteDefinition = IndexerDefinition | MteamDefinition;
|
||||
export type { SiteDefinition };
|
||||
|
||||
export const SITE_DEFINITIONS: Record<string, SiteDefinition> = {
|
||||
audiences: audiencesDefinition,
|
||||
mteam: mteamDefinition,
|
||||
};
|
||||
/** 全部内置定义(含 enabled:false 的未实测站点) */
|
||||
export const SITE_DEFINITIONS: Readonly<Record<string, SiteDefinition>> = BUILTIN_SITE_DEFINITIONS;
|
||||
|
||||
function isMteam(def: SiteDefinition): def is MteamDefinition {
|
||||
return (def as MteamDefinition).parser === "mTorrent";
|
||||
/** MP `SitesHelper.get_indexer`:按 id 取定义;未知返回 undefined */
|
||||
export function getSiteDefinition(id: string): SiteDefinition | undefined {
|
||||
return SITE_DEFINITIONS[id];
|
||||
}
|
||||
|
||||
/** MP `SitesHelper.get_indexers`:列出定义;默认含未启用站点,`enabledOnly` 只列 enabled !== false */
|
||||
export function listSiteDefinitions(opts: { enabledOnly?: boolean } = {}): SiteDefinition[] {
|
||||
return Object.values(SITE_DEFINITIONS)
|
||||
.filter((d) => !opts.enabledOnly || d.enabled !== false)
|
||||
.sort((a, b) => a.id.localeCompare(b.id));
|
||||
}
|
||||
|
||||
/**
|
||||
* MP `IndexerModule` 按 `indexer.parser` 分派:mTorrent → 馒头适配器;SPIDER_PARSER_CLASSES 其余键(umt.kind=api)
|
||||
* → 专用 spider 适配器;其余 → 通用 SiteSpider。
|
||||
*/
|
||||
export function createSiteAdapter(def: SiteDefinition, deps: SiteAdapterDeps): SiteAdapter {
|
||||
return isMteam(def) ? createMteamAdapter(def, deps) : createSpiderSiteAdapter(def, deps);
|
||||
switch (def.umt.kind) {
|
||||
case "mteam":
|
||||
return createMteamAdapter(def as MteamDefinition, deps);
|
||||
case "api":
|
||||
return createSpecializedAdapterFor(def as SpecializedSiteDefinition, deps);
|
||||
default:
|
||||
return createSpiderSiteAdapter(def as IndexerDefinition, deps);
|
||||
}
|
||||
}
|
||||
|
||||
export function getSiteAdapter(siteId: string, deps: SiteAdapterDeps): SiteAdapter {
|
||||
|
||||
@@ -11,9 +11,9 @@
|
||||
* 本项目调整:登录页不再"返回空列表",而是抛 `auth_expired`(REQ §4.7.1 不展示空列表冒充无内容);
|
||||
* 3xx 不自动跟随(避免把 cookie 带到站外,并把跳转 login 识别为失效);凭证经 SecretResolver 注入。
|
||||
*/
|
||||
import { redactText } from "../../rss/redact";
|
||||
import { buildHttpContext, type SiteAdapterDeps } from "../deps";
|
||||
import { readBody, readText, siteFetch } from "../http";
|
||||
import { redactText } from "../../rss/redact.ts";
|
||||
import { buildHttpContext, type SiteAdapterDeps } from "../deps.ts";
|
||||
import { readBody, readText, siteFetch } from "../http.ts";
|
||||
import {
|
||||
type DownloadTarget,
|
||||
type SiteAdapter,
|
||||
@@ -23,12 +23,12 @@ import {
|
||||
type SiteListFilters,
|
||||
type SitePage,
|
||||
type SiteReleaseDetail,
|
||||
} from "../types";
|
||||
import { inspectTorrent } from "../util";
|
||||
import { parseDetailPage } from "./detail";
|
||||
import { spiderTorrentToRelease } from "./map";
|
||||
import type { IndexerDefinition } from "./schema";
|
||||
import { loadHtml, normalizeSelector, SiteSpider } from "./spider";
|
||||
} from "../types.ts";
|
||||
import { inspectTorrent } from "../util.ts";
|
||||
import { parseDetailPage } from "./detail.ts";
|
||||
import { spiderTorrentToRelease } from "./map.ts";
|
||||
import type { IndexerDefinition } from "./schema.ts";
|
||||
import { loadHtml, normalizeSelector, SiteSpider } from "./spider.ts";
|
||||
|
||||
/** MP `text_tools.remove_punctuation(text, replacement=" ", allow_space=True)` */
|
||||
export function clearSearchText(text: string): string {
|
||||
|
||||
@@ -4,11 +4,12 @@
|
||||
* 促销/H&R 复用站点定义中 downloadvolumefactor / uploadvolumefactor / freedate / hr 字段(与列表同语义)。
|
||||
*/
|
||||
import type { CheerioAPI } from "cheerio";
|
||||
import type { SiteReleaseDetail } from "../types";
|
||||
import { extractDoubanId, extractImdbId, parseCount, parseSiteTime } from "../util";
|
||||
import { categoryName, promotionFromFactors } from "./map";
|
||||
import type { IndexerDefinition } from "./schema";
|
||||
import { loadHtml, parseSizeMp, SiteSpider } from "./spider";
|
||||
import type { SiteReleaseDetail } from "../types.ts";
|
||||
import { extractDoubanId, extractImdbId, parseCount, parseSiteTime } from "../util.ts";
|
||||
import { categoryName, promotionFromFactors } from "./map.ts";
|
||||
import { mcSelect } from "./mc-selector.ts";
|
||||
import type { IndexerDefinition } from "./schema.ts";
|
||||
import { loadHtml, parseSizeMp, SiteSpider } from "./spider.ts";
|
||||
|
||||
export type DetailPageKind = "ok" | "login" | "not_found" | "unknown";
|
||||
|
||||
@@ -38,15 +39,23 @@ export function parseDetailPage(
|
||||
return found as ReturnType<CheerioAPI> | null;
|
||||
};
|
||||
|
||||
const title = h1
|
||||
// 逐字段选择器(MovieClaw detail_*_css)优先于 rowhead 表格模型
|
||||
const f = conf.fields ?? {};
|
||||
const title = (
|
||||
f.title
|
||||
? (mcSelect($, null, f.title) ?? "")
|
||||
: h1
|
||||
.contents()
|
||||
.filter((_, n) => n.type === "text")
|
||||
.text()
|
||||
)
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
const subtitle = row(conf.labels.subtitle)?.text().trim();
|
||||
const subtitle = f.subtitle ? (mcSelect($, null, f.subtitle) ?? undefined) : row(conf.labels.subtitle)?.text().trim();
|
||||
const basicText = row(conf.labels.basicInfo)?.text().replace(/\s+/g, " ") ?? "";
|
||||
const sizeText = /(?:大小|Size)\s*[:：]\s*([\d.,]+\s*[KMGTP]?i?B)/i.exec(basicText)?.[1] ?? null;
|
||||
const sizeText = f.size
|
||||
? (/([\d.,]+\s*[KMGTP]?i?B)/i.exec(mcSelect($, null, f.size) ?? "")?.[1] ?? null)
|
||||
: (/(?:大小|Size)\s*[:：]\s*([\d.,]+\s*[KMGTP]?i?B)/i.exec(basicText)?.[1] ?? null);
|
||||
const catText = /(?:类型|Type)\s*[:：]\s*(\S+)/i.exec(basicText)?.[1] ?? null;
|
||||
const catId = catText
|
||||
? (Object.entries(def.umt.categoryNames ?? {}).find(([, name]) => name === catText)?.[0] ?? null)
|
||||
@@ -102,9 +111,9 @@ export function parseDetailPage(
|
||||
};
|
||||
if (subtitle) detail.subtitle = subtitle;
|
||||
if (descText) detail.descriptionText = descText.slice(0, 20_000);
|
||||
const imdb = extractImdbId(descHtml) ?? extractImdbId(descText);
|
||||
const imdb = extractImdbId(mcSelect($, null, f.imdb)) ?? extractImdbId(descHtml) ?? extractImdbId(descText);
|
||||
if (imdb) detail.imdbId = imdb;
|
||||
const douban = extractDoubanId(descHtml);
|
||||
const douban = extractDoubanId(mcSelect($, null, f.douban)) ?? extractDoubanId(descHtml);
|
||||
if (douban) detail.doubanId = douban;
|
||||
return { kind: "ok", detail };
|
||||
}
|
||||
@@ -1,8 +1,8 @@
|
||||
import { redactUrl } from "../../rss/redact";
|
||||
import type { PromotionType, SitePromotion, SiteRelease } from "../types";
|
||||
import { parseSiteTime } from "../util";
|
||||
import type { IndexerDefinition } from "./schema";
|
||||
import type { SpiderTorrent } from "./spider";
|
||||
import { redactUrl } from "../../rss/redact.ts";
|
||||
import type { PromotionType, SitePromotion, SiteRelease } from "../types.ts";
|
||||
import { parseSiteTime } from "../util.ts";
|
||||
import type { IndexerDefinition } from "./schema.ts";
|
||||
import type { SpiderTorrent } from "./spider.ts";
|
||||
|
||||
/**
|
||||
* MP 系数 → 统一促销类型。MP 只有 downloadvolumefactor/uploadvolumefactor/freedate 三个字段,
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
/**
|
||||
* MovieClaw 选择器语法求值(parsel/cssselect 扩展伪元素),用于 MovieClaw v0.27.0 转换来的
|
||||
* `umt.detail.fields` / `umt.profile` 选择器。语义对照 MovieClaw `NexusPHPSite._css_text`(MIT,见
|
||||
* ../definitions/NOTICE-MovieClaw.md):
|
||||
* - 空选择器 → null
|
||||
* - `X::next_sibling_text` → X 首个命中元素之后第一个文本兄弟节点(XPath `following-sibling::text()[1]`),trim
|
||||
* - `X::attr(a)` → 首个命中元素的属性 a;`X::text` → 首个命中元素的直接文本节点(parsel `::text` 取第一个)
|
||||
* - 普通选择器 → 所有命中元素的全部后代文本,逐段 trim 后以空格连接
|
||||
* `:contains()` 由 cheerio-select 原生支持。
|
||||
*/
|
||||
import type { CheerioAPI } from "cheerio";
|
||||
|
||||
type Scope = ReturnType<CheerioAPI>;
|
||||
|
||||
export function mcSelect($: CheerioAPI, scope: Scope | null, css: string | null | undefined): string | null {
|
||||
if (!css) return null;
|
||||
const find = (sel: string) => (scope ? scope.find(sel) : $(sel));
|
||||
if (css.endsWith("::next_sibling_text")) {
|
||||
const el = find(css.slice(0, -"::next_sibling_text".length)).first();
|
||||
if (el.length === 0) return null;
|
||||
let n = el[0]!.next;
|
||||
while (n && n.type !== "text") n = n.next;
|
||||
const t = n && n.type === "text" ? (n as unknown as { data: string }).data.trim() : "";
|
||||
return t || null;
|
||||
}
|
||||
const attr = /::attr\(([^)]+)\)$/.exec(css);
|
||||
if (attr) {
|
||||
const v = find(css.slice(0, attr.index)).first().attr(attr[1]!.trim());
|
||||
return v?.trim() || null;
|
||||
}
|
||||
if (css.endsWith("::text")) {
|
||||
const el = find(css.slice(0, -"::text".length)).first();
|
||||
const t = el
|
||||
.contents()
|
||||
.filter((_, n) => n.type === "text")
|
||||
.first()
|
||||
.text()
|
||||
.trim();
|
||||
return t || null;
|
||||
}
|
||||
const texts: string[] = [];
|
||||
find(css).each((_, el) => {
|
||||
const walk = (n: { type: string; data?: string; children?: unknown[] }) => {
|
||||
if (n.type === "text") {
|
||||
const t = (n.data ?? "").trim();
|
||||
if (t) texts.push(t);
|
||||
} else for (const c of (n.children ?? []) as Array<typeof n>) walk(c);
|
||||
};
|
||||
walk(el as unknown as { type: string });
|
||||
});
|
||||
return texts.length ? texts.join(" ") : null;
|
||||
}
|
||||
@@ -63,8 +63,25 @@ export interface SpiderFields {
|
||||
[name: string]: SpiderFieldSelector | undefined;
|
||||
}
|
||||
|
||||
/** 定义来源(MovieClaw v0.27.0 MIT 转换 / 手写) */
|
||||
export interface DefinitionProvenance {
|
||||
source: "movieclaw" | "handwritten";
|
||||
ref?: string;
|
||||
commit?: string;
|
||||
license?: string;
|
||||
file?: string;
|
||||
/** 本项目手工覆盖层(深合并,优先于来源值) */
|
||||
overrides?: string;
|
||||
note?: string;
|
||||
}
|
||||
|
||||
export interface IndexerDefinition {
|
||||
id: string;
|
||||
/** 是否随产品启用(未实测站点为 false) */
|
||||
enabled?: boolean;
|
||||
/** MP 用户信息解析器 schema(`SiteSchema` 值,如 NexusPhp / NexusHhanclub) */
|
||||
schema?: string;
|
||||
provenance?: DefinitionProvenance;
|
||||
name: string;
|
||||
domain: string;
|
||||
encoding?: string;
|
||||
@@ -95,6 +112,26 @@ export interface UmtSiteExtension {
|
||||
/** 定义版本,入 site_snapshot */
|
||||
version: string;
|
||||
kind: "nexusphp";
|
||||
/** 需要自定义代码的站点(MovieClaw custom_class,如 TTG);通用 spider 结果不可靠 */
|
||||
customClass?: string;
|
||||
/** H&R 考核做种小时数(MovieClaw hr_seed_hours;暂无站点配置) */
|
||||
hrSeedHours?: number;
|
||||
/** 支持的凭据方式(MovieClaw auth.supported:cookie / credential / apikey) */
|
||||
authSupported?: string[];
|
||||
/** 登录表单配置(MovieClaw login_*;M2 账号密码登录用) */
|
||||
login?: {
|
||||
path: string;
|
||||
usernameField: string;
|
||||
passwordField: string;
|
||||
errorCss: string;
|
||||
extraFormData?: Array<[string, string]>;
|
||||
selectDefaults?: Array<[string, string]>;
|
||||
};
|
||||
/**
|
||||
* 用户资料选择器(MovieClaw profile_*,选择器语法见 spider/mc-selector.ts:支持 ::attr(x) / ::text / ::next_sibling_text)。
|
||||
* 主路径仍是 MP 用户信息解析器(`schema`),此处为备用数据,M2 使用。
|
||||
*/
|
||||
profile?: Record<string, string>;
|
||||
displayName?: string;
|
||||
/** 站点时间时区偏移,如 "+08:00";pubdate/freedate 按此转 UTC */
|
||||
timezone: string;
|
||||
@@ -119,6 +156,13 @@ export interface UmtSiteExtension {
|
||||
/** 促销/HR 标记所在范围 */
|
||||
headerScope: string;
|
||||
notFoundMarkers: string[];
|
||||
/**
|
||||
* 逐字段选择器(MovieClaw detail_*_css,mc-selector 语法)。配置时优先于 rowhead 表格模型,
|
||||
* 用于 HHClub 这类 div 详情页。
|
||||
*/
|
||||
fields?: Partial<
|
||||
Record<"title" | "subtitle" | "size" | "description" | "download" | "imdb" | "douban" | "fileList", string>
|
||||
>;
|
||||
};
|
||||
/** 下载路径(按资源 ID 拼接) */
|
||||
downloadPath: string;
|
||||
|
||||
@@ -9,7 +9,7 @@ import {
|
||||
SiteError,
|
||||
SiteSpider,
|
||||
staticSecretResolver,
|
||||
} from "../index";
|
||||
} from "../index.ts";
|
||||
|
||||
let site: MockNexusPhpSite | undefined;
|
||||
afterEach(async () => {
|
||||
|
||||
@@ -19,8 +19,8 @@
|
||||
* 3. 不发请求:HTTP/Cookie/UA 由 adapter 注入,本类只负责 URL 构造与 HTML 解析(便于限流与脱敏统一)。
|
||||
*/
|
||||
import { type CheerioAPI, load } from "cheerio";
|
||||
import type { IndexerDefinition, SpiderFieldSelector, SpiderFields, SpiderFilter, SpiderListConfig } from "./schema";
|
||||
import { renderTemplate } from "./template";
|
||||
import type { IndexerDefinition, SpiderFieldSelector, SpiderFields, SpiderFilter, SpiderListConfig } from "./schema.ts";
|
||||
import { renderTemplate } from "./template.ts";
|
||||
|
||||
type Sel = ReturnType<CheerioAPI>;
|
||||
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/__init__.py @ e6820a8
|
||||
*
|
||||
* 专用 spider 站点适配器:对应 MP `IndexerModule.search_torrents/refresh_torrents` 按 `indexer.parser`
|
||||
* 分派到 `SPIDER_PARSER_CLASSES` 并以 `_SPECIALIZED_SEARCH_ARGUMENTS` 传参;
|
||||
* 下载对应 `submission.download_torrent`(两段式换票由 spider.resolveDownload 现场完成)。
|
||||
*
|
||||
* 与 MP 的差异:MP `search` 失败返回 `(True, [])`;本项目抛 SiteError(凭据/限流/解析分类),
|
||||
* 健康检查即一次空关键词列表。list 对外页码 1 起;`pageSize=null` 的站(Haidan、TorrentLeech 关键词)只有第 1 页。
|
||||
*/
|
||||
import { buildHttpContext, type SiteAdapterDeps } from "../deps.ts";
|
||||
import { healthFromError } from "../spider/adapter.ts";
|
||||
import {
|
||||
type DownloadTarget,
|
||||
type SiteAdapter,
|
||||
type SiteCredentialKind,
|
||||
SiteError,
|
||||
type SiteHealth,
|
||||
type SiteListFilters,
|
||||
type SitePage,
|
||||
} from "../types.ts";
|
||||
import { specializedTorrentToRelease } from "./map.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedSpider, SpiderContext } from "./types.ts";
|
||||
|
||||
export function createSpecializedSiteAdapter(
|
||||
input: SpecializedSiteDefinition,
|
||||
spider: SpecializedSpider,
|
||||
deps: SiteAdapterDeps,
|
||||
): SiteAdapter {
|
||||
const def: SpecializedSiteDefinition = deps.baseUrl
|
||||
? {
|
||||
...input,
|
||||
domain: deps.baseUrl.endsWith("/") ? deps.baseUrl : `${deps.baseUrl}/`,
|
||||
umt: { ...input.umt, apiBaseUrl: deps.baseUrl.replace(/\/+$/, "") },
|
||||
}
|
||||
: input;
|
||||
const http = buildHttpContext(
|
||||
def.id,
|
||||
{ ...(def.timeout ? { timeoutMs: def.timeout * 1000 } : {}), ...deps },
|
||||
def.umt.rateLimit,
|
||||
);
|
||||
const cache = new Map<string, { value: unknown; expiresAt: number }>();
|
||||
/** 最近搜索结果的直链(仅内存,供 downloadTarget 复用;含 passkey 类参数,不外泄) */
|
||||
const enclosures = new Map<string, string>();
|
||||
|
||||
function ctx(signal?: AbortSignal): SpiderContext {
|
||||
const c: SpiderContext = {
|
||||
def,
|
||||
http,
|
||||
cache,
|
||||
async secret(kind: SiteCredentialKind, required = false) {
|
||||
const v = await deps.secrets.resolve({ siteId: def.id, kind });
|
||||
if (!v && required) {
|
||||
throw new SiteError(
|
||||
def.id,
|
||||
"auth_missing",
|
||||
`${def.name} 未配置${kind === "cookie" ? " Cookie" : " API Key"}`,
|
||||
);
|
||||
}
|
||||
return v;
|
||||
},
|
||||
};
|
||||
if (deps.userAgent) c.userAgent = deps.userAgent;
|
||||
if (signal) c.signal = signal;
|
||||
return c;
|
||||
}
|
||||
|
||||
async function doSearch(page: number, filters: SiteListFilters, keyword: string | null, signal?: AbortSignal) {
|
||||
const p = Math.max(1, page);
|
||||
const fetchedAt = http.now().toISOString();
|
||||
const size = spider.pageSize(keyword);
|
||||
if (size === null && p > 1) {
|
||||
return { items: [], page: p, totalPages: 1, totalPagesEstimated: true, fetchedAt } satisfies SitePage;
|
||||
}
|
||||
const q: Parameters<SpecializedSpider["search"]>[1] = { keyword, page: p - 1 };
|
||||
if (filters.mtype) q.mtype = filters.mtype;
|
||||
if (filters.categoryIds?.length) q.cat = filters.categoryIds.join(",");
|
||||
const rows = await spider.search(ctx(signal), q);
|
||||
let items = rows.flatMap((t) => {
|
||||
const r = specializedTorrentToRelease(t, def, fetchedAt);
|
||||
if (r && t.enclosure) enclosures.set(r.siteReleaseId, t.enclosure);
|
||||
return r ? [r] : [];
|
||||
});
|
||||
if (filters.categoryIds?.length) {
|
||||
const allow = new Set(filters.categoryIds.map(String));
|
||||
items = items.filter((r) => r.category.id === null || allow.has(r.category.id));
|
||||
}
|
||||
if (filters.freeOnly) items = items.filter((r) => r.promotion.downloadFactor === 0);
|
||||
return { items, page: p, totalPages: null, totalPagesEstimated: false, fetchedAt } satisfies SitePage;
|
||||
}
|
||||
|
||||
return {
|
||||
id: def.id,
|
||||
kind: "api",
|
||||
displayName: def.umt.displayName,
|
||||
capabilities: def.umt.capabilities,
|
||||
definitionVersion: def.umt.version,
|
||||
list(page, filters = {}, opts = {}) {
|
||||
return doSearch(page, filters, null, opts.signal);
|
||||
},
|
||||
search(query, filters = {}, opts = {}) {
|
||||
return doSearch(filters.page ?? 1, filters, query.trim() || null, opts.signal);
|
||||
},
|
||||
async detail() {
|
||||
throw new SiteError(def.id, "unsupported", `${def.name}:专用 API spider 不提供详情接口`);
|
||||
},
|
||||
async downloadTarget(siteReleaseId, opts = {}): Promise<DownloadTarget> {
|
||||
return spider.resolveDownload(ctx(opts.signal), siteReleaseId, enclosures.get(String(siteReleaseId)));
|
||||
},
|
||||
async healthCheck(opts = {}): Promise<SiteHealth> {
|
||||
const started = Date.now();
|
||||
const checkedAt = http.now().toISOString();
|
||||
try {
|
||||
await spider.search(ctx(opts.signal), { keyword: null, page: 0 });
|
||||
return { siteId: def.id, status: "ok", ok: true, checkedAt, latencyMs: Date.now() - started };
|
||||
} catch (e) {
|
||||
return healthFromError(def.id, e, checkedAt);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,371 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/chain/download/submission.py @ e6820a8
|
||||
* @mp-source app/foundation/temporal.py @ e6820a8
|
||||
*
|
||||
* 专用 spider 共用 IO 与工具:
|
||||
* - `submission._resolve_indirect_download_url` → `resolveIndirectDownloadUrl`(配置对象在内存中构造,不再 base64 编进 enclosure)
|
||||
* - `submission._normalize_indirect_download_url` → `normalizeIndirectDownloadUrl`
|
||||
* - `submission.download_torrent`(带/不带 cookie 下载并校验)→ `downloadTorrentFile`
|
||||
* - `temporal.format_timestamp` → `formatTimestampTz`;`urllib.parse.quote` → `pyQuote`
|
||||
* - 各 spider `__process_response` 的错误分支 → `apiRequest`(HTTP 401/403 → auth_invalid)+ `businessError`
|
||||
*/
|
||||
import { redactText, redactUrl } from "../../rss/redact.ts";
|
||||
import { readBody, readText, siteFetch } from "../http.ts";
|
||||
import { type DownloadTarget, SiteError, type SiteErrorKind } from "../types.ts";
|
||||
import { inspectTorrent } from "../util.ts";
|
||||
import type { SpiderContext } from "./types.ts";
|
||||
|
||||
/** 下载链接中需要脱敏的额外参数名(HDDolby downhash 等效 passkey) */
|
||||
export const EXTRA_SECRET_KEYS = ["downhash", "token", "credential", "sign", "key"];
|
||||
|
||||
const RATE_KEYWORDS = ["頻繁", "频繁", "too many", "rate limit", "稍后"];
|
||||
const NOT_FOUND_KEYWORDS = ["不存在", "not found", "not exist"];
|
||||
const AUTH_KEYWORDS = [
|
||||
"停用",
|
||||
"封禁",
|
||||
"禁用",
|
||||
"banned",
|
||||
"disabled",
|
||||
"無效",
|
||||
"无效",
|
||||
"invalid",
|
||||
"unauthorized",
|
||||
"未授權",
|
||||
"未授权",
|
||||
"未登录",
|
||||
"登录",
|
||||
"api key",
|
||||
"apikey",
|
||||
"token",
|
||||
"permission",
|
||||
"权限",
|
||||
"forbidden",
|
||||
];
|
||||
|
||||
/** 业务错误文案 → SiteError 类别(默认 parse_error) */
|
||||
export function classifyMessage(message: string | null | undefined): SiteErrorKind {
|
||||
const lower = String(message ?? "").toLowerCase();
|
||||
if (RATE_KEYWORDS.some((k) => lower.includes(k))) return "rate_limited";
|
||||
if (NOT_FOUND_KEYWORDS.some((k) => lower.includes(k))) return "not_found";
|
||||
if (AUTH_KEYWORDS.some((k) => lower.includes(k.toLowerCase()))) return "auth_invalid";
|
||||
return "parse_error";
|
||||
}
|
||||
|
||||
export function businessError(ctx: SpiderContext, what: string, message: unknown): SiteError {
|
||||
const msg = message == null || message === "" ? "未知错误" : String(message).slice(0, 200);
|
||||
return new SiteError(ctx.def.id, classifyMessage(msg), redactText(`${ctx.def.name} ${what}失败:${msg}`));
|
||||
}
|
||||
|
||||
export interface ApiRequestInit {
|
||||
method?: "GET" | "POST";
|
||||
headers?: Record<string, string>;
|
||||
query?: Record<string, string | number | null | undefined>;
|
||||
json?: unknown;
|
||||
/** 原样 query 串(已编码,不含 ?) */
|
||||
rawQuery?: string;
|
||||
}
|
||||
|
||||
export function withQuery(url: string, init: Pick<ApiRequestInit, "query" | "rawQuery">): string {
|
||||
const parts: string[] = [];
|
||||
if (init.rawQuery) parts.push(init.rawQuery);
|
||||
if (init.query) {
|
||||
const qs = new URLSearchParams();
|
||||
for (const [k, v] of Object.entries(init.query)) if (v !== undefined && v !== null) qs.append(k, String(v));
|
||||
const s = qs.toString();
|
||||
if (s) parts.push(s);
|
||||
}
|
||||
if (!parts.length) return url;
|
||||
return `${url}${url.includes("?") ? "&" : "?"}${parts.join("&")}`;
|
||||
}
|
||||
|
||||
/** 不跟随跳转(请求头可能带 cookie/key);跳 login 视为登录失效 */
|
||||
function checkRedirect(ctx: SpiderContext, res: Response, safe: string): void {
|
||||
if (res.status < 300 || res.status >= 400) return;
|
||||
void res.body?.cancel().catch(() => {});
|
||||
const loc = res.headers.get("location") ?? "";
|
||||
if (/login/i.test(loc))
|
||||
throw new SiteError(ctx.def.id, "auth_expired", `${ctx.def.name} 请求 ${safe}:跳转登录页,登录已失效`);
|
||||
throw new SiteError(ctx.def.id, "network", `${ctx.def.name} 请求 ${safe}:意外跳转 HTTP ${res.status}`, {
|
||||
httpStatus: res.status,
|
||||
});
|
||||
}
|
||||
|
||||
/** 发请求并解析 JSON;HTTP 401/403 → auth_invalid,404 → not_found,其它非 2xx → network,非 JSON → parse_error/site_maintenance */
|
||||
export async function apiRequest(ctx: SpiderContext, url: string, init: ApiRequestInit = {}): Promise<unknown> {
|
||||
const headers: Record<string, string> = { ...(init.headers ?? {}) };
|
||||
if (ctx.userAgent && !Object.keys(headers).some((k) => k.toLowerCase() === "user-agent"))
|
||||
headers["User-Agent"] = ctx.userAgent;
|
||||
let body: string | undefined;
|
||||
if (init.json !== undefined) {
|
||||
body = JSON.stringify(init.json);
|
||||
if (!Object.keys(headers).some((k) => k.toLowerCase() === "content-type"))
|
||||
headers["Content-Type"] = "application/json";
|
||||
}
|
||||
const full = withQuery(url, init);
|
||||
const safe = redactUrl(full, EXTRA_SECRET_KEYS);
|
||||
const res = await siteFetch(ctx.http, full, {
|
||||
method: init.method ?? (body !== undefined ? "POST" : "GET"),
|
||||
headers,
|
||||
...(body !== undefined ? { body } : {}),
|
||||
redirect: "manual",
|
||||
...(ctx.signal ? { signal: ctx.signal } : {}),
|
||||
});
|
||||
checkRedirect(ctx, res, safe);
|
||||
if (res.status === 401 || res.status === 403) {
|
||||
await res.body?.cancel().catch(() => {});
|
||||
throw new SiteError(
|
||||
ctx.def.id,
|
||||
"auth_invalid",
|
||||
`${ctx.def.name} 请求 ${safe}:HTTP ${res.status},凭据无效或无权限`,
|
||||
{
|
||||
httpStatus: res.status,
|
||||
},
|
||||
);
|
||||
}
|
||||
if (!res.ok) {
|
||||
await res.body?.cancel().catch(() => {});
|
||||
throw new SiteError(
|
||||
ctx.def.id,
|
||||
res.status === 404 ? "not_found" : "network",
|
||||
`${ctx.def.name} 请求 ${safe}:HTTP ${res.status}`,
|
||||
{
|
||||
httpStatus: res.status,
|
||||
},
|
||||
);
|
||||
}
|
||||
const text = await readText(ctx.http, res);
|
||||
try {
|
||||
return JSON.parse(text) as unknown;
|
||||
} catch {
|
||||
const maintenance = /維護|维护|maintenance/i.test(text);
|
||||
throw new SiteError(
|
||||
ctx.def.id,
|
||||
maintenance ? "site_maintenance" : "parse_error",
|
||||
`${ctx.def.name} 请求 ${safe}:响应不是有效 JSON${maintenance ? "(站点维护)" : ""}`,
|
||||
{ httpStatus: res.status },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** 发请求取文本(TNode 首页取 CSRF token) */
|
||||
export async function textRequest(ctx: SpiderContext, url: string, headers: Record<string, string>): Promise<string> {
|
||||
const h = { ...headers };
|
||||
if (ctx.userAgent) h["User-Agent"] = ctx.userAgent;
|
||||
const res = await siteFetch(ctx.http, url, {
|
||||
method: "GET",
|
||||
headers: h,
|
||||
redirect: "manual",
|
||||
...(ctx.signal ? { signal: ctx.signal } : {}),
|
||||
});
|
||||
checkRedirect(ctx, res, redactUrl(url));
|
||||
if (res.status !== 200) {
|
||||
await res.body?.cancel().catch(() => {});
|
||||
throw new SiteError(
|
||||
ctx.def.id,
|
||||
res.status === 401 || res.status === 403 ? "auth_invalid" : "network",
|
||||
`${ctx.def.name} 请求 ${redactUrl(url)}:HTTP ${res.status}`,
|
||||
{ httpStatus: res.status },
|
||||
);
|
||||
}
|
||||
return readText(ctx.http, res);
|
||||
}
|
||||
|
||||
export function isPlainObject(v: unknown): v is Record<string, unknown> {
|
||||
return v != null && typeof v === "object" && !Array.isArray(v);
|
||||
}
|
||||
|
||||
// ---- 时间 ----
|
||||
|
||||
const pad = (n: number) => String(n).padStart(2, "0");
|
||||
|
||||
function offsetMinutes(tz: string): number {
|
||||
const m = /^([+-])(\d{2}):?(\d{2})$/.exec(tz.trim());
|
||||
if (!m) return 0;
|
||||
return (m[1] === "-" ? -1 : 1) * (Number(m[2]) * 60 + Number(m[3]));
|
||||
}
|
||||
|
||||
/**
|
||||
* MP `format_timestamp`:非纯数字字符串原样返回;Unix 秒 → 本地墙钟 "YYYY-MM-DD HH:MM:SS"。
|
||||
* MP 用进程本地时区,这里用站点时区(`umt.timezone`),随后 map 再按同一时区转 UTC,结果等价。
|
||||
*/
|
||||
export function formatTimestampTz(timestamp: unknown, tz: string): string | undefined {
|
||||
if (timestamp == null || timestamp === "") return undefined;
|
||||
if (typeof timestamp === "string" && !/^\d+$/.test(timestamp)) return timestamp;
|
||||
const n = Number(timestamp);
|
||||
if (!Number.isFinite(n)) return String(timestamp);
|
||||
const d = new Date(Math.trunc(n) * 1000 + offsetMinutes(tz) * 60_000);
|
||||
return `${d.getUTCFullYear()}-${pad(d.getUTCMonth() + 1)}-${pad(d.getUTCDate())} ${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())}:${pad(d.getUTCSeconds())}`;
|
||||
}
|
||||
|
||||
/** 带显式时区(Z / ±HH:MM)的 ISO 时间 → UTC ISO;无时区返回 undefined */
|
||||
export function isoWithZoneToUtc(v: unknown): string | undefined {
|
||||
if (typeof v !== "string") return undefined;
|
||||
if (!/^\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}(:\d{2}(\.\d+)?)?\s*(Z|[+-]\d{2}:?\d{2})$/i.test(v.trim())) return undefined;
|
||||
const t = Date.parse(v.trim().replace(" ", "T"));
|
||||
return Number.isNaN(t) ? undefined : new Date(t).toISOString();
|
||||
}
|
||||
|
||||
/** Python `urllib.parse.quote`(safe='/') */
|
||||
export function pyQuote(s: string): string {
|
||||
return encodeURIComponent(s)
|
||||
.replace(/[!'()*]/g, (c) => `%${c.charCodeAt(0).toString(16).toUpperCase()}`)
|
||||
.replace(/%2F/g, "/");
|
||||
}
|
||||
|
||||
/** Python `int(x or 0)`;非法值 0(MP 会抛异常丢整页,这里保守取 0) */
|
||||
export function toInt(v: unknown): number {
|
||||
if (typeof v === "number") return Number.isFinite(v) ? Math.trunc(v) : 0;
|
||||
const n = Number.parseInt(String(v ?? "0") || "0", 10);
|
||||
return Number.isFinite(n) ? n : 0;
|
||||
}
|
||||
|
||||
/** Python `float(x)`;非法值回退 */
|
||||
export function toFloat(v: unknown, fallback = 1): number {
|
||||
const n = typeof v === "number" ? v : Number(v);
|
||||
return Number.isFinite(n) ? n : fallback;
|
||||
}
|
||||
|
||||
// ---- 缓存 ----
|
||||
|
||||
export function cacheGet<T>(ctx: SpiderContext, key: string): T | undefined {
|
||||
const hit = ctx.cache.get(key);
|
||||
if (!hit) return undefined;
|
||||
if (hit.expiresAt <= ctx.http.now().getTime()) {
|
||||
ctx.cache.delete(key);
|
||||
return undefined;
|
||||
}
|
||||
return hit.value as T;
|
||||
}
|
||||
|
||||
export function cacheSet(ctx: SpiderContext, key: string, value: unknown, ttlMs: number): void {
|
||||
ctx.cache.set(key, { value, expiresAt: ctx.http.now().getTime() + ttlMs });
|
||||
}
|
||||
|
||||
// ---- 下载 ----
|
||||
|
||||
/** MP 两段式下载的请求配置(原本 base64 编进 enclosure;这里只在内存中构造) */
|
||||
export interface IndirectRequestConfig {
|
||||
method: "get" | "post";
|
||||
header: Record<string, string>;
|
||||
/** requests params → query string */
|
||||
params?: Record<string, string | number>;
|
||||
/** 成功标志键(Yema: "success") */
|
||||
success?: string;
|
||||
/** 结果字段路径,点分(如 "data.download_url") */
|
||||
result?: string;
|
||||
result_base_url?: string;
|
||||
result_path?: string;
|
||||
result_query_param?: string;
|
||||
}
|
||||
|
||||
/** MP `_normalize_indirect_download_url`:把换票结果约束到可信 API 来源 */
|
||||
export function normalizeIndirectDownloadUrl(url: string, baseUrl?: string): string {
|
||||
if (!url || !baseUrl) return url;
|
||||
let base: URL;
|
||||
try {
|
||||
base = new URL(baseUrl);
|
||||
} catch {
|
||||
return url;
|
||||
}
|
||||
let u: URL;
|
||||
try {
|
||||
u = new URL(url);
|
||||
} catch {
|
||||
return new URL(url, `${baseUrl.replace(/\/+$/, "")}/`).toString();
|
||||
}
|
||||
u.protocol = base.protocol;
|
||||
u.host = base.host;
|
||||
return u.toString();
|
||||
}
|
||||
|
||||
/** MP `_resolve_indirect_download_url`:换票 → 临时直链(失败抛 SiteError,MP 返回 None) */
|
||||
export async function resolveIndirectDownloadUrl(
|
||||
ctx: SpiderContext,
|
||||
requestUrl: string,
|
||||
cfg: IndirectRequestConfig,
|
||||
): Promise<string> {
|
||||
const data = await apiRequest(ctx, requestUrl, {
|
||||
method: cfg.method === "get" ? "GET" : "POST",
|
||||
headers: cfg.header,
|
||||
...(cfg.params ? { query: cfg.params } : {}),
|
||||
});
|
||||
const fail = (why: string): never => {
|
||||
const msg = isPlainObject(data) ? (data.message ?? data.msg ?? data.errorMessage) : undefined;
|
||||
throw new SiteError(
|
||||
ctx.def.id,
|
||||
msg ? classifyMessage(String(msg)) : "parse_error",
|
||||
redactText(`${ctx.def.name} 换取下载地址失败:${msg ? String(msg).slice(0, 200) : why}`),
|
||||
);
|
||||
};
|
||||
if (!cfg.result) return typeof data === "string" ? data : fail("响应不是字符串");
|
||||
if (cfg.success && !(isPlainObject(data) && data[cfg.success])) fail("success 标志为假");
|
||||
let cur: unknown = data;
|
||||
for (const key of cfg.result.split(".")) {
|
||||
if (!isPlainObject(cur)) fail(`缺少字段 ${cfg.result}`);
|
||||
cur = (cur as Record<string, unknown>)[key];
|
||||
if (!cur) fail(`字段 ${cfg.result} 为空`);
|
||||
}
|
||||
if (cfg.result_path && cfg.result_query_param) {
|
||||
const u = new URL(cfg.result_path.replace(/^\/+/, ""), `${String(cfg.result_base_url).replace(/\/+$/, "")}/`);
|
||||
return `${u.toString()}?${new URLSearchParams({ [cfg.result_query_param]: String(cur) }).toString()}`;
|
||||
}
|
||||
if (typeof cur !== "string") return fail("下载地址不是字符串");
|
||||
return normalizeIndirectDownloadUrl(cur, cfg.result_base_url);
|
||||
}
|
||||
|
||||
/**
|
||||
* MP `download_torrent`:GET 种子并校验。cookie 为空表示"涉及解析地址的不使用 Cookie 下载"。
|
||||
* 带 cookie 时不跟随跳转(避免 cookie 外泄;跳 login 视为失效;跳磁力原样返回)。
|
||||
*/
|
||||
export async function downloadTorrentFile(
|
||||
ctx: SpiderContext,
|
||||
url: string,
|
||||
opts: { cookie?: string | null; headers?: Record<string, string> } = {},
|
||||
): Promise<DownloadTarget> {
|
||||
const headers: Record<string, string> = { ...(opts.headers ?? {}) };
|
||||
if (ctx.userAgent) headers["User-Agent"] = ctx.userAgent;
|
||||
if (opts.cookie) headers.cookie = opts.cookie;
|
||||
const safe = redactUrl(url, EXTRA_SECRET_KEYS);
|
||||
const res = await siteFetch(ctx.http, url, {
|
||||
method: "GET",
|
||||
headers,
|
||||
...(opts.cookie ? { redirect: "manual" as const } : {}),
|
||||
...(ctx.signal ? { signal: ctx.signal } : {}),
|
||||
});
|
||||
const verifiedAt = ctx.http.now().toISOString();
|
||||
if (res.status >= 300 && res.status < 400) {
|
||||
await res.body?.cancel().catch(() => {});
|
||||
const loc = res.headers.get("location") ?? "";
|
||||
if (loc.startsWith("magnet:")) return { kind: "magnet", uri: loc, verifiedAt };
|
||||
if (/login/i.test(loc)) throw new SiteError(ctx.def.id, "auth_expired", `下载 ${safe}:跳转登录页,Cookie 已失效`);
|
||||
throw new SiteError(ctx.def.id, "network", `下载 ${safe}:意外跳转 HTTP ${res.status}`, { httpStatus: res.status });
|
||||
}
|
||||
if (res.status === 401 || res.status === 403) {
|
||||
await res.body?.cancel().catch(() => {});
|
||||
throw new SiteError(ctx.def.id, "auth_invalid", `下载 ${safe}:HTTP ${res.status}`, { httpStatus: res.status });
|
||||
}
|
||||
if (!res.ok) {
|
||||
await res.body?.cancel().catch(() => {});
|
||||
throw new SiteError(ctx.def.id, res.status === 404 ? "not_found" : "network", `下载 ${safe}:HTTP ${res.status}`, {
|
||||
httpStatus: res.status,
|
||||
});
|
||||
}
|
||||
const bytes = await readBody(ctx.http, res);
|
||||
const info = inspectTorrent(bytes);
|
||||
if (!info) throw new SiteError(ctx.def.id, "parse_error", `下载 ${safe}:返回内容不是 .torrent`);
|
||||
return { kind: "torrent", bytes, infoHash: info.infoHash, name: info.name, fileName: null, verifiedAt };
|
||||
}
|
||||
|
||||
/** 站点前端地址(以 / 结尾) */
|
||||
export function siteBase(def: { domain: string }): string {
|
||||
return def.domain.endsWith("/") ? def.domain : `${def.domain}/`;
|
||||
}
|
||||
|
||||
/** 提取资源 ID:纯数字或 URL 末段数字 */
|
||||
export function checkReleaseId(siteId: string, id: string): string {
|
||||
const m = /(\d+)\/?$/.exec(String(id).trim());
|
||||
if (!m) throw new SiteError(siteId, "not_found", "资源 ID 非法");
|
||||
return m[1]!;
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
import { createMockFetch, HAIDAN_SEARCH_ERROR, HAIDAN_SEARCH_OK } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { haidanSearchQuery, haidanSpider } from "./haidan.ts";
|
||||
import { adapterWith, expectSiteError, testDef, withTorrent } from "./testing.ts";
|
||||
|
||||
const def = testDef("Haidan", { domain: "https://www.haidan.video/" });
|
||||
|
||||
describe("HaiDanSpider(MP haidan.py)", () => {
|
||||
it("__get_params:isapi/search_area/空关键词/分类逗号拼接", () => {
|
||||
expect(haidanSearchQuery(null)).toBe("isapi=1&search_area=0&search=&search_mode=0&cat=");
|
||||
expect(haidanSearchQuery("tt1160419", "tv")).toBe(
|
||||
"isapi=1&search_area=4&search=tt1160419&search_mode=0&cat=402%2C403%2C404%2C405",
|
||||
);
|
||||
});
|
||||
|
||||
it("列表解析:字段映射、分类投影、促销系数表、passkey 脱敏", async () => {
|
||||
const m = createMockFetch({ "/torrents.php": { body: HAIDAN_SEARCH_OK } });
|
||||
const page = await adapterWith(def, haidanSpider, m.fetch).search("Dune", { mtype: "movie" });
|
||||
expect(m.requests[0]!.headers.cookie).toBe("c=COOKIESECRET");
|
||||
expect(m.requests[0]!.url.searchParams.get("cat")).toBe("401,404,405");
|
||||
const by = Object.fromEntries(page.items.map((i) => [i.siteReleaseId, i]));
|
||||
expect(by["30001"]).toMatchObject({
|
||||
title: "Dune.Part.Two.2024.2160p.UHD.BluRay.REMUX.HDR.HEVC.Atmos-HDH",
|
||||
subtitle: "沙丘2 | 4K 原盘 REMUX",
|
||||
category: { id: "401" },
|
||||
sizeBytes: 76543210987,
|
||||
seeders: 88,
|
||||
leechers: 3,
|
||||
snatched: 412,
|
||||
promotion: { type: "free", downloadFactor: 0, uploadFactor: 1 },
|
||||
hr: "unknown",
|
||||
uploadedAt: "2024-09-20T00:00:00.000Z",
|
||||
detailUrl: "https://www.haidan.video/details.php?group_id=9001&torrent_id=30001",
|
||||
raw: expect.objectContaining({ category: "电影" }),
|
||||
});
|
||||
expect(by["30002"]!.promotion).toMatchObject({ type: "2x50%", downloadFactor: 0.5, uploadFactor: 2 });
|
||||
expect(by["30002"]!.raw.category).toBe("电视剧");
|
||||
expect(by["30003"]!.raw.category).toBe("音乐");
|
||||
expect(JSON.stringify(page)).not.toContain("HDPASSKEY123");
|
||||
expect(page.totalPages).toBeNull();
|
||||
// 不翻页:第 2 页直接为空
|
||||
expect((await adapterWith(def, haidanSpider, m.fetch).list(2)).items).toEqual([]);
|
||||
});
|
||||
|
||||
it("业务错误 / 缺 Cookie / 下载沿用缓存直链带 Cookie", async () => {
|
||||
const bad = createMockFetch({ "/torrents.php": { body: HAIDAN_SEARCH_ERROR } });
|
||||
await expectSiteError(adapterWith(def, haidanSpider, bad.fetch).list(1), "auth_invalid");
|
||||
await expectSiteError(adapterWith(def, haidanSpider, bad.fetch, {}).list(1), "auth_missing");
|
||||
const m = createMockFetch({ "/torrents.php": { body: HAIDAN_SEARCH_OK } });
|
||||
const t = withTorrent(m.fetch, (u) => u.pathname === "/download.php");
|
||||
const a = adapterWith(def, haidanSpider, t.fetch);
|
||||
await a.list(1);
|
||||
const target = await a.downloadTarget("30001");
|
||||
expect(target).toMatchObject({ kind: "torrent", infoHash: t.infoHash });
|
||||
expect(t.hits[0]!.url.searchParams.get("passkey")).toBe("HDPASSKEY123");
|
||||
expect(t.hits[0]!.headers.get("cookie")).toBe("c=COOKIESECRET");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,141 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/spider/haidan.py @ e6820a8
|
||||
*
|
||||
* 海胆 haidan.video(parser "Haidan",Cookie + `torrents.php?isapi=1` JSON)。函数对照 `HaiDanSpider`:
|
||||
* - `_movie_category/_tv_category/_music_category/_dl_state/_up_state` → 同名常量
|
||||
* - `get_search_page_size` → `pageSize`(恒 null:不翻页)
|
||||
* - `__get_params`(含 `__dict_to_query`)→ `haidanSearchQuery`
|
||||
* - `__parse_result` → `parseHaidanResult`;`__process_response` → `processHaidanResponse`
|
||||
* - `search/async_search` → `haidanSpider.search`
|
||||
* - `__get_downloadvolumefactor/__get_uploadvolumefactor` → `haidanDownloadVolumeFactor/haidanUploadVolumeFactor`
|
||||
* 下载:MP 用结果里的 `url` 字段(可能含 passkey)带 Cookie 下载;缓存缺失时回退 NexusPHP `download.php?id=`(TODO(verify))。
|
||||
*/
|
||||
import type { MpMediaType } from "../spider/spider.ts";
|
||||
import { SiteError } from "../types.ts";
|
||||
import {
|
||||
apiRequest,
|
||||
businessError,
|
||||
downloadTorrentFile,
|
||||
formatTimestampTz,
|
||||
isPlainObject,
|
||||
siteBase,
|
||||
toInt,
|
||||
} from "./common.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedSpider, SpecializedTorrent, SpiderSearchQuery } from "./types.ts";
|
||||
|
||||
export const HAIDAN_MOVIE_CATEGORY = ["401", "404", "405"];
|
||||
export const HAIDAN_TV_CATEGORY = ["402", "403", "404", "405"];
|
||||
/** 音乐搜索同时覆盖高品质音频和音乐视频/演唱会分区 */
|
||||
export const HAIDAN_MUSIC_CATEGORY = ["406", "408"];
|
||||
|
||||
/** 促销状态 1-普通,2-免费,3-2X,4-2X免费,5-50%,6-2X50%,7-30% */
|
||||
export const HAIDAN_DL_STATE: Record<string, number> = { "1": 1, "2": 0, "3": 1, "4": 0, "5": 0.5, "6": 0.5, "7": 0.3 };
|
||||
export const HAIDAN_UP_STATE: Record<string, number> = { "1": 1, "2": 1, "3": 2, "4": 2, "5": 1, "6": 2, "7": 1 };
|
||||
|
||||
export function haidanDownloadVolumeFactor(discount: unknown): number {
|
||||
return discount ? (HAIDAN_DL_STATE[String(discount)] ?? 1) : 1;
|
||||
}
|
||||
|
||||
export function haidanUploadVolumeFactor(discount: unknown): number {
|
||||
return discount ? (HAIDAN_UP_STATE[String(discount)] ?? 1) : 1;
|
||||
}
|
||||
|
||||
/** MP `__get_params`:返回已编码的 query 串(列表转逗号分隔;空关键词显式传空串) */
|
||||
export function haidanSearchQuery(keyword: string | null | undefined, mtype?: SpiderSearchQuery["mtype"]): string {
|
||||
let categories: string[];
|
||||
if (!mtype) categories = [];
|
||||
else if (mtype === "tv") categories = HAIDAN_TV_CATEGORY;
|
||||
else if (mtype === "music") categories = HAIDAN_MUSIC_CATEGORY;
|
||||
else categories = HAIDAN_MOVIE_CATEGORY;
|
||||
// 0-标题 1-简介(较慢)3-发种用户名 4-IMDb
|
||||
const searchArea = keyword?.startsWith("tt") ? "4" : "0";
|
||||
return new URLSearchParams({
|
||||
isapi: "1",
|
||||
search_area: searchArea,
|
||||
search: keyword || "",
|
||||
search_mode: "0", // 0-与 1-或 2-精准
|
||||
cat: categories.join(","),
|
||||
}).toString();
|
||||
}
|
||||
|
||||
export interface HaidanItem {
|
||||
group_id?: string | number;
|
||||
category?: string | number | null;
|
||||
name?: string;
|
||||
small_descr?: string;
|
||||
url?: string;
|
||||
added?: string | number;
|
||||
size?: string | number;
|
||||
seeders?: string | number;
|
||||
leechers?: string | number;
|
||||
times_completed?: string | number;
|
||||
sp_state?: string | number;
|
||||
[k: string]: unknown;
|
||||
}
|
||||
|
||||
/** MP `__parse_result`(data 为 {tid: item} 字典) */
|
||||
export function parseHaidanResult(
|
||||
result: { data?: Record<string, HaidanItem> | null },
|
||||
def: SpecializedSiteDefinition,
|
||||
) {
|
||||
const torrents: SpecializedTorrent[] = [];
|
||||
const base = siteBase(def);
|
||||
for (const [tid, item] of Object.entries(result.data ?? {})) {
|
||||
// 分类字段在每条种子内,接口顶层没有 category 字段
|
||||
const categoryValue = String(item.category || "");
|
||||
let category: MpMediaType;
|
||||
if (HAIDAN_MUSIC_CATEGORY.includes(categoryValue)) category = "音乐";
|
||||
else if (HAIDAN_TV_CATEGORY.includes(categoryValue) && !HAIDAN_MOVIE_CATEGORY.includes(categoryValue))
|
||||
category = "电视剧";
|
||||
else if (HAIDAN_MOVIE_CATEGORY.includes(categoryValue)) category = "电影";
|
||||
else category = "未知";
|
||||
const t: SpecializedTorrent = {
|
||||
title: item.name,
|
||||
description: item.small_descr,
|
||||
size: toInt(item.size),
|
||||
seeders: toInt(item.seeders),
|
||||
peers: toInt(item.leechers),
|
||||
grabs: toInt(item.times_completed),
|
||||
downloadvolumefactor: haidanDownloadVolumeFactor(item.sp_state),
|
||||
uploadvolumefactor: haidanUploadVolumeFactor(item.sp_state),
|
||||
page_url: `${base}details.php?group_id=${item.group_id}&torrent_id=${tid}`,
|
||||
labels: [],
|
||||
category,
|
||||
site_release_id: tid,
|
||||
};
|
||||
if (item.url) t.enclosure = String(item.url);
|
||||
const pubdate = formatTimestampTz(item.added, def.umt.timezone);
|
||||
if (pubdate) t.pubdate = pubdate;
|
||||
if (categoryValue) t.category_id = categoryValue;
|
||||
torrents.push(t);
|
||||
}
|
||||
return torrents;
|
||||
}
|
||||
|
||||
export const haidanSpider: SpecializedSpider = {
|
||||
parser: "Haidan",
|
||||
credential: "cookie",
|
||||
pageSize: () => null,
|
||||
async search(ctx, q) {
|
||||
const cookie = await ctx.secret("cookie", true);
|
||||
const data = await apiRequest(ctx, `${siteBase(ctx.def)}torrents.php`, {
|
||||
headers: { cookie: cookie! },
|
||||
rawQuery: haidanSearchQuery(q.keyword, q.mtype),
|
||||
});
|
||||
// MP __process_response:code != 0 → 失败
|
||||
if (!isPlainObject(data) || data.code !== 0) {
|
||||
throw businessError(ctx, "搜索", isPlainObject(data) ? data.msg : "响应结构无效");
|
||||
}
|
||||
return parseHaidanResult(data as { data?: Record<string, HaidanItem> }, ctx.def);
|
||||
},
|
||||
async resolveDownload(ctx, siteReleaseId, enclosure) {
|
||||
const cookie = await ctx.secret("cookie", true);
|
||||
const id = /(\d+)\/?$/.exec(String(siteReleaseId))?.[1];
|
||||
if (!id) throw new SiteError(ctx.def.id, "not_found", "资源 ID 非法");
|
||||
// TODO(verify):无缓存直链时回退 download.php?id=
|
||||
const url = enclosure ?? `${siteBase(ctx.def)}download.php?id=${id}`;
|
||||
return downloadTorrentFile(ctx, url, { cookie });
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,61 @@
|
||||
import { createMockFetch, HDDOLBY_SEARCH_ERROR, HDDOLBY_SEARCH_OK } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { hddolbyApiBase, hddolbySearchParams, hddolbySpider } from "./hddolby.ts";
|
||||
import { adapterWith, expectSiteError, testDef, withTorrent } from "./testing.ts";
|
||||
|
||||
const def = testDef("HDDolby", { domain: "https://www.hddolby.com/" });
|
||||
|
||||
describe("HddolbySpider(MP hddolby.py)", () => {
|
||||
it("__get_params 与 API 地址:page_number 为 0 起、无类型时 movie+tv 去重升序", () => {
|
||||
expect(hddolbyApiBase(def)).toBe("https://api.hddolby.com");
|
||||
expect(hddolbySearchParams("Lost")).toEqual({
|
||||
keyword: "Lost",
|
||||
page_number: 0,
|
||||
page_size: 100,
|
||||
categories: [401, 402, 403, 404, 405],
|
||||
visible: 1,
|
||||
});
|
||||
expect(hddolbySearchParams(null, "music", 2).categories).toEqual([406, 408]);
|
||||
});
|
||||
|
||||
it("列表解析:促销类型表、标签表、分类优先级(405 归电视剧)、downhash 脱敏", async () => {
|
||||
const m = createMockFetch({ "POST https://api.hddolby.com/api/v1/torrent/search": { body: HDDOLBY_SEARCH_OK } });
|
||||
const page = await adapterWith(def, hddolbySpider, m.fetch).list(1);
|
||||
const req = m.requests[0]!;
|
||||
expect(req.headers["x-api-key"]).toBe("APIKEYSECRET");
|
||||
expect(req.headers.cookie).toBe("c=COOKIESECRET");
|
||||
expect(JSON.parse(req.body!)).toMatchObject({ keyword: null, page_number: 0 });
|
||||
const [a, b, c] = page.items;
|
||||
expect(a).toMatchObject({
|
||||
siteReleaseId: "120202",
|
||||
title: "[DBY] Lost S06 2010 Complete 1080p Netflix WEB-DL AVC DDP5.1-DBTV",
|
||||
category: { id: "402" },
|
||||
promotion: { type: "free", downloadFactor: 0, uploadFactor: 1, expiresAt: "2026-09-30T04:00:00.000Z" },
|
||||
tags: ["官方", "中文字幕"],
|
||||
uploadedAt: "2025-02-18T11:47:56.000Z",
|
||||
detailUrl: "https://www.hddolby.com/details.php?id=120202&hit=1",
|
||||
hr: "unknown",
|
||||
});
|
||||
expect(a!.raw.category).toBe("电视剧");
|
||||
// promotion_time_type 6 → 2x(下载 1 上传 2),0000 截止无效
|
||||
expect(b!.promotion).toEqual({ type: "2x", downloadFactor: 1, uploadFactor: 2, expiresAt: null });
|
||||
expect(b!.raw.category).toBe("电影");
|
||||
expect(c!.raw.category).toBe("音乐");
|
||||
expect(c!.tags).toEqual([]);
|
||||
expect(JSON.stringify(page)).not.toContain("DOWNHASHSECRET");
|
||||
const t = withTorrent(m.fetch, (u) => u.pathname === "/download.php");
|
||||
const ad = adapterWith(def, hddolbySpider, t.fetch);
|
||||
await ad.list(1);
|
||||
await ad.downloadTarget("120202");
|
||||
expect(t.hits[0]!.url.searchParams.get("downhash")).toBe("DOWNHASHSECRET");
|
||||
});
|
||||
|
||||
it("error 字段 / HTTP 401 / 缺 API Key", async () => {
|
||||
const m = createMockFetch({ "/api/v1/torrent/search": { body: HDDOLBY_SEARCH_ERROR } });
|
||||
const e = await expectSiteError(adapterWith(def, hddolbySpider, m.fetch).list(1), "auth_invalid");
|
||||
expect(e.message).toContain("invalid api key");
|
||||
const h = createMockFetch({ "/api/v1/torrent/search": { status: 401, body: "no" } });
|
||||
await expectSiteError(adapterWith(def, hddolbySpider, h.fetch).list(1), "auth_invalid");
|
||||
await expectSiteError(adapterWith(def, hddolbySpider, m.fetch, { cookie: "c=1" }).list(1), "auth_missing");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,171 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/spider/hddolby.py @ e6820a8
|
||||
*
|
||||
* HDDolby(parser "HDDolby",`https://api.<域>/api/v1/torrent/search`,x-api-key + Cookie)。函数对照 `HddolbySpider`:
|
||||
* - `_movie_category/_tv_category/_music_category/_labels` → 同名常量
|
||||
* - `get_search_page_size` → `pageSize`(100)
|
||||
* - `__get_params` → `hddolbySearchParams`(注意 MP 的 page_number 直接传 0 起页码,原样保留)
|
||||
* - `__parse_result` → `parseHddolbyResult`;`__process_response` → `hddolbySpider.search` 内的错误分支
|
||||
* - `__get_downloadvolumefactor/__get_uploadvolumefactor` → `hddolbyDownloadVolumeFactor/hddolbyUploadVolumeFactor`
|
||||
* - `__get_download_url` → `hddolbyDownloadUrl`(含 downhash,raw/downloadUrl 中脱敏)
|
||||
* 差异:API 返回的 `hr` 字段 MP 未映射,这里同样不映射(H&R = unknown,待核验语义)。
|
||||
*/
|
||||
import type { MpMediaType } from "../spider/spider.ts";
|
||||
import { extractDomain } from "../user-parsers/common.ts";
|
||||
import { apiRequest, businessError, checkReleaseId, downloadTorrentFile, isPlainObject, siteBase } from "./common.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedSpider, SpecializedTorrent, SpiderSearchQuery } from "./types.ts";
|
||||
|
||||
export const HDDOLBY_MOVIE_CATEGORY = [401, 405];
|
||||
export const HDDOLBY_TV_CATEGORY = [402, 403, 404, 405];
|
||||
/** 音乐搜索同时覆盖高品质音频和音乐视频/演唱会分区 */
|
||||
export const HDDOLBY_MUSIC_CATEGORY = [406, 408];
|
||||
|
||||
export const HDDOLBY_LABELS: Record<string, string> = {
|
||||
gf: "官方",
|
||||
gy: "国语",
|
||||
yy: "粤语",
|
||||
ja: "日语",
|
||||
ko: "韩语",
|
||||
zz: "中文字幕",
|
||||
jz: "禁转",
|
||||
xz: "限转",
|
||||
diy: "DIY",
|
||||
sf: "首发",
|
||||
yq: "应求",
|
||||
m0: "零魔",
|
||||
yc: "原创",
|
||||
gz: "官字",
|
||||
db: "Dolby Vision",
|
||||
hdr10: "HDR10",
|
||||
hdrm: "HDR10+",
|
||||
tx: "特效",
|
||||
lz: "连载",
|
||||
wj: "完结",
|
||||
hdrv: "HDR Vivid",
|
||||
hlg: "HLG",
|
||||
hq: "高码率",
|
||||
hfr: "高帧率",
|
||||
};
|
||||
|
||||
export function hddolbyDownloadVolumeFactor(discount: unknown): number {
|
||||
const map: Record<number, number> = { 2: 0, 5: 0.5, 6: 1, 7: 0.3 };
|
||||
return discount ? (map[Number(discount)] ?? 1) : 1;
|
||||
}
|
||||
|
||||
export function hddolbyUploadVolumeFactor(discount: unknown): number {
|
||||
const map: Record<number, number> = { 3: 2, 4: 2, 6: 2 };
|
||||
return discount ? (map[Number(discount)] ?? 1) : 1;
|
||||
}
|
||||
|
||||
export function hddolbyApiBase(def: SpecializedSiteDefinition): string {
|
||||
return (def.umt.apiBaseUrl ?? `https://api.${extractDomain(def.domain)}`).replace(/\/+$/, "");
|
||||
}
|
||||
|
||||
/** MP `__get_params` */
|
||||
export function hddolbySearchParams(keyword: string | null | undefined, mtype?: SpiderSearchQuery["mtype"], page = 0) {
|
||||
let categories: number[];
|
||||
if (mtype === "tv") categories = HDDOLBY_TV_CATEGORY;
|
||||
else if (mtype === "movie") categories = HDDOLBY_MOVIE_CATEGORY;
|
||||
else if (mtype === "music") categories = HDDOLBY_MUSIC_CATEGORY;
|
||||
// list(set(movie + tv)):CPython 小整数集合的迭代顺序即升序
|
||||
else categories = [...new Set([...HDDOLBY_MOVIE_CATEGORY, ...HDDOLBY_TV_CATEGORY])].sort((a, b) => a - b);
|
||||
return { keyword: keyword ?? null, page_number: page, page_size: 100, categories, visible: 1 };
|
||||
}
|
||||
|
||||
/** MP `__get_download_url` */
|
||||
export function hddolbyDownloadUrl(def: SpecializedSiteDefinition, id: unknown, downhash: unknown): string {
|
||||
return `${siteBase(def)}download.php?id=${id}&downhash=${downhash}`;
|
||||
}
|
||||
|
||||
export interface HddolbyItem {
|
||||
id: number | string;
|
||||
promotion_time_type?: number;
|
||||
promotion_until?: string;
|
||||
category?: number;
|
||||
name?: string;
|
||||
small_descr?: string;
|
||||
times_completed?: number;
|
||||
size?: number;
|
||||
added?: string;
|
||||
seeders?: number;
|
||||
leechers?: number;
|
||||
downhash?: string;
|
||||
tags?: string | null;
|
||||
[k: string]: unknown;
|
||||
}
|
||||
|
||||
/** MP `__parse_result` */
|
||||
export function parseHddolbyResult(results: HddolbyItem[] | null | undefined, def: SpecializedSiteDefinition) {
|
||||
const torrents: SpecializedTorrent[] = [];
|
||||
if (!results?.length) return torrents;
|
||||
for (const result of results) {
|
||||
const cv = result.category;
|
||||
let category: MpMediaType;
|
||||
if (cv != null && HDDOLBY_MUSIC_CATEGORY.includes(cv)) category = "音乐";
|
||||
else if (cv != null && HDDOLBY_TV_CATEGORY.includes(cv)) category = "电视剧";
|
||||
else if (cv != null && HDDOLBY_MOVIE_CATEGORY.includes(cv)) category = "电影";
|
||||
else category = "未知";
|
||||
const labels: string[] = [];
|
||||
for (const labelId of String(result.tags ?? "").split(";")) {
|
||||
const l = HDDOLBY_LABELS[labelId];
|
||||
if (l !== undefined) labels.push(l);
|
||||
}
|
||||
const t: SpecializedTorrent = {
|
||||
title: result.name,
|
||||
description: result.small_descr,
|
||||
enclosure: hddolbyDownloadUrl(def, result.id, result.downhash),
|
||||
size: result.size,
|
||||
seeders: result.seeders,
|
||||
peers: result.leechers,
|
||||
grabs: result.times_completed,
|
||||
downloadvolumefactor: hddolbyDownloadVolumeFactor(result.promotion_time_type),
|
||||
uploadvolumefactor: hddolbyUploadVolumeFactor(result.promotion_time_type),
|
||||
page_url: `${siteBase(def)}details.php?id=${result.id}&hit=1`,
|
||||
labels,
|
||||
category,
|
||||
site_release_id: String(result.id),
|
||||
};
|
||||
if (result.added) t.pubdate = result.added;
|
||||
if (result.promotion_until) t.freedate = result.promotion_until;
|
||||
if (cv != null) t.category_id = String(cv);
|
||||
torrents.push(t);
|
||||
}
|
||||
return torrents;
|
||||
}
|
||||
|
||||
export const hddolbySpider: SpecializedSpider = {
|
||||
parser: "HDDolby",
|
||||
credential: "apiKey",
|
||||
pageSize: () => 100,
|
||||
async search(ctx, q) {
|
||||
const apikey = await ctx.secret("apiKey", true);
|
||||
const cookie = await ctx.secret("cookie");
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "application/json, text/plain, */*",
|
||||
"x-api-key": apikey!,
|
||||
Referer: siteBase(ctx.def),
|
||||
};
|
||||
if (cookie) headers.cookie = cookie;
|
||||
const data = await apiRequest(ctx, `${hddolbyApiBase(ctx.def)}/api/v1/torrent/search`, {
|
||||
method: "POST",
|
||||
headers,
|
||||
json: hddolbySearchParams(q.keyword, q.mtype, q.page ?? 0),
|
||||
});
|
||||
if (!isPlainObject(data)) throw businessError(ctx, "搜索", "响应结构无效");
|
||||
if (data.error) {
|
||||
const err = data.error;
|
||||
throw businessError(ctx, "搜索", isPlainObject(err) ? err.message : err);
|
||||
}
|
||||
return parseHddolbyResult(data.data as HddolbyItem[], ctx.def);
|
||||
},
|
||||
async resolveDownload(ctx, siteReleaseId, enclosure) {
|
||||
const cookie = await ctx.secret("cookie");
|
||||
const id = checkReleaseId(ctx.def.id, siteReleaseId);
|
||||
// TODO(verify):无 downhash 时是否仅凭 Cookie 可下载
|
||||
const url = enclosure ?? `${siteBase(ctx.def)}download.php?id=${id}`;
|
||||
return downloadTorrentFile(ctx, url, { cookie });
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,70 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/__init__.py @ e6820a8
|
||||
*
|
||||
* 专用 spider 汇总:`SPECIALIZED_SPIDERS` = MP `SPIDER_PARSER_CLASSES`。
|
||||
*/
|
||||
import type { SiteAdapterDeps } from "../deps.ts";
|
||||
import type { SiteAdapter } from "../types.ts";
|
||||
import { createSpecializedSiteAdapter } from "./adapter.ts";
|
||||
import { haidanSpider } from "./haidan.ts";
|
||||
import { hddolbySpider } from "./hddolby.ts";
|
||||
import { mtorrentSpider } from "./mtorrent.ts";
|
||||
import { rousiSpider } from "./rousi.ts";
|
||||
import { sunnyptSpider } from "./sunnypt.ts";
|
||||
import { tnodeSpider } from "./tnode.ts";
|
||||
import { torrentleechSpider } from "./torrentleech.ts";
|
||||
import type { SpecializedParser, SpecializedSiteDefinition, SpecializedSpider } from "./types.ts";
|
||||
import { yemaSpider } from "./yema.ts";
|
||||
|
||||
export const SPECIALIZED_SPIDERS: Record<SpecializedParser, SpecializedSpider> = {
|
||||
TNodeSpider: tnodeSpider,
|
||||
TorrentLeech: torrentleechSpider,
|
||||
mTorrent: mtorrentSpider,
|
||||
Yema: yemaSpider,
|
||||
Haidan: haidanSpider,
|
||||
HDDolby: hddolbySpider,
|
||||
RousiPro: rousiSpider,
|
||||
SunnyPT: sunnyptSpider,
|
||||
};
|
||||
|
||||
export function isSpecializedParser(v: unknown): v is SpecializedParser {
|
||||
return typeof v === "string" && Object.hasOwn(SPECIALIZED_SPIDERS, v);
|
||||
}
|
||||
|
||||
/** 按定义的 parser 键选择专用 spider 创建适配器 */
|
||||
export function createSpecializedAdapterFor(def: SpecializedSiteDefinition, deps: SiteAdapterDeps): SiteAdapter {
|
||||
const spider = SPECIALIZED_SPIDERS[def.parser];
|
||||
if (!spider) throw new Error(`未知专用 spider:${def.parser}`);
|
||||
return createSpecializedSiteAdapter(def, spider, deps);
|
||||
}
|
||||
|
||||
export { createSpecializedSiteAdapter } from "./adapter.ts";
|
||||
export {
|
||||
apiRequest,
|
||||
classifyMessage,
|
||||
downloadTorrentFile,
|
||||
formatTimestampTz,
|
||||
type IndirectRequestConfig,
|
||||
normalizeIndirectDownloadUrl,
|
||||
pyQuote,
|
||||
resolveIndirectDownloadUrl,
|
||||
} from "./common.ts";
|
||||
export * from "./haidan.ts";
|
||||
export * from "./hddolby.ts";
|
||||
export { releaseIdFromPageUrl, specializedTorrentToRelease } from "./map.ts";
|
||||
export * from "./mtorrent.ts";
|
||||
export * from "./rousi.ts";
|
||||
export * from "./sunnypt.ts";
|
||||
export * from "./tnode.ts";
|
||||
export * from "./torrentleech.ts";
|
||||
export type {
|
||||
SpecializedParser,
|
||||
SpecializedSiteDefinition,
|
||||
SpecializedSpider,
|
||||
SpecializedTorrent,
|
||||
SpiderContext,
|
||||
SpiderSearchQuery,
|
||||
} from "./types.ts";
|
||||
export * from "./yema.ts";
|
||||
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/__init__.py @ e6820a8
|
||||
*
|
||||
* 专用 spider 输出(MP TorrentInfo 字段)→ 统一 `SiteRelease`。
|
||||
* 促销类型由 MP 系数组合推出(复用 spider/map.ts `promotionFromFactors`);H&R 未提供时为 unknown;
|
||||
* raw 为完整 MP 字段,enclosure 脱敏(HDDolby downhash、Haidan passkey 等)。
|
||||
*/
|
||||
import { redactUrl } from "../../rss/redact.ts";
|
||||
import { promotionFromFactors } from "../spider/map.ts";
|
||||
import type { SiteRelease } from "../types.ts";
|
||||
import { parseSiteTime } from "../util.ts";
|
||||
import { EXTRA_SECRET_KEYS } from "./common.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedTorrent } from "./types.ts";
|
||||
|
||||
function num(v: unknown): number | null {
|
||||
if (v == null || v === "") return null;
|
||||
const n = typeof v === "number" ? v : Number(v);
|
||||
return Number.isFinite(n) ? n : null;
|
||||
}
|
||||
|
||||
function factor(v: unknown): number | undefined {
|
||||
const n = num(v);
|
||||
return n === null ? undefined : n;
|
||||
}
|
||||
|
||||
/** 从详情链接取资源 ID:torrent_id= → id= → 末段数字(含 #/torrent/detail/1/ 形式) */
|
||||
export function releaseIdFromPageUrl(url: string | undefined): string | null {
|
||||
if (!url) return null;
|
||||
return (
|
||||
/[?&]torrent_id=(\d+)/.exec(url)?.[1] ??
|
||||
/[?&]id=(\d+)/.exec(url)?.[1] ??
|
||||
/\/(\d+)\/?(?:[?#].*)?$/.exec(url)?.[1] ??
|
||||
null
|
||||
);
|
||||
}
|
||||
|
||||
export function specializedCategoryName(def: SpecializedSiteDefinition, id: string | undefined): string | null {
|
||||
if (!id) return null;
|
||||
const named = def.umt.categoryNames?.[id];
|
||||
if (named) return named;
|
||||
for (const k of ["movie", "tv", "music"] as const) {
|
||||
const hit = (def.category?.[k] ?? []).find((c) => String(c.id) === id);
|
||||
if (hit?.desc) return hit.desc;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function specializedTorrentToRelease(
|
||||
t: SpecializedTorrent,
|
||||
def: SpecializedSiteDefinition,
|
||||
observedAt: string,
|
||||
): SiteRelease | null {
|
||||
const id = t.site_release_id ?? releaseIdFromPageUrl(t.page_url);
|
||||
if (!id || !t.title) return null;
|
||||
const tz = def.umt.timezone;
|
||||
const freedate = t.freedate_utc ?? (t.freedate ? parseSiteTime(t.freedate, tz) : null);
|
||||
const raw: Record<string, unknown> = { ...t };
|
||||
if (t.enclosure) raw.enclosure = redactUrl(t.enclosure, EXTRA_SECRET_KEYS);
|
||||
const release: SiteRelease = {
|
||||
siteId: def.id,
|
||||
siteReleaseId: id,
|
||||
title: String(t.title).trim(),
|
||||
category: { id: t.category_id ?? null, name: specializedCategoryName(def, t.category_id) },
|
||||
sizeBytes: num(t.size),
|
||||
seeders: num(t.seeders),
|
||||
leechers: num(t.peers),
|
||||
snatched: num(t.grabs),
|
||||
promotion: promotionFromFactors(factor(t.downloadvolumefactor), factor(t.uploadvolumefactor), freedate),
|
||||
hr: t.hit_and_run === undefined ? "unknown" : t.hit_and_run,
|
||||
uploadedAt: t.pubdate_utc ?? parseSiteTime(t.pubdate, tz),
|
||||
detailUrl: t.page_url ?? `${def.domain}`,
|
||||
tags: t.labels ?? [],
|
||||
raw,
|
||||
observedAt,
|
||||
};
|
||||
if (t.enclosure) release.downloadUrl = redactUrl(t.enclosure, EXTRA_SECRET_KEYS);
|
||||
if (t.description) release.subtitle = String(t.description).trim();
|
||||
if (t.imdbid) release.imdbId = t.imdbid;
|
||||
return release;
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/spider/mtorrent.py @ e6820a8
|
||||
*
|
||||
* `mTorrent` 键的专用 spider 形态,复用 W1-3b 已移植的 mteam/spider.ts(`__get_params`/`__parse_result`)
|
||||
* 与 mteam/client.ts(`__process_response` + 业务码检查),供 `SPECIALIZED_SPIDERS` 统一分派。
|
||||
* 完整适配(详情、商城免费、healthCheck=profile)仍以 `createMteamAdapter` 为准。
|
||||
*/
|
||||
import { mteamPost } from "../mteam/client.ts";
|
||||
import { rootDomain } from "../mteam/definition.ts";
|
||||
import { type MteamTorrentJson, mteamSearchParams, parseMteamResult } from "../mteam/spider.ts";
|
||||
import { SiteError } from "../types.ts";
|
||||
import { checkReleaseId, downloadTorrentFile, siteBase } from "./common.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedSpider, SpecializedTorrent } from "./types.ts";
|
||||
|
||||
export function mtorrentApiBase(def: SpecializedSiteDefinition): string {
|
||||
return (def.umt.apiBaseUrl ?? `https://api.${rootDomain(def.domain)}`).replace(/\/+$/, "");
|
||||
}
|
||||
|
||||
export const mtorrentSpider: SpecializedSpider = {
|
||||
parser: "mTorrent",
|
||||
credential: "apiKey",
|
||||
pageSize: () => 100,
|
||||
async search(ctx, q) {
|
||||
const apikey = (await ctx.secret("apiKey", true))!;
|
||||
const params = mteamSearchParams(q.keyword ?? null, q.mtype, q.page ?? 0);
|
||||
const data = await mteamPost<{ data?: MteamTorrentJson[] | null }>(
|
||||
ctx.http,
|
||||
mtorrentApiBase(ctx.def),
|
||||
apikey,
|
||||
"/api/torrent/search",
|
||||
{ json: params },
|
||||
ctx.signal,
|
||||
);
|
||||
const src = data?.data ?? [];
|
||||
return parseMteamResult(src, siteBase(ctx.def)).map(
|
||||
(t, i): SpecializedTorrent => ({ ...t, site_release_id: String(src[i]!.id) }),
|
||||
);
|
||||
},
|
||||
async resolveDownload(ctx, siteReleaseId) {
|
||||
const apikey = (await ctx.secret("apiKey", true))!;
|
||||
const id = checkReleaseId(ctx.def.id, siteReleaseId);
|
||||
const url = await mteamPost<string | null>(
|
||||
ctx.http,
|
||||
mtorrentApiBase(ctx.def),
|
||||
apikey,
|
||||
"/api/torrent/genDlToken",
|
||||
{ query: { id } },
|
||||
ctx.signal,
|
||||
);
|
||||
if (!url || typeof url !== "string")
|
||||
throw new SiteError(ctx.def.id, "rate_limited", `genDlToken ${id}:未返回下载链接`);
|
||||
// 涉及解析地址的不使用 Cookie 下载
|
||||
return downloadTorrentFile(ctx, url);
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,54 @@
|
||||
import { createMockFetch, ROUSI_DETAIL_OK, ROUSI_SEARCH_ERROR, ROUSI_SEARCH_OK } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { rousiCategoryNamesByIds, rousiSearchParams, rousiSpider } from "./rousi.ts";
|
||||
import { adapterWith, expectSiteError, testDef, withTorrent } from "./testing.ts";
|
||||
|
||||
const def = testDef("RousiPro", { domain: "https://rousi.pro/" });
|
||||
|
||||
describe("RousiSpider(MP rousi.py)", () => {
|
||||
it("__get_params:页码 +1、cat 映射只取第一个、否则按 mtype", () => {
|
||||
expect(rousiCategoryNamesByIds("3, 6,99")).toEqual(["documentary", "variety"]);
|
||||
expect(rousiSearchParams({ keyword: "x", cat: "6,1", page: 0 })).toEqual({
|
||||
page: 1,
|
||||
page_size: 100,
|
||||
keyword: "x",
|
||||
category: "variety",
|
||||
});
|
||||
expect(rousiSearchParams({ mtype: "tv", page: 2 })).toEqual({ page: 3, page_size: 100, category: "tv" });
|
||||
});
|
||||
|
||||
it("列表解析:Bearer、促销 is_active、带时区时间转 UTC、分类 slug", async () => {
|
||||
const m = createMockFetch({ "GET https://rousi.pro/api/v1/torrents": { body: ROUSI_SEARCH_OK } });
|
||||
const page = await adapterWith(def, rousiSpider, m.fetch).search("bear", { mtype: "tv" });
|
||||
expect(m.requests[0]!.headers.authorization).toBe("Bearer APIKEYSECRET");
|
||||
expect(m.requests[0]!.url.searchParams.get("category")).toBe("tv");
|
||||
const [a, b, c] = page.items;
|
||||
expect(a).toMatchObject({
|
||||
siteReleaseId: "501",
|
||||
category: { id: "tv" },
|
||||
sizeBytes: 50000000000,
|
||||
snatched: 77,
|
||||
promotion: { type: "2xfree", downloadFactor: 0, uploadFactor: 2, expiresAt: "2026-09-27T16:00:00.000Z" },
|
||||
uploadedAt: "2026-09-20T02:00:00.000Z",
|
||||
detailUrl: "https://rousi.pro/torrents/501",
|
||||
});
|
||||
expect(a!.raw.category).toBe("电视剧");
|
||||
// 未激活促销 → 1/1
|
||||
expect(b!.promotion).toMatchObject({ type: "none", downloadFactor: 1, uploadFactor: 1 });
|
||||
expect(b!.uploadedAt).toBe("2026-09-19T02:00:00.000Z");
|
||||
expect(c!.raw.category).toBe("未知");
|
||||
});
|
||||
|
||||
it("业务错误 / 缺 Key / 下载经详情接口换取 download_url,直链不带凭据", async () => {
|
||||
const bad = createMockFetch({ "/api/v1/torrents": { body: ROUSI_SEARCH_ERROR } });
|
||||
await expectSiteError(adapterWith(def, rousiSpider, bad.fetch).list(1), "auth_invalid");
|
||||
await expectSiteError(adapterWith(def, rousiSpider, bad.fetch, {}).list(1), "auth_missing");
|
||||
const m = createMockFetch({ "GET https://rousi.pro/api/v1/torrents/501": { body: ROUSI_DETAIL_OK } });
|
||||
const t = withTorrent(m.fetch, (u) => u.host === "dl.rousi.invalid");
|
||||
const target = await adapterWith(def, rousiSpider, t.fetch).downloadTarget("501");
|
||||
expect(target).toMatchObject({ kind: "torrent", infoHash: t.infoHash });
|
||||
expect(m.requests[0]!.headers.authorization).toBe("Bearer APIKEYSECRET");
|
||||
expect(t.hits[0]!.headers.get("authorization")).toBeNull();
|
||||
expect(t.hits[0]!.headers.get("cookie")).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,175 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/spider/rousi.py @ e6820a8
|
||||
*
|
||||
* Rousi.pro PeerGo 兼容 API(parser "RousiPro",Bearer 个人 API Key,需 torrent:read + torrent:download)。
|
||||
* 函数对照 `RousiSpider`:
|
||||
* - `_movie_category/_tv_category/_music_category` → `ROUSI_*_CATEGORY`
|
||||
* - `get_search_page_size` → `pageSize`(100)
|
||||
* - `__get_params` → `rousiSearchParams`;`__get_category_names_by_ids` → `rousiCategoryNamesByIds`
|
||||
* - `__process_response` → `rousiSpider.search` 错误分支;`__parse_result` → `parseRousiResult`
|
||||
* - `__get_download_url`(GET 详情接口,result=data.download_url)→ `rousiSpider.resolveDownload`(现场换票,不生成含 key 的 enclosure)
|
||||
*/
|
||||
import type { MpMediaType } from "../spider/spider.ts";
|
||||
import { extractDomain, normalizeDatetime } from "../user-parsers/common.ts";
|
||||
import {
|
||||
apiRequest,
|
||||
businessError,
|
||||
checkReleaseId,
|
||||
downloadTorrentFile,
|
||||
isoWithZoneToUtc,
|
||||
isPlainObject,
|
||||
resolveIndirectDownloadUrl,
|
||||
toFloat,
|
||||
toInt,
|
||||
} from "./common.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedSpider, SpecializedTorrent, SpiderSearchQuery } from "./types.ts";
|
||||
|
||||
/** API 不支持多分类搜索,每次只使用一个分类 */
|
||||
export const ROUSI_MOVIE_CATEGORY = "movie";
|
||||
export const ROUSI_TV_CATEGORY = "tv";
|
||||
export const ROUSI_MUSIC_CATEGORY = "music";
|
||||
|
||||
const ROUSI_ID_TO_NAME: Record<string, string> = {
|
||||
"1": "movie",
|
||||
"2": "tv",
|
||||
"3": "documentary",
|
||||
"4": "animation",
|
||||
"5": "music",
|
||||
"6": "variety",
|
||||
};
|
||||
|
||||
/** 站点 origin(MP `https://%s` % extract_domain(domain)) */
|
||||
export function rousiBase(def: SpecializedSiteDefinition): string {
|
||||
return (def.umt.apiBaseUrl ?? `https://${extractDomain(def.domain)}`).replace(/\/+$/, "");
|
||||
}
|
||||
|
||||
/** MP `__get_category_names_by_ids` */
|
||||
export function rousiCategoryNamesByIds(cat: string | null | undefined): string[] | null {
|
||||
if (!cat) return null;
|
||||
const ids = cat
|
||||
.split(",")
|
||||
.map((c) => c.trim())
|
||||
.filter(Boolean);
|
||||
const names = ids.filter((id) => id in ROUSI_ID_TO_NAME).map((id) => ROUSI_ID_TO_NAME[id]!);
|
||||
return names.length ? names : null;
|
||||
}
|
||||
|
||||
/** MP `__get_params` */
|
||||
export function rousiSearchParams(q: SpiderSearchQuery): Record<string, string | number> {
|
||||
const params: Record<string, string | number> = { page: Number(q.page ?? 0) + 1, page_size: 100 };
|
||||
if (q.keyword) params.keyword = q.keyword;
|
||||
if (q.cat) {
|
||||
const names = rousiCategoryNamesByIds(q.cat);
|
||||
if (names) params.category = names[0]!;
|
||||
} else if (q.mtype) {
|
||||
if (q.mtype === "movie") params.category = ROUSI_MOVIE_CATEGORY;
|
||||
else if (q.mtype === "tv") params.category = ROUSI_TV_CATEGORY;
|
||||
else if (q.mtype === "music") params.category = ROUSI_MUSIC_CATEGORY;
|
||||
}
|
||||
return params;
|
||||
}
|
||||
|
||||
export interface RousiItem {
|
||||
id: number | string;
|
||||
title?: string;
|
||||
subtitle?: string;
|
||||
category?: string | { slug?: string; name?: string } | null;
|
||||
size?: number | string;
|
||||
seeders?: number | string;
|
||||
leechers?: number | string;
|
||||
downloads?: number | string;
|
||||
created_at?: string;
|
||||
promotion?: { is_active?: boolean; down_multiplier?: number; up_multiplier?: number; until?: string } | null;
|
||||
[k: string]: unknown;
|
||||
}
|
||||
|
||||
/** MP `__parse_result` */
|
||||
export function parseRousiResult(results: RousiItem[] | null | undefined, def: SpecializedSiteDefinition) {
|
||||
const torrents: SpecializedTorrent[] = [];
|
||||
if (!results?.length) return torrents;
|
||||
const base = rousiBase(def);
|
||||
for (const result of results) {
|
||||
const rawCat = result.category;
|
||||
let catVal: string | null | undefined = null;
|
||||
let category: MpMediaType = "未知";
|
||||
if (rawCat && typeof rawCat === "object") catVal = rawCat.slug || rawCat.name;
|
||||
else if (typeof rawCat === "string") catVal = rawCat;
|
||||
if (catVal) {
|
||||
catVal = String(catVal).toLowerCase();
|
||||
if (catVal === ROUSI_MOVIE_CATEGORY) category = "电影";
|
||||
else if (catVal === ROUSI_TV_CATEGORY) category = "电视剧";
|
||||
else if (catVal === ROUSI_MUSIC_CATEGORY) category = "音乐";
|
||||
}
|
||||
// API 后端已处理全站促销优先级,直接使用返回的 promotion 数据
|
||||
let dl = 1.0;
|
||||
let ul = 1.0;
|
||||
let freedate: string | undefined;
|
||||
let freedateUtc: string | undefined;
|
||||
const promotion = result.promotion;
|
||||
if (promotion?.is_active) {
|
||||
dl = toFloat(promotion.down_multiplier ?? 1.0);
|
||||
ul = toFloat(promotion.up_multiplier ?? 1.0);
|
||||
if (promotion.until) {
|
||||
freedate = normalizeDatetime(promotion.until) ?? undefined;
|
||||
freedateUtc = isoWithZoneToUtc(promotion.until);
|
||||
}
|
||||
}
|
||||
const id = result.id;
|
||||
const t: SpecializedTorrent = {
|
||||
title: result.title,
|
||||
description: result.subtitle,
|
||||
size: toInt(result.size),
|
||||
seeders: toInt(result.seeders),
|
||||
peers: toInt(result.leechers),
|
||||
grabs: toInt(result.downloads),
|
||||
downloadvolumefactor: dl,
|
||||
uploadvolumefactor: ul,
|
||||
page_url: `${base}/torrents/${id}`,
|
||||
labels: [],
|
||||
category,
|
||||
site_release_id: String(id),
|
||||
};
|
||||
const pubdate = normalizeDatetime(result.created_at);
|
||||
if (pubdate) t.pubdate = pubdate;
|
||||
const pubUtc = isoWithZoneToUtc(result.created_at);
|
||||
if (pubUtc) t.pubdate_utc = pubUtc;
|
||||
if (freedate) t.freedate = freedate;
|
||||
if (freedateUtc) t.freedate_utc = freedateUtc;
|
||||
if (catVal) t.category_id = catVal;
|
||||
torrents.push(t);
|
||||
}
|
||||
return torrents;
|
||||
}
|
||||
|
||||
export const rousiSpider: SpecializedSpider = {
|
||||
parser: "RousiPro",
|
||||
credential: "apiKey",
|
||||
pageSize: () => 100,
|
||||
async search(ctx, q) {
|
||||
const apikey = await ctx.secret("apiKey", true);
|
||||
const data = await apiRequest(ctx, `${rousiBase(ctx.def)}/api/v1/torrents`, {
|
||||
method: "GET",
|
||||
headers: { Authorization: `Bearer ${apikey}`, Accept: "application/json" },
|
||||
query: rousiSearchParams(q),
|
||||
});
|
||||
if (!isPlainObject(data)) throw businessError(ctx, "搜索", "搜索响应结构无效");
|
||||
if (data.code === 0) {
|
||||
const d = isPlainObject(data.data) ? data.data : {};
|
||||
return parseRousiResult((d.torrents as RousiItem[]) ?? [], ctx.def);
|
||||
}
|
||||
throw businessError(ctx, "搜索", data.message);
|
||||
},
|
||||
async resolveDownload(ctx, siteReleaseId) {
|
||||
const apikey = await ctx.secret("apiKey", true);
|
||||
const id = checkReleaseId(ctx.def.id, siteReleaseId);
|
||||
// MP __get_download_url:method=get, cookie=False, Bearer, result=data.download_url
|
||||
const url = await resolveIndirectDownloadUrl(ctx, `${rousiBase(ctx.def)}/api/v1/torrents/${id}`, {
|
||||
method: "get",
|
||||
header: { Authorization: `Bearer ${apikey}`, Accept: "application/json" },
|
||||
result: "data.download_url",
|
||||
});
|
||||
return downloadTorrentFile(ctx, url);
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,71 @@
|
||||
import {
|
||||
createMockFetch,
|
||||
SUNNYPT_CATEGORIES_OK,
|
||||
SUNNYPT_DOWNLOAD_TOKEN_OK,
|
||||
SUNNYPT_SEARCH_ERROR,
|
||||
SUNNYPT_SEARCH_OK,
|
||||
} from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { sunnyptBuildParams, sunnyptParseCategoryItems, sunnyptSpider } from "./sunnypt.ts";
|
||||
import { adapterWith, expectSiteError, testDef, withTorrent } from "./testing.ts";
|
||||
|
||||
const def = testDef("SunnyPT", {
|
||||
domain: "https://sunnypt.top/",
|
||||
api_url: "https://api.sunnypt.top/api/v1/mp",
|
||||
category: { movie: [{ id: 401 }], tv: [{ id: 402 }] },
|
||||
});
|
||||
|
||||
describe("SunnyPTSpider(MP sunnypt.py)", () => {
|
||||
it("_parse_category_items / _build_params", () => {
|
||||
const map = sunnyptParseCategoryItems(SUNNYPT_CATEGORIES_OK.data);
|
||||
expect(map).toEqual({ movie: ["401", "405"], tv: ["402", "405"], music: ["406"] });
|
||||
expect(sunnyptBuildParams({ keyword: "x", mtype: "tv", page: 1 }, map)).toEqual({
|
||||
page: 2,
|
||||
page_size: 100,
|
||||
sort: "created_at",
|
||||
order: "desc",
|
||||
keyword: "x",
|
||||
media_type: "tv",
|
||||
categories: "402,405",
|
||||
});
|
||||
});
|
||||
|
||||
it("列表解析:分类接口缓存、促销、H&R 布尔、标签", async () => {
|
||||
const m = createMockFetch({
|
||||
"GET https://api.sunnypt.top/api/v1/mp/categories": { body: SUNNYPT_CATEGORIES_OK },
|
||||
"GET https://api.sunnypt.top/api/v1/mp/torrents": { body: SUNNYPT_SEARCH_OK },
|
||||
});
|
||||
const a = adapterWith(def, sunnyptSpider, m.fetch);
|
||||
const page = await a.search("arcane", { mtype: "tv" });
|
||||
await a.search("arcane", { mtype: "tv" });
|
||||
expect(m.requests.filter((r) => r.url.pathname.endsWith("/categories"))).toHaveLength(1);
|
||||
expect(m.requests[0]!.headers["x-api-key"]).toBe("APIKEYSECRET");
|
||||
expect(m.requests[1]!.url.searchParams.get("categories")).toBe("402,405");
|
||||
const [x, y] = page.items;
|
||||
expect(x).toMatchObject({
|
||||
siteReleaseId: "7001",
|
||||
category: { id: "402" },
|
||||
promotion: { type: "free", downloadFactor: 0, uploadFactor: 1, expiresAt: "2026-09-30T15:59:59.000Z" },
|
||||
hr: true,
|
||||
tags: ["官方", "中字"],
|
||||
imdbId: "tt11126994",
|
||||
detailUrl: "https://sunnypt.top/details.php?id=7001",
|
||||
});
|
||||
expect(y).toMatchObject({ hr: false, promotion: { type: "none" }, sizeBytes: 9000000000 });
|
||||
});
|
||||
|
||||
it("业务错误 / 分类接口失败回退配置 / 下载 token 相对地址按 API 基址补全", async () => {
|
||||
const bad = createMockFetch({ "/api/v1/mp/torrents": { body: SUNNYPT_SEARCH_ERROR } });
|
||||
await expectSiteError(adapterWith(def, sunnyptSpider, bad.fetch).list(1), "auth_invalid");
|
||||
const fb = createMockFetch({
|
||||
"/api/v1/mp/categories": { status: 500, body: "x" },
|
||||
"/api/v1/mp/torrents": { body: SUNNYPT_SEARCH_OK },
|
||||
});
|
||||
await adapterWith(def, sunnyptSpider, fb.fetch).search("x", { mtype: "movie" });
|
||||
expect(fb.requests.at(-1)!.url.searchParams.get("categories")).toBe("401");
|
||||
const d = createMockFetch({ "POST /api/v1/mp/torrents/7001/download-token": { body: SUNNYPT_DOWNLOAD_TOKEN_OK } });
|
||||
const t = withTorrent(d.fetch, (u) => u.pathname.endsWith("/download"));
|
||||
await adapterWith(def, sunnyptSpider, t.fetch).downloadTarget("7001");
|
||||
expect(t.hits[0]!.url.toString()).toBe("https://api.sunnypt.top/api/v1/mp/torrents/7001/download?token=TOK");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,225 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/spider/sunnypt.py @ e6820a8
|
||||
*
|
||||
* SunnyPT MoviePilot API(parser "SunnyPT",X-API-Key)。函数对照 `SunnyPTSpider`:
|
||||
* - `_size/_category_cache_ttl` → `SUNNYPT_PAGE_SIZE/SUNNYPT_CATEGORY_CACHE_TTL_MS`
|
||||
* - `_parse_configured_categories` → `sunnyptConfiguredCategories`;`_parse_category_items` → `sunnyptParseCategoryItems`
|
||||
* - `_get_cached_categories/_set_cached_categories`(类级缓存)→ 适配器实例缓存 `ctx.cache`
|
||||
* - `_request_headers` → `sunnyptHeaders`;`_response_data` → `sunnyptResponseData`
|
||||
* - `_load_categories/_async_load_categories` → `sunnyptLoadCategories`
|
||||
* - `_media_type_value` → `mtype` 直用;`_build_params` → `sunnyptBuildParams`;`_parse_result` → `parseSunnyptResult`
|
||||
* - `_process_search_response` + `search` → `sunnyptSpider.search`
|
||||
* - `_build_download_url`(POST download-token,result=data.download_url,result_base_url=api_url)→ `resolveDownload`
|
||||
* 差异:分类接口失败时与 MP 一样回退站点配置分类,但凭据错误(HTTP 401/403)直接抛出,不再静默。
|
||||
*/
|
||||
import type { MpMediaType } from "../spider/spider.ts";
|
||||
import { SiteError } from "../types.ts";
|
||||
import { normalizeDatetime } from "../user-parsers/common.ts";
|
||||
import {
|
||||
apiRequest,
|
||||
businessError,
|
||||
cacheGet,
|
||||
cacheSet,
|
||||
checkReleaseId,
|
||||
downloadTorrentFile,
|
||||
isoWithZoneToUtc,
|
||||
isPlainObject,
|
||||
resolveIndirectDownloadUrl,
|
||||
toFloat,
|
||||
toInt,
|
||||
} from "./common.ts";
|
||||
import type {
|
||||
SpecializedSiteDefinition,
|
||||
SpecializedSpider,
|
||||
SpecializedTorrent,
|
||||
SpiderContext,
|
||||
SpiderSearchQuery,
|
||||
} from "./types.ts";
|
||||
|
||||
export const SUNNYPT_PAGE_SIZE = 100;
|
||||
export const SUNNYPT_CATEGORY_CACHE_TTL_MS = 3600 * 1000;
|
||||
|
||||
export type SunnyptCategoryMap = { movie: string[]; tv: string[]; music: string[] };
|
||||
|
||||
export function sunnyptApiUrl(def: SpecializedSiteDefinition): string {
|
||||
return String(def.umt.apiBaseUrl ?? def.api_url ?? "https://api.sunnypt.top/api/v1/mp").replace(/\/+$/, "");
|
||||
}
|
||||
|
||||
/** MP `_parse_configured_categories` */
|
||||
export function sunnyptConfiguredCategories(config: SpecializedSiteDefinition["category"]): SunnyptCategoryMap {
|
||||
const map: SunnyptCategoryMap = { movie: [], tv: [], music: [] };
|
||||
for (const k of ["movie", "tv", "music"] as const) {
|
||||
for (const item of config?.[k] ?? []) {
|
||||
const id = typeof item === "object" ? item.id : item;
|
||||
if (id != null) map[k].push(String(id));
|
||||
}
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
/** MP `_parse_category_items` */
|
||||
export function sunnyptParseCategoryItems(items: unknown[]): SunnyptCategoryMap {
|
||||
const map: SunnyptCategoryMap = { movie: [], tv: [], music: [] };
|
||||
for (const item of items ?? []) {
|
||||
if (!isPlainObject(item) || item.id == null) continue;
|
||||
const id = String(item.id);
|
||||
for (const mt of (item.media_types as unknown[]) ?? []) {
|
||||
if (mt === "movie" || mt === "tv" || mt === "music") {
|
||||
if (!map[mt].includes(id)) map[mt].push(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
function sunnyptHeaders(apikey: string): Record<string, string> {
|
||||
return { Accept: "application/json", "X-API-Key": apikey };
|
||||
}
|
||||
|
||||
/** MP `_response_data`:code 不为 "0" → 失败 */
|
||||
function sunnyptResponseData(ctx: SpiderContext, payload: unknown, operation: string): unknown {
|
||||
if (!isPlainObject(payload)) throw businessError(ctx, operation, "响应结构无效");
|
||||
if (String(payload.code) !== "0") throw businessError(ctx, operation, payload.msg || "未知错误");
|
||||
return payload.data;
|
||||
}
|
||||
|
||||
/** MP `_load_categories`:读取并缓存分类映射,失败回退站点配置 */
|
||||
export async function sunnyptLoadCategories(ctx: SpiderContext, apikey: string): Promise<SunnyptCategoryMap> {
|
||||
const api = sunnyptApiUrl(ctx.def);
|
||||
const cached = cacheGet<SunnyptCategoryMap>(ctx, `sunnypt:categories:${api}`);
|
||||
if (cached) return cached;
|
||||
const configured = sunnyptConfiguredCategories(ctx.def.category);
|
||||
let items: unknown;
|
||||
try {
|
||||
items = sunnyptResponseData(
|
||||
ctx,
|
||||
await apiRequest(ctx, `${api}/categories`, { method: "GET", headers: sunnyptHeaders(apikey) }),
|
||||
"获取分类",
|
||||
);
|
||||
} catch (e) {
|
||||
if (e instanceof SiteError && e.isAuthError) throw e;
|
||||
return configured;
|
||||
}
|
||||
const map = Array.isArray(items) ? sunnyptParseCategoryItems(items) : null;
|
||||
if (!map || !Object.values(map).some((v) => v.length)) return configured;
|
||||
cacheSet(ctx, `sunnypt:categories:${api}`, map, SUNNYPT_CATEGORY_CACHE_TTL_MS);
|
||||
return map;
|
||||
}
|
||||
|
||||
/** MP `_build_params` */
|
||||
export function sunnyptBuildParams(q: SpiderSearchQuery, categoryMap: SunnyptCategoryMap) {
|
||||
const params: Record<string, string | number> = {
|
||||
page: Number(q.page || 0) + 1,
|
||||
page_size: SUNNYPT_PAGE_SIZE,
|
||||
sort: "created_at",
|
||||
order: "desc",
|
||||
};
|
||||
if (q.keyword) params.keyword = q.keyword;
|
||||
const apiMediaType = q.mtype ?? null;
|
||||
if (apiMediaType) params.media_type = apiMediaType;
|
||||
let categories = q.cat;
|
||||
if (!categories && apiMediaType) categories = (categoryMap[apiMediaType] ?? []).join(",");
|
||||
if (categories) params.categories = categories;
|
||||
return params;
|
||||
}
|
||||
|
||||
export interface SunnyptItem {
|
||||
id: number | string;
|
||||
title?: string;
|
||||
subtitle?: string;
|
||||
media_type?: string;
|
||||
category_id?: number | string;
|
||||
promotion?: { is_active?: boolean; down_multiplier?: number; up_multiplier?: number; until?: string } | null;
|
||||
created_at?: string;
|
||||
size?: number | string;
|
||||
seeders?: number | string;
|
||||
leechers?: number | string;
|
||||
completed?: number | string;
|
||||
details_url?: string;
|
||||
imdb_id?: string;
|
||||
tags?: string[];
|
||||
hit_and_run?: boolean | number;
|
||||
[k: string]: unknown;
|
||||
}
|
||||
|
||||
/** MP `_parse_result` */
|
||||
export function parseSunnyptResult(results: unknown[] | null | undefined): SpecializedTorrent[] {
|
||||
const torrents: SpecializedTorrent[] = [];
|
||||
for (const r of results ?? []) {
|
||||
if (!isPlainObject(r)) continue;
|
||||
const result = r as SunnyptItem;
|
||||
let category: MpMediaType;
|
||||
if (result.media_type === "movie") category = "电影";
|
||||
else if (result.media_type === "tv") category = "电视剧";
|
||||
else if (result.media_type === "music") category = "音乐";
|
||||
else category = "未知";
|
||||
const promotion = result.promotion ?? {};
|
||||
const active = Boolean(promotion.is_active);
|
||||
const t: SpecializedTorrent = {
|
||||
title: result.title,
|
||||
description: result.subtitle,
|
||||
size: toInt(result.size),
|
||||
seeders: toInt(result.seeders),
|
||||
peers: toInt(result.leechers),
|
||||
grabs: toInt(result.completed),
|
||||
downloadvolumefactor: active ? toFloat(promotion.down_multiplier ?? 1.0) : 1.0,
|
||||
uploadvolumefactor: active ? toFloat(promotion.up_multiplier ?? 1.0) : 1.0,
|
||||
labels: result.tags ?? [],
|
||||
hit_and_run: Boolean(result.hit_and_run),
|
||||
category,
|
||||
site_release_id: String(result.id),
|
||||
};
|
||||
if (result.details_url) t.page_url = result.details_url;
|
||||
if (result.imdb_id) t.imdbid = result.imdb_id;
|
||||
if (active && promotion.until) {
|
||||
const fd = normalizeDatetime(promotion.until);
|
||||
if (fd) t.freedate = fd;
|
||||
const fu = isoWithZoneToUtc(promotion.until);
|
||||
if (fu) t.freedate_utc = fu;
|
||||
}
|
||||
const pubdate = normalizeDatetime(result.created_at);
|
||||
if (pubdate) t.pubdate = pubdate;
|
||||
const pu = isoWithZoneToUtc(result.created_at);
|
||||
if (pu) t.pubdate_utc = pu;
|
||||
if (result.category_id != null) t.category_id = String(result.category_id);
|
||||
torrents.push(t);
|
||||
}
|
||||
return torrents;
|
||||
}
|
||||
|
||||
export const sunnyptSpider: SpecializedSpider = {
|
||||
parser: "SunnyPT",
|
||||
credential: "apiKey",
|
||||
pageSize: () => SUNNYPT_PAGE_SIZE,
|
||||
async search(ctx, q) {
|
||||
const apikey = (await ctx.secret("apiKey", true))!;
|
||||
const categoryMap =
|
||||
q.mtype && !q.cat ? await sunnyptLoadCategories(ctx, apikey) : sunnyptConfiguredCategories(ctx.def.category);
|
||||
const data = sunnyptResponseData(
|
||||
ctx,
|
||||
await apiRequest(ctx, `${sunnyptApiUrl(ctx.def)}/torrents`, {
|
||||
method: "GET",
|
||||
headers: sunnyptHeaders(apikey),
|
||||
query: sunnyptBuildParams(q, categoryMap),
|
||||
}),
|
||||
"搜索",
|
||||
);
|
||||
if (!isPlainObject(data)) throw businessError(ctx, "搜索", "data 结构无效");
|
||||
return parseSunnyptResult((data.items as unknown[]) ?? []);
|
||||
},
|
||||
async resolveDownload(ctx, siteReleaseId) {
|
||||
const apikey = (await ctx.secret("apiKey", true))!;
|
||||
const id = checkReleaseId(ctx.def.id, siteReleaseId);
|
||||
const api = sunnyptApiUrl(ctx.def);
|
||||
// MP _build_download_url:POST download-token,result=data.download_url,result_base_url=api_url
|
||||
const url = await resolveIndirectDownloadUrl(ctx, `${api}/torrents/${id}/download-token`, {
|
||||
method: "post",
|
||||
header: { "X-API-Key": apikey, Accept: "application/json" },
|
||||
result: "data.download_url",
|
||||
result_base_url: api,
|
||||
});
|
||||
return downloadTorrentFile(ctx, url);
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,74 @@
|
||||
/** 测试辅助(非公开 API):构造适配器、在 mock fetch 前挂二进制 .torrent 路由。 */
|
||||
import { buildFakeTorrent } from "@umt/testkit";
|
||||
import { createSiteRateLimiter } from "../rate-limit.ts";
|
||||
import { type SiteAdapter, type SiteCredentialKind, SiteError, staticSecretResolver } from "../types.ts";
|
||||
import { createSpecializedSiteAdapter } from "./adapter.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedSpider } from "./types.ts";
|
||||
|
||||
export function testDef(
|
||||
parser: SpecializedSiteDefinition["parser"],
|
||||
over: Partial<SpecializedSiteDefinition> = {},
|
||||
): SpecializedSiteDefinition {
|
||||
return {
|
||||
id: "demo",
|
||||
name: "Demo",
|
||||
domain: "https://pt.example.com/",
|
||||
parser,
|
||||
...over,
|
||||
umt: {
|
||||
version: "[email protected]",
|
||||
kind: "api",
|
||||
displayName: "Demo",
|
||||
timezone: "+08:00",
|
||||
rateLimit: { minIntervalMs: 0, burst: 10 },
|
||||
capabilities: { list: true, search: true, detail: false, download: true, rss: false },
|
||||
...(over.umt ?? {}),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function adapterWith(
|
||||
def: SpecializedSiteDefinition,
|
||||
spider: SpecializedSpider,
|
||||
fetchImpl: typeof globalThis.fetch,
|
||||
secrets: Partial<Record<SiteCredentialKind, string>> = { cookie: "c=COOKIESECRET", apiKey: "APIKEYSECRET" },
|
||||
): SiteAdapter {
|
||||
return createSpecializedSiteAdapter(def, spider, {
|
||||
fetch: fetchImpl,
|
||||
secrets: staticSecretResolver({ [def.id]: secrets }),
|
||||
limiter: createSiteRateLimiter({ minIntervalMs: 0 }),
|
||||
});
|
||||
}
|
||||
|
||||
/** 包一层 fetch:命中 predicate 的请求返回合法 .torrent 字节 */
|
||||
export function withTorrent(
|
||||
inner: typeof globalThis.fetch,
|
||||
predicate: (url: URL) => boolean,
|
||||
name = "demo",
|
||||
): { fetch: typeof globalThis.fetch; infoHash: string; hits: Array<{ url: URL; headers: Headers }> } {
|
||||
const t = buildFakeTorrent({ name });
|
||||
const hits: Array<{ url: URL; headers: Headers }> = [];
|
||||
const f = (async (input: string | URL | Request, init?: RequestInit) => {
|
||||
const url = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url);
|
||||
if (predicate(url)) {
|
||||
hits.push({ url, headers: new Headers(init?.headers) });
|
||||
return new Response(t.bytes as Uint8Array<ArrayBuffer>, {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/x-bittorrent" },
|
||||
});
|
||||
}
|
||||
return inner(input, init);
|
||||
}) as typeof globalThis.fetch;
|
||||
return { fetch: f, infoHash: t.infoHash, hits };
|
||||
}
|
||||
|
||||
export async function expectSiteError(p: Promise<unknown>, kind: string): Promise<SiteError> {
|
||||
try {
|
||||
await p;
|
||||
} catch (e) {
|
||||
if (!(e instanceof SiteError)) throw e;
|
||||
if (e.kind !== kind) throw new Error(`expected SiteError ${kind}, got ${e.kind}: ${e.message}`);
|
||||
return e;
|
||||
}
|
||||
throw new Error(`expected SiteError ${kind}`);
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
import { createMockFetch, TNODE_HOME_HTML, TNODE_HOME_NO_TOKEN, TNODE_SEARCH_OK } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { adapterWith, expectSiteError, testDef, withTorrent } from "./testing.ts";
|
||||
import { parseTnodeToken, tnodeSearchParams, tnodeSpider } from "./tnode.ts";
|
||||
|
||||
const def = testDef("TNodeSpider", { domain: "https://tnode.example/" });
|
||||
|
||||
describe("TNodeSpider(MP tnode.py)", () => {
|
||||
it("__get_params / token 解析", () => {
|
||||
expect(tnodeSearchParams("tt0923811", 1)).toMatchObject({
|
||||
page: 2,
|
||||
size: 100,
|
||||
type: "imdbid",
|
||||
keyword: "tt0923811",
|
||||
});
|
||||
expect(tnodeSearchParams(null)).toMatchObject({ type: "title", keyword: "", category: [501, 502, 503, 504] });
|
||||
expect(parseTnodeToken(TNODE_HOME_HTML)).toBe("CSRF-TOKEN-XYZ");
|
||||
});
|
||||
|
||||
it("列表解析:CSRF token 缓存、系数直取、时间戳转 UTC", async () => {
|
||||
const m = createMockFetch({
|
||||
"GET /": { body: TNODE_HOME_HTML },
|
||||
"POST /api/torrent/advancedSearch": { body: TNODE_SEARCH_OK },
|
||||
});
|
||||
const a = adapterWith(def, tnodeSpider, m.fetch);
|
||||
const page = await a.search("eva");
|
||||
await a.list(1);
|
||||
expect(m.requests.filter((r) => r.url.pathname === "/")).toHaveLength(1);
|
||||
expect(m.requests[1]!.headers["x-csrf-token"]).toBe("CSRF-TOKEN-XYZ");
|
||||
const [x, y] = page.items;
|
||||
expect(x).toMatchObject({
|
||||
siteReleaseId: "3301",
|
||||
promotion: { type: "free", downloadFactor: 0, uploadFactor: 1 },
|
||||
uploadedAt: "2024-09-20T00:00:00.000Z",
|
||||
imdbId: "tt0923811",
|
||||
detailUrl: "https://tnode.example/torrent/info/3301",
|
||||
hr: "unknown",
|
||||
});
|
||||
expect(y!.promotion.type).toBe("2x50%");
|
||||
const t = withTorrent(m.fetch, (u) => u.pathname.startsWith("/api/torrent/download/"));
|
||||
await adapterWith(def, tnodeSpider, t.fetch).downloadTarget("3301");
|
||||
expect(t.hits[0]!.headers.get("cookie")).toBe("c=COOKIESECRET");
|
||||
});
|
||||
|
||||
it("取不到 token → auth_expired;缺 Cookie → auth_missing;搜索 HTTP 403 → auth_invalid", async () => {
|
||||
const m = createMockFetch({ "GET /": { body: TNODE_HOME_NO_TOKEN } });
|
||||
await expectSiteError(adapterWith(def, tnodeSpider, m.fetch).list(1), "auth_expired");
|
||||
await expectSiteError(adapterWith(def, tnodeSpider, m.fetch, {}).list(1), "auth_missing");
|
||||
const f = createMockFetch({ "GET /": { body: TNODE_HOME_HTML }, "/api/torrent/advancedSearch": { status: 403 } });
|
||||
await expectSiteError(adapterWith(def, tnodeSpider, f.fetch).list(1), "auth_invalid");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,127 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/spider/tnode.py @ e6820a8
|
||||
*
|
||||
* TNode(parser "TNodeSpider",Cookie + 首页 CSRF token + `api/torrent/advancedSearch`)。函数对照 `TNodeSpider`:
|
||||
* - `get_search_page_size` → `pageSize`(100)
|
||||
* - `__get_token/__async_get_token`(@cached 24h)→ `tnodeToken`(适配器实例缓存,TTL 24h)
|
||||
* - `__parse_token_response` → `parseTnodeToken`
|
||||
* - `__get_params` → `tnodeSearchParams`;`__parse_result` → `parseTnodeResult`;`__process_response` → `tnodeSpider.search`
|
||||
* - `_downloadurl/_pageurl` → `resolveDownload`(带 Cookie 下载)/ page_url
|
||||
* 差异:取不到 token 时 MP 返回 `(True, [])`;这里抛 auth_expired(首页取不到 token 基本意味着 Cookie 失效)。
|
||||
*/
|
||||
import { SiteError } from "../types.ts";
|
||||
import {
|
||||
apiRequest,
|
||||
cacheGet,
|
||||
cacheSet,
|
||||
checkReleaseId,
|
||||
downloadTorrentFile,
|
||||
formatTimestampTz,
|
||||
isPlainObject,
|
||||
siteBase,
|
||||
textRequest,
|
||||
} from "./common.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedSpider, SpecializedTorrent, SpiderContext } from "./types.ts";
|
||||
|
||||
export const TNODE_TOKEN_TTL_MS = 24 * 3600 * 1000;
|
||||
|
||||
/** MP `__parse_token_response` */
|
||||
export function parseTnodeToken(html: string): string | null {
|
||||
return /<meta name="x-csrf-token" content="(.+?)">/.exec(html)?.[1] ?? null;
|
||||
}
|
||||
|
||||
async function tnodeToken(ctx: SpiderContext, cookie: string): Promise<string> {
|
||||
const key = `tnode:token:${ctx.def.domain}`;
|
||||
const cached = cacheGet<string>(ctx, key);
|
||||
if (cached) return cached;
|
||||
const token = parseTnodeToken(await textRequest(ctx, siteBase(ctx.def), { cookie }));
|
||||
if (!token)
|
||||
throw new SiteError(ctx.def.id, "auth_expired", `${ctx.def.name} 未获取到 token,无法搜索(Cookie 可能失效)`);
|
||||
cacheSet(ctx, key, token, TNODE_TOKEN_TTL_MS);
|
||||
return token;
|
||||
}
|
||||
|
||||
/** MP `__get_params` */
|
||||
export function tnodeSearchParams(keyword: string | null | undefined, page = 0) {
|
||||
return {
|
||||
page: Number(page) + 1,
|
||||
size: 100,
|
||||
type: keyword?.startsWith("tt") ? "imdbid" : "title",
|
||||
keyword: keyword || "",
|
||||
sorter: "id",
|
||||
order: "desc",
|
||||
tags: [],
|
||||
category: [501, 502, 503, 504],
|
||||
medium: [],
|
||||
videoCoding: [],
|
||||
audioCoding: [],
|
||||
resolution: [],
|
||||
group: [],
|
||||
};
|
||||
}
|
||||
|
||||
export interface TnodeItem {
|
||||
id: number | string;
|
||||
title?: string;
|
||||
subtitle?: string;
|
||||
upload_time?: number | string;
|
||||
size?: number;
|
||||
seeding?: number;
|
||||
leeching?: number;
|
||||
complete?: number;
|
||||
downloadRate?: number;
|
||||
uploadRate?: number;
|
||||
imdb?: string;
|
||||
[k: string]: unknown;
|
||||
}
|
||||
|
||||
/** MP `__parse_result` */
|
||||
export function parseTnodeResult(results: TnodeItem[] | null | undefined, def: SpecializedSiteDefinition) {
|
||||
const torrents: SpecializedTorrent[] = [];
|
||||
if (!results?.length) return torrents;
|
||||
const base = siteBase(def);
|
||||
for (const result of results) {
|
||||
const t: SpecializedTorrent = {
|
||||
title: result.title,
|
||||
description: result.subtitle,
|
||||
enclosure: `${base}api/torrent/download/${result.id}`,
|
||||
size: result.size,
|
||||
seeders: result.seeding,
|
||||
peers: result.leeching,
|
||||
grabs: result.complete,
|
||||
page_url: `${base}torrent/info/${result.id}`,
|
||||
site_release_id: String(result.id),
|
||||
};
|
||||
if (result.downloadRate != null) t.downloadvolumefactor = result.downloadRate;
|
||||
if (result.uploadRate != null) t.uploadvolumefactor = result.uploadRate;
|
||||
if (result.imdb) t.imdbid = result.imdb;
|
||||
const pubdate = formatTimestampTz(result.upload_time, def.umt.timezone);
|
||||
if (pubdate) t.pubdate = pubdate;
|
||||
torrents.push(t);
|
||||
}
|
||||
return torrents;
|
||||
}
|
||||
|
||||
export const tnodeSpider: SpecializedSpider = {
|
||||
parser: "TNodeSpider",
|
||||
credential: "cookie",
|
||||
pageSize: () => 100,
|
||||
async search(ctx, q) {
|
||||
const cookie = (await ctx.secret("cookie", true))!;
|
||||
const token = await tnodeToken(ctx, cookie);
|
||||
const data = await apiRequest(ctx, `${siteBase(ctx.def)}api/torrent/advancedSearch`, {
|
||||
method: "POST",
|
||||
headers: { "X-CSRF-TOKEN": token, "Content-Type": "application/json; charset=utf-8", cookie },
|
||||
json: tnodeSearchParams(q.keyword, q.page ?? 0),
|
||||
});
|
||||
const d = isPlainObject(data) && isPlainObject(data.data) ? data.data : {};
|
||||
return parseTnodeResult((d.torrents as TnodeItem[]) ?? [], ctx.def);
|
||||
},
|
||||
async resolveDownload(ctx, siteReleaseId, enclosure) {
|
||||
const cookie = await ctx.secret("cookie", true);
|
||||
const id = checkReleaseId(ctx.def.id, siteReleaseId);
|
||||
return downloadTorrentFile(ctx, enclosure ?? `${siteBase(ctx.def)}api/torrent/download/${id}`, { cookie });
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,64 @@
|
||||
import { createMockFetch, TORRENTLEECH_SEARCH_OK } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { adapterWith, expectSiteError, testDef, withTorrent } from "./testing.ts";
|
||||
import { torrentleechSearchUrl, torrentleechSpider } from "./torrentleech.ts";
|
||||
|
||||
const def = testDef("TorrentLeech", {
|
||||
domain: "https://www.torrentleech.org/",
|
||||
category: { movie: [{ id: 47 }, { id: 8 }], tv: [{ id: 32 }] },
|
||||
umt: {
|
||||
version: "tl@test",
|
||||
kind: "api",
|
||||
displayName: "TL",
|
||||
timezone: "+00:00",
|
||||
rateLimit: { minIntervalMs: 0, burst: 10 },
|
||||
capabilities: { list: true, search: true, detail: false, download: true, rss: false },
|
||||
},
|
||||
});
|
||||
|
||||
describe("TorrentLeech(MP torrentleech.py)", () => {
|
||||
it("__get_search_url:分类 exact 查询与 quote 编码;浏览页码 +1", async () => {
|
||||
expect(torrentleechSearchUrl(def, "The Matrix (1999)", "movie")).toBe(
|
||||
"https://www.torrentleech.org/torrents/browse/list/categories/47,8/exact/1/query/The%20Matrix%20%281999%29",
|
||||
);
|
||||
expect(torrentleechSearchUrl(def, "a/b")).toBe("https://www.torrentleech.org/torrents/browse/list/query/a/b");
|
||||
const m = createMockFetch({ "/torrents/browse/list/page/3": { body: TORRENTLEECH_SEARCH_OK } });
|
||||
await adapterWith(def, torrentleechSpider, m.fetch).list(3);
|
||||
expect(m.requests[0]!.url.pathname).toBe("/torrents/browse/list/page/3");
|
||||
});
|
||||
|
||||
it("列表解析:download_multiplier、请求分类投影、UTC 时间", async () => {
|
||||
const m = createMockFetch({
|
||||
"/torrents/browse/list/categories/47,8/exact/1/query/Matrix": { body: TORRENTLEECH_SEARCH_OK },
|
||||
});
|
||||
const page = await adapterWith(def, torrentleechSpider, m.fetch).search("Matrix", { mtype: "movie" });
|
||||
const [a, b] = page.items;
|
||||
expect(a).toMatchObject({
|
||||
siteReleaseId: "240001",
|
||||
promotion: { type: "free", downloadFactor: 0, uploadFactor: 1 },
|
||||
uploadedAt: "2026-09-20T08:00:00.000Z",
|
||||
imdbId: "tt0133093",
|
||||
category: { id: "47" },
|
||||
detailUrl: "https://www.torrentleech.org/torrent/240001",
|
||||
downloadUrl: "https://www.torrentleech.org/download/240001/The.Matrix.1999.2160p.UHD.BluRay.x265-TL.torrent",
|
||||
});
|
||||
expect(a!.raw.category).toBe("电影");
|
||||
expect(b!.raw.category).toBe("未知");
|
||||
expect(b!.promotion.type).toBe("none");
|
||||
// 关键词搜索不翻页
|
||||
expect((await adapterWith(def, torrentleechSpider, m.fetch).search("Matrix", { page: 2 })).items).toEqual([]);
|
||||
});
|
||||
|
||||
it("中文关键词返回空且不发请求;缺 Cookie;跳登录页 → auth_expired;下载带 Cookie", async () => {
|
||||
const m = createMockFetch({
|
||||
"/torrents/browse/list/page/1": { status: 302, headers: { location: "/user/account/login" } },
|
||||
});
|
||||
expect((await adapterWith(def, torrentleechSpider, m.fetch).search("黑客帝国")).items).toEqual([]);
|
||||
expect(m.requests).toHaveLength(0);
|
||||
await expectSiteError(adapterWith(def, torrentleechSpider, m.fetch, {}).list(1), "auth_missing");
|
||||
await expectSiteError(adapterWith(def, torrentleechSpider, m.fetch).list(1), "auth_expired");
|
||||
const t = withTorrent(m.fetch, (u) => u.pathname.startsWith("/download/"));
|
||||
await adapterWith(def, torrentleechSpider, t.fetch).downloadTarget("240001");
|
||||
expect(t.hits[0]!.headers.get("cookie")).toBe("c=COOKIESECRET");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,121 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/spider/torrentleech.py @ e6820a8
|
||||
*
|
||||
* TorrentLeech JSON 搜索接口(parser "TorrentLeech",Cookie)。函数对照 `TorrentLeech`:
|
||||
* - `get_search_page_size` → `pageSize`(关键词搜索 null,浏览 100)
|
||||
* - `__category_ids` → `torrentleechCategoryIds`;`__get_search_url` → `torrentleechSearchUrl`
|
||||
* - `_browseurl` → `torrentleechBrowseUrl`;`__parse_result` → `parseTorrentleechResult`
|
||||
* - `__process_response` + `search` → `torrentleechSpider.search`(含中文关键词直接返回空,MP 行为)
|
||||
* 差异:MP 只在请求了媒体类型时写 category;这里另外总是输出原始 `category_id`。
|
||||
*/
|
||||
import type { MediaKind } from "../spider/spider.ts";
|
||||
import { containsChinese } from "../user-parsers/common.ts";
|
||||
import {
|
||||
apiRequest,
|
||||
checkReleaseId,
|
||||
downloadTorrentFile,
|
||||
formatTimestampTz,
|
||||
isPlainObject,
|
||||
pyQuote,
|
||||
siteBase,
|
||||
} from "./common.ts";
|
||||
import type { SpecializedSiteDefinition, SpecializedSpider, SpecializedTorrent } from "./types.ts";
|
||||
|
||||
const MEDIA_VALUE = { movie: "电影", tv: "电视剧", music: "音乐" } as const;
|
||||
|
||||
/** MP `__category_ids` */
|
||||
export function torrentleechCategoryIds(def: SpecializedSiteDefinition, mtype?: MediaKind): string[] {
|
||||
if (!mtype) return [];
|
||||
return (def.category?.[mtype] ?? []).filter((i) => i && i.id != null).map((i) => String(i.id));
|
||||
}
|
||||
|
||||
/** MP `__get_search_url` */
|
||||
export function torrentleechSearchUrl(def: SpecializedSiteDefinition, keyword: string, mtype?: MediaKind): string {
|
||||
const domain = siteBase(def);
|
||||
const encoded = pyQuote(keyword);
|
||||
const ids = torrentleechCategoryIds(def, mtype);
|
||||
if (ids.length) return `${domain}torrents/browse/list/categories/${ids.join(",")}/exact/1/query/${encoded}`;
|
||||
return `${domain}torrents/browse/list/query/${encoded}`;
|
||||
}
|
||||
|
||||
export function torrentleechBrowseUrl(def: SpecializedSiteDefinition, page = 0): string {
|
||||
return `${siteBase(def)}torrents/browse/list/page/${Number(page) + 1}`;
|
||||
}
|
||||
|
||||
export interface TorrentleechItem {
|
||||
fid: number | string;
|
||||
filename?: string;
|
||||
name?: string;
|
||||
addedTimestamp?: string | number;
|
||||
size?: number;
|
||||
seeders?: number;
|
||||
leechers?: number;
|
||||
completed?: number;
|
||||
download_multiplier?: number;
|
||||
imdbID?: string;
|
||||
categoryID?: number | string;
|
||||
[k: string]: unknown;
|
||||
}
|
||||
|
||||
/** MP `__parse_result` */
|
||||
export function parseTorrentleechResult(
|
||||
results: TorrentleechItem[] | null | undefined,
|
||||
def: SpecializedSiteDefinition,
|
||||
mtype?: MediaKind,
|
||||
) {
|
||||
const torrents: SpecializedTorrent[] = [];
|
||||
if (!results?.length) return torrents;
|
||||
const domain = siteBase(def);
|
||||
const requested = new Set(torrentleechCategoryIds(def, mtype));
|
||||
for (const result of results) {
|
||||
const t: SpecializedTorrent = {
|
||||
title: result.name,
|
||||
enclosure: `${domain}download/${result.fid}/${result.filename}`,
|
||||
size: result.size,
|
||||
seeders: result.seeders,
|
||||
peers: result.leechers,
|
||||
grabs: result.completed,
|
||||
uploadvolumefactor: 1,
|
||||
page_url: `${domain}torrent/${result.fid}`,
|
||||
site_release_id: String(result.fid),
|
||||
};
|
||||
if (result.download_multiplier != null) t.downloadvolumefactor = result.download_multiplier;
|
||||
if (result.imdbID) t.imdbid = result.imdbID;
|
||||
const pubdate = formatTimestampTz(result.addedTimestamp, def.umt.timezone);
|
||||
if (pubdate) t.pubdate = pubdate;
|
||||
if (requested.size && mtype) {
|
||||
t.category = requested.has(String(result.categoryID)) ? MEDIA_VALUE[mtype] : "未知";
|
||||
}
|
||||
if (result.categoryID != null) t.category_id = String(result.categoryID);
|
||||
torrents.push(t);
|
||||
}
|
||||
return torrents;
|
||||
}
|
||||
|
||||
export const torrentleechSpider: SpecializedSpider = {
|
||||
parser: "TorrentLeech",
|
||||
credential: "cookie",
|
||||
pageSize: (keyword) => (keyword ? null : 100),
|
||||
async search(ctx, q) {
|
||||
// 不支持中文(MP 返回 (True, []))
|
||||
if (q.keyword && containsChinese(q.keyword)) return [];
|
||||
const cookie = (await ctx.secret("cookie", true))!;
|
||||
const url = q.keyword
|
||||
? torrentleechSearchUrl(ctx.def, q.keyword, q.mtype)
|
||||
: torrentleechBrowseUrl(ctx.def, q.page ?? 0);
|
||||
const data = await apiRequest(ctx, url, {
|
||||
method: "GET",
|
||||
headers: { "Content-Type": "application/json; charset=utf-8", cookie },
|
||||
});
|
||||
const list = isPlainObject(data) ? (data.torrentList as TorrentleechItem[]) : [];
|
||||
return parseTorrentleechResult(list ?? [], ctx.def, q.mtype);
|
||||
},
|
||||
async resolveDownload(ctx, siteReleaseId, enclosure) {
|
||||
const cookie = await ctx.secret("cookie", true);
|
||||
const id = checkReleaseId(ctx.def.id, siteReleaseId);
|
||||
// TODO(verify):无缓存直链时文件名段是否可任意
|
||||
return downloadTorrentFile(ctx, enclosure ?? `${siteBase(ctx.def)}download/${id}/${id}.torrent`, { cookie });
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,100 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/__init__.py @ e6820a8
|
||||
*
|
||||
* 专用 spider(API 型站点)共用类型。
|
||||
* - `SpecializedParser` = MP `SPIDER_PARSER_CLASSES` 的键;`SPECIALIZED_SEARCH_ARGUMENTS` 的参数集合 → `SpiderSearchQuery`
|
||||
* - `SpecializedSiteDefinition` = 这些 spider 的 `__init__(indexer)` 读取的 indexer dict 键 + 本项目 `umt` 扩展
|
||||
*/
|
||||
import type { SiteHttpContext } from "../http.ts";
|
||||
import type { SpiderCategoryConfig } from "../spider/schema.ts";
|
||||
import type { MediaKind, SpiderTorrent } from "../spider/spider.ts";
|
||||
import type { DownloadTarget, SiteCapabilities, SiteCredentialKind } from "../types.ts";
|
||||
|
||||
export type SpecializedParser =
|
||||
| "Haidan"
|
||||
| "HDDolby"
|
||||
| "RousiPro"
|
||||
| "SunnyPT"
|
||||
| "TNodeSpider"
|
||||
| "TorrentLeech"
|
||||
| "Yema"
|
||||
| "mTorrent";
|
||||
|
||||
export interface SpecializedSiteDefinition {
|
||||
id: string;
|
||||
name: string;
|
||||
enabled?: boolean;
|
||||
/** MP 用户信息解析器 schema(SiteSchema 值) */
|
||||
schema?: string;
|
||||
provenance?: import("../spider/schema.ts").DefinitionProvenance;
|
||||
/** 站点前端地址(MP indexer['domain'],以 / 结尾) */
|
||||
domain: string;
|
||||
parser: SpecializedParser;
|
||||
timeout?: number;
|
||||
proxy?: boolean;
|
||||
public?: boolean;
|
||||
/** 分类组(TorrentLeech 按媒体类型取分类 ID;SunnyPT 分类接口失败时兜底) */
|
||||
category?: SpiderCategoryConfig;
|
||||
/** 独立 API 地址(MP indexer['api_url'],SunnyPT) */
|
||||
api_url?: string;
|
||||
/** 本项目扩展(MP 无对应) */
|
||||
umt: {
|
||||
version: string;
|
||||
kind: "api";
|
||||
displayName: string;
|
||||
/** 站点时间时区偏移(pubdate/freedate 无时区时按此转 UTC) */
|
||||
timezone: string;
|
||||
rateLimit: { minIntervalMs: number; burst: number };
|
||||
capabilities: SiteCapabilities;
|
||||
categoryNames?: Record<string, string>;
|
||||
/** 覆盖 API 基址(测试指向 mock,或站点换域) */
|
||||
apiBaseUrl?: string;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 专用 spider 输出:MP TorrentInfo 字段 + 本项目扩展:
|
||||
* - `site_release_id`:站点资源 ID(MP 只在 page_url/enclosure 里隐含)
|
||||
* - `pubdate_utc` / `freedate_utc`:源数据自带时区时的精确 UTC(MP normalize_datetime 会丢时区)
|
||||
*/
|
||||
export interface SpecializedTorrent extends SpiderTorrent {
|
||||
site_release_id?: string;
|
||||
pubdate_utc?: string;
|
||||
freedate_utc?: string;
|
||||
}
|
||||
|
||||
/** MP `_SPECIALIZED_SEARCH_ARGUMENTS` 的并集;page 为 MP 内部 0 起页码 */
|
||||
export interface SpiderSearchQuery {
|
||||
keyword?: string | null;
|
||||
mtype?: MediaKind;
|
||||
/** 逗号分隔的站点分类 ID(RousiPro/SunnyPT) */
|
||||
cat?: string;
|
||||
page?: number;
|
||||
}
|
||||
|
||||
export interface SpiderContext {
|
||||
def: SpecializedSiteDefinition;
|
||||
http: SiteHttpContext;
|
||||
/** 取凭据;required 时缺失抛 SiteError('auth_missing') */
|
||||
secret(kind: SiteCredentialKind, required?: boolean): Promise<string | null>;
|
||||
/** 实例内缓存(TNode CSRF token、SunnyPT 分类);值带过期时间 */
|
||||
cache: Map<string, { value: unknown; expiresAt: number }>;
|
||||
userAgent?: string;
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
export interface SpecializedSpider {
|
||||
parser: SpecializedParser;
|
||||
/** 必需凭据(MP search 开头的检查) */
|
||||
credential: SiteCredentialKind;
|
||||
/** MP `get_search_page_size`:null 表示不支持翻页 */
|
||||
pageSize(keyword?: string | null): number | null;
|
||||
search(ctx: SpiderContext, query: SpiderSearchQuery): Promise<SpecializedTorrent[]>;
|
||||
/**
|
||||
* 下载前现场取种子(MP `__get_download_url` + submission `_resolve_indirect_download_url` + `download_torrent`)。
|
||||
* `enclosure` 为最近一次搜索结果里的直链(仅内存,不落库);缺失时按站点规则构造。
|
||||
*/
|
||||
resolveDownload(ctx: SpiderContext, siteReleaseId: string, enclosure?: string): Promise<DownloadTarget>;
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
import {
|
||||
createMockFetch,
|
||||
YEMA_DOWNLOAD_KEY_OK,
|
||||
YEMA_SEARCH_ERROR,
|
||||
YEMA_SEARCH_MOVIE,
|
||||
YEMA_SEARCH_TV_A,
|
||||
} from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { adapterWith, expectSiteError, testDef, withTorrent } from "./testing.ts";
|
||||
import { yemaSpider } from "./yema.ts";
|
||||
|
||||
const def = testDef("Yema", { domain: "https://www.yemapt.org/" });
|
||||
const URL_SEARCH = "POST /openApi/torrent/fetchOpenTorrentList.json";
|
||||
|
||||
describe("YemaSpider(MP yema.py)", () => {
|
||||
it("列表解析:促销枚举、标签表(未知 ID 丢弃)、H&R、非整数 ID 跳过", async () => {
|
||||
const m = createMockFetch({ [URL_SEARCH]: { body: YEMA_SEARCH_MOVIE } });
|
||||
const page = await adapterWith(def, yemaSpider, m.fetch).search("past", { mtype: "movie" });
|
||||
expect(m.requests[0]!.headers.authorization).toBe("APIKEYSECRET");
|
||||
expect(JSON.parse(m.requests[0]!.body!)).toEqual({
|
||||
pageParam: { current: 1, pageSize: 40 },
|
||||
sorter: {},
|
||||
keyword: "past",
|
||||
categoryId: 4,
|
||||
});
|
||||
expect(page.items).toHaveLength(1);
|
||||
expect(page.items[0]).toMatchObject({
|
||||
siteReleaseId: "88001",
|
||||
promotion: { type: "2xfree", downloadFactor: 0, uploadFactor: 2, expiresAt: "2026-09-28T16:00:00.000Z" },
|
||||
tags: ["官方", "中字"],
|
||||
hr: true,
|
||||
detailUrl: "https://www.yemapt.org/#/torrent/detail/88001/",
|
||||
category: { id: "4" },
|
||||
});
|
||||
});
|
||||
|
||||
it("电视剧按 6 个分类逐个查询,部分失败仍合并去重", async () => {
|
||||
let n = 0;
|
||||
const m = createMockFetch({
|
||||
[URL_SEARCH]: () => {
|
||||
n++;
|
||||
return n === 2 ? { body: { success: false, errorMessage: "系统繁忙" } } : { body: YEMA_SEARCH_TV_A };
|
||||
},
|
||||
});
|
||||
const page = await adapterWith(def, yemaSpider, m.fetch).list(1, { mtype: "tv" });
|
||||
expect(m.requests.map((r) => JSON.parse(r.body!).categoryId)).toEqual([5, 6, 13, 14, 15, 17]);
|
||||
expect(page.items.map((i) => i.siteReleaseId)).toEqual(["88002"]);
|
||||
expect(page.items[0]!.promotion).toMatchObject({ type: "50%", downloadFactor: 0.5, uploadFactor: 1.5 });
|
||||
});
|
||||
|
||||
it("全部失败才报错;缺 AuthKey;下载 key 拼成 download1?token=", async () => {
|
||||
const bad = createMockFetch({ [URL_SEARCH]: { body: YEMA_SEARCH_ERROR } });
|
||||
await expectSiteError(adapterWith(def, yemaSpider, bad.fetch).list(1), "auth_invalid");
|
||||
await expectSiteError(adapterWith(def, yemaSpider, bad.fetch, {}).list(1), "auth_missing");
|
||||
const d = createMockFetch({ "POST /openApi/torrent/generateDownloadKey.json": { body: YEMA_DOWNLOAD_KEY_OK } });
|
||||
const t = withTorrent(d.fetch, (u) => u.pathname === "/api/torrent/download1");
|
||||
await adapterWith(def, yemaSpider, t.fetch).downloadTarget("88001");
|
||||
expect(d.requests[0]!.url.searchParams.get("id")).toBe("88001");
|
||||
expect(t.hits[0]!.url.toString()).toBe("https://www.yemapt.org/api/torrent/download1?token=DLKEY-123");
|
||||
expect(t.hits[0]!.headers.get("authorization")).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,226 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/spider/yema.py @ e6820a8
|
||||
*
|
||||
* YemaPT 开放 API(parser "Yema",Authorization: AuthKey)。函数对照 `YemaSpider`:
|
||||
* - `_size/_movie_category/_tv_category/_music_category/_labels` → `YEMA_*`
|
||||
* - `_request_headers` → `yemaHeaders`;`_build_params` → `yemaBuildParams`
|
||||
* - `_search_category_ids` → `yemaSearchCategoryIds`;`_merge_search_results` → `yemaMergeSearchResults`
|
||||
* - `_parse_result` → `parseYemaResult`;`_process_search_response` → `yemaProcessSearchResponse`
|
||||
* - `_prepare_search_requests/_finalize_search_responses/search` → `yemaSpider.search`(全部分类失败才算失败)
|
||||
* - `_download_factor/_upload_factor` → `yemaDownloadFactor/yemaUploadFactor`
|
||||
* - `_build_download_url`(POST generateDownloadKey.json?id=,success/result=data → api/torrent/download1?token=)→ `resolveDownload`
|
||||
*/
|
||||
import type { MpMediaType } from "../spider/spider.ts";
|
||||
import { SiteError } from "../types.ts";
|
||||
import { normalizeDatetime } from "../user-parsers/common.ts";
|
||||
import {
|
||||
apiRequest,
|
||||
businessError,
|
||||
checkReleaseId,
|
||||
downloadTorrentFile,
|
||||
isoWithZoneToUtc,
|
||||
isPlainObject,
|
||||
resolveIndirectDownloadUrl,
|
||||
} from "./common.ts";
|
||||
import type {
|
||||
SpecializedSiteDefinition,
|
||||
SpecializedSpider,
|
||||
SpecializedTorrent,
|
||||
SpiderContext,
|
||||
SpiderSearchQuery,
|
||||
} from "./types.ts";
|
||||
|
||||
/** YemaPT 开放 API 使用更大的分页容量时会返回空结果 */
|
||||
export const YEMA_PAGE_SIZE = 40;
|
||||
export const YEMA_MOVIE_CATEGORY = [4];
|
||||
export const YEMA_TV_CATEGORY = [5, 6, 13, 14, 15, 17];
|
||||
export const YEMA_MUSIC_CATEGORY = [8, 16];
|
||||
export const YEMA_LABELS: Record<string, string> = {
|
||||
"1": "禁转",
|
||||
"2": "首发",
|
||||
"3": "官方",
|
||||
"4": "自制",
|
||||
"5": "国语",
|
||||
"6": "中字",
|
||||
"7": "粤语",
|
||||
"8": "英字",
|
||||
"9": "HDR10",
|
||||
"10": "杜比视界",
|
||||
"11": "分集",
|
||||
"12": "完结",
|
||||
};
|
||||
|
||||
export function yemaSiteUrl(def: SpecializedSiteDefinition): string {
|
||||
return String(def.umt.apiBaseUrl ?? def.domain ?? "https://www.yemapt.org/").replace(/\/+$/, "");
|
||||
}
|
||||
|
||||
function yemaHeaders(apikey: string): Record<string, string> {
|
||||
return { Authorization: apikey, "Content-Type": "application/json", Accept: "application/json, text/plain, */*" };
|
||||
}
|
||||
|
||||
/** MP `_build_params` */
|
||||
export function yemaBuildParams(
|
||||
keyword: string | null | undefined,
|
||||
page: number | undefined,
|
||||
categoryId?: number | null,
|
||||
) {
|
||||
const params: Record<string, unknown> = {
|
||||
pageParam: { current: Number(page || 0) + 1, pageSize: YEMA_PAGE_SIZE },
|
||||
sorter: {},
|
||||
};
|
||||
if (keyword) params.keyword = keyword;
|
||||
if (categoryId != null) params.categoryId = categoryId;
|
||||
return params;
|
||||
}
|
||||
|
||||
/** MP `_search_category_ids`:分类参数不支持数组,逐个查询 */
|
||||
export function yemaSearchCategoryIds(mtype?: SpiderSearchQuery["mtype"]): Array<number | null> {
|
||||
if (mtype === "movie") return [...YEMA_MOVIE_CATEGORY];
|
||||
if (mtype === "tv") return [...YEMA_TV_CATEGORY];
|
||||
if (mtype === "music") return [...YEMA_MUSIC_CATEGORY];
|
||||
return [null];
|
||||
}
|
||||
|
||||
/** MP `_merge_search_results`:按详情链接合并去重,保持首次出现顺序 */
|
||||
export function yemaMergeSearchResults(groups: SpecializedTorrent[][]): SpecializedTorrent[] {
|
||||
const merged: SpecializedTorrent[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (const results of groups) {
|
||||
for (const t of results) {
|
||||
const identity = t.page_url || t.enclosure;
|
||||
if (identity && seen.has(identity)) continue;
|
||||
if (identity) seen.add(identity);
|
||||
merged.push(t);
|
||||
}
|
||||
}
|
||||
return merged;
|
||||
}
|
||||
|
||||
export function yemaDownloadFactor(promotion: string): number {
|
||||
return ({ free: 0, half: 0.5, none: 1 } as Record<string, number>)[promotion] ?? 1;
|
||||
}
|
||||
|
||||
export function yemaUploadFactor(promotion: string): number {
|
||||
return ({ none: 1, one_half: 1.5, double_upload: 2 } as Record<string, number>)[promotion] ?? 1;
|
||||
}
|
||||
|
||||
export interface YemaItem {
|
||||
id: number;
|
||||
showName?: string;
|
||||
shortDesc?: string;
|
||||
categoryId?: number;
|
||||
tagList?: string[];
|
||||
listingTime?: string;
|
||||
fileSize?: number;
|
||||
seedNum?: number;
|
||||
leechNum?: number;
|
||||
completedNum?: number;
|
||||
downloadPromotion?: string;
|
||||
uploadPromotion?: string;
|
||||
downloadPromotionEndTime?: string;
|
||||
hrPunishEnable?: boolean;
|
||||
[k: string]: unknown;
|
||||
}
|
||||
|
||||
/** MP `_parse_result` */
|
||||
export function parseYemaResult(results: unknown[] | null | undefined, def: SpecializedSiteDefinition) {
|
||||
const torrents: SpecializedTorrent[] = [];
|
||||
const site = yemaSiteUrl(def);
|
||||
for (const r of results ?? []) {
|
||||
if (!isPlainObject(r)) continue;
|
||||
const result = r as YemaItem;
|
||||
const cv = result.categoryId;
|
||||
let category: MpMediaType;
|
||||
if (cv != null && YEMA_TV_CATEGORY.includes(cv)) category = "电视剧";
|
||||
else if (cv != null && YEMA_MOVIE_CATEGORY.includes(cv)) category = "电影";
|
||||
else if (cv != null && YEMA_MUSIC_CATEGORY.includes(cv)) category = "音乐";
|
||||
else category = "未知";
|
||||
// MP:label_id in self._labels(键为字符串)
|
||||
const labels = (result.tagList ?? [])
|
||||
.filter((id) => typeof id === "string" && id in YEMA_LABELS)
|
||||
.map((id) => YEMA_LABELS[id]!);
|
||||
const id = result.id;
|
||||
if (typeof id !== "number" || !Number.isInteger(id)) continue;
|
||||
const t: SpecializedTorrent = {
|
||||
title: result.showName,
|
||||
description: result.shortDesc,
|
||||
size: result.fileSize,
|
||||
seeders: result.seedNum,
|
||||
peers: result.leechNum,
|
||||
grabs: result.completedNum,
|
||||
downloadvolumefactor: yemaDownloadFactor(String(result.downloadPromotion || "")),
|
||||
uploadvolumefactor: yemaUploadFactor(String(result.uploadPromotion || "")),
|
||||
page_url: `${site}/#/torrent/detail/${id}/`,
|
||||
labels,
|
||||
hit_and_run: Boolean(result.hrPunishEnable),
|
||||
category,
|
||||
site_release_id: String(id),
|
||||
};
|
||||
const pubdate = normalizeDatetime(result.listingTime);
|
||||
if (pubdate) t.pubdate = pubdate;
|
||||
const pu = isoWithZoneToUtc(result.listingTime);
|
||||
if (pu) t.pubdate_utc = pu;
|
||||
const fd = normalizeDatetime(result.downloadPromotionEndTime);
|
||||
if (fd) t.freedate = fd;
|
||||
const fu = isoWithZoneToUtc(result.downloadPromotionEndTime);
|
||||
if (fu) t.freedate_utc = fu;
|
||||
if (cv != null) t.category_id = String(cv);
|
||||
torrents.push(t);
|
||||
}
|
||||
return torrents;
|
||||
}
|
||||
|
||||
/** MP `_process_search_response` */
|
||||
export function yemaProcessSearchResponse(ctx: SpiderContext, payload: unknown): SpecializedTorrent[] {
|
||||
if (!isPlainObject(payload)) throw businessError(ctx, "搜索", "搜索响应结构无效");
|
||||
if (!payload.success) throw businessError(ctx, "搜索", payload.errorMessage || "未知错误");
|
||||
if (!Array.isArray(payload.data)) throw businessError(ctx, "搜索", "搜索响应 data 不是数组");
|
||||
return parseYemaResult(payload.data, ctx.def);
|
||||
}
|
||||
|
||||
export const yemaSpider: SpecializedSpider = {
|
||||
parser: "Yema",
|
||||
credential: "apiKey",
|
||||
pageSize: () => YEMA_PAGE_SIZE,
|
||||
async search(ctx, q) {
|
||||
const apikey = (await ctx.secret("apiKey", true))!;
|
||||
const url = `${yemaSiteUrl(ctx.def)}/openApi/torrent/fetchOpenTorrentList.json`;
|
||||
const groups: SpecializedTorrent[][] = [];
|
||||
const errors: unknown[] = [];
|
||||
for (const categoryId of yemaSearchCategoryIds(q.mtype)) {
|
||||
try {
|
||||
const payload = await apiRequest(ctx, url, {
|
||||
method: "POST",
|
||||
headers: yemaHeaders(apikey),
|
||||
json: yemaBuildParams(q.keyword, q.page, categoryId),
|
||||
});
|
||||
groups.push(yemaProcessSearchResponse(ctx, payload));
|
||||
} catch (e) {
|
||||
// 凭据错误所有分类都会失败,直接抛出
|
||||
if (e instanceof SiteError && e.isAuthError) throw e;
|
||||
errors.push(e);
|
||||
}
|
||||
}
|
||||
// MP:all(errors) 才算失败
|
||||
if (!groups.length && errors.length) throw errors[0];
|
||||
return yemaMergeSearchResults(groups);
|
||||
},
|
||||
async resolveDownload(ctx, siteReleaseId) {
|
||||
const apikey = (await ctx.secret("apiKey", true))!;
|
||||
const id = checkReleaseId(ctx.def.id, siteReleaseId);
|
||||
const site = yemaSiteUrl(ctx.def);
|
||||
const url = await resolveIndirectDownloadUrl(ctx, `${site}/openApi/torrent/generateDownloadKey.json`, {
|
||||
method: "post",
|
||||
header: { Authorization: apikey, Accept: "application/json" },
|
||||
params: { id: Number(id) },
|
||||
success: "success",
|
||||
result: "data",
|
||||
result_base_url: site,
|
||||
result_path: "api/torrent/download1",
|
||||
result_query_param: "token",
|
||||
});
|
||||
return downloadTorrentFile(ctx, url);
|
||||
},
|
||||
};
|
||||
@@ -1,6 +1,7 @@
|
||||
/** 站点适配器自有类型(不依赖 @umt/contracts;W2 负责映射到 source_release / site_snapshot)。 */
|
||||
|
||||
export type SiteKind = "nexusphp" | "mteam";
|
||||
/** api = 专用 API spider(spiders/*,Haidan/HDDolby/RousiPro/SunnyPT/TNode/TorrentLeech/Yema) */
|
||||
export type SiteKind = "nexusphp" | "mteam" | "api";
|
||||
|
||||
export interface SiteCapabilities {
|
||||
list: boolean;
|
||||
|
||||
@@ -0,0 +1,448 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/__init__.py @ e6820a8
|
||||
*
|
||||
* 站点用户数据解析器基类。
|
||||
*
|
||||
* 迁移自 MoviePilot e6820a8(GPLv3)`app/modules/indexer/parser/__init__.py` `SiteParserBase`,
|
||||
* 函数一一对应(Python 私有 `_x_y` → TS `_xY`):
|
||||
* - `parse` → `parse`(页面获取顺序:首页/登录判定 → `_parse_site_page` → 基础信息 → 详情 → 未读消息 → 流量 → 做种分页)
|
||||
* - `_pase_unread_msgs` → `_paseUnreadMsgs`(保留 MP 拼写)、`_parse_seeding_pages` → `_parseSeedingPages`
|
||||
* - `_prepare_html_text` → `prepareHtmlText`、`num_filesize` → `numFilesize`
|
||||
* - `_get_page_content` → `_getPageContent`(cookie/apikey 两种请求模式、JSON Accept、200/500/403 视为有内容、Cloudflare 挑战返回空)
|
||||
* - `_parse_logged_in` → `_parseLoggedIn`、`clear` / `to_dict` → `clear` / `toDict`
|
||||
* - 抽象方法 `_parse_site_page/_parse_user_base_info/_parse_user_traffic_info/_parse_user_torrent_seeding_info/
|
||||
* _parse_user_detail_info/_parse_message_unread_links/_parse_message_content` → 同名 camelCase
|
||||
*
|
||||
* 与 MP 的差异(安全与健壮性):
|
||||
* 1. 凭据经构造参数注入,只放请求头;`toDict()`/`raw` 不含 cookie/apikey/token。
|
||||
* 2. 做种页与消息页翻页设上限(`MAX_PAGES`=50),防止站点返回自引用"下一页"导致死循环(MP 无上限)。
|
||||
* 3. 网络异常与非 200/500/403 状态:与 MP 一样返回空串继续,另把脱敏后的原因记在 `httpErrors`(入 raw)。
|
||||
* 4. 响应解码:按 Content-Type / `<meta charset>` 选 TextDecoder(MP `get_decoded_html_content` 用 chardet 探测)。
|
||||
*/
|
||||
import { redactText } from "../../rss/redact.ts";
|
||||
import { parseSizeMp } from "../spider/spider.ts";
|
||||
import { extractDomain, isLoggedIn, underChallenge } from "./common.ts";
|
||||
import type { SiteSchemaValue, SiteUnreadMessage, SiteUserInfo, SiteUserParserOptions } from "./types.ts";
|
||||
|
||||
export type Params = Record<string, unknown>;
|
||||
export type Headers_ = Record<string, string>;
|
||||
/** 做种解析返回:下一页地址;null/undefined/false 表示结束("" 表示同地址再取一页,MTorrent 用法) */
|
||||
export type NextPage = string | null | undefined | false;
|
||||
|
||||
const MAX_PAGES = 50;
|
||||
|
||||
/** Python `urljoin` */
|
||||
export function urljoin(base: string, path: string): string {
|
||||
try {
|
||||
return new URL(path, base).toString();
|
||||
} catch {
|
||||
return path;
|
||||
}
|
||||
}
|
||||
|
||||
function detectCharset(contentType: string | null, bytes: Uint8Array): string {
|
||||
const fromHeader = /charset=["']?([\w-]+)/i.exec(contentType ?? "")?.[1];
|
||||
if (fromHeader) return fromHeader.toLowerCase();
|
||||
const head = new TextDecoder("latin1").decode(bytes.subarray(0, 2048));
|
||||
const fromMeta = /<meta[^>]+charset=["']?([\w-]+)/i.exec(head)?.[1];
|
||||
return (fromMeta ?? "utf-8").toLowerCase();
|
||||
}
|
||||
|
||||
function decode(bytes: Uint8Array, charset: string): string {
|
||||
try {
|
||||
return new TextDecoder(charset).decode(bytes);
|
||||
} catch {
|
||||
return new TextDecoder("utf-8").decode(bytes);
|
||||
}
|
||||
}
|
||||
|
||||
export abstract class SiteParserBase {
|
||||
/** 站点模版(MP `schema`) */
|
||||
static schema: SiteSchemaValue | null = null;
|
||||
/** 请求模式 cookie/apikey(MP `request_mode`) */
|
||||
requestMode: "cookie" | "apikey" = "cookie";
|
||||
|
||||
// 站点信息
|
||||
apikey: string | null;
|
||||
token: string | null;
|
||||
protected _siteName: string;
|
||||
protected _siteUrl: string;
|
||||
protected _siteDomain: string;
|
||||
protected _baseUrl: string;
|
||||
protected _siteCookie: string | null;
|
||||
protected _apiUrl: string | null;
|
||||
protected _ua: string | null;
|
||||
protected _indexHtml: string | Record<string, never> = "";
|
||||
protected readonly _opts: SiteUserParserOptions;
|
||||
|
||||
// 用户信息
|
||||
username: string | null = null;
|
||||
userid: string | null = null;
|
||||
userLevel: string | null = null;
|
||||
joinAt: string | null = null;
|
||||
bonus = 0.0;
|
||||
// 流量信息
|
||||
upload = 0;
|
||||
download = 0;
|
||||
ratio = 0;
|
||||
// 做种信息
|
||||
seeding = 0;
|
||||
leeching = 0;
|
||||
seedingSize = 0;
|
||||
leechingSize = 0;
|
||||
uploaded = 0;
|
||||
completed = 0;
|
||||
incomplete = 0;
|
||||
uploadedSize = 0;
|
||||
completedSize = 0;
|
||||
incompleteSize = 0;
|
||||
/** 做种人数, 种子大小 */
|
||||
seedingInfo: Array<[number, number]> = [];
|
||||
// 未读消息
|
||||
messageUnread = 0;
|
||||
messageUnreadContents: Array<[string | null, string | null, string | null]> = [];
|
||||
messageReadForce = false;
|
||||
// 全局附加请求头
|
||||
protected _additionHeaders: Headers_ | null = null;
|
||||
// 用户基础信息
|
||||
protected _userBasicPage: string | null = null;
|
||||
protected _userBasicParams: Params | null = null;
|
||||
protected _userBasicMethod: string | null = null;
|
||||
protected _userBasicHeaders: Headers_ | null = null;
|
||||
// 用户详情
|
||||
protected _userDetailPage: string | null = "userdetails.php?id=";
|
||||
protected _userDetailParams: Params | null = null;
|
||||
protected _userDetailHeaders: Headers_ | null = null;
|
||||
// 流量
|
||||
protected _userTrafficPage: string | null = "index.php";
|
||||
protected _userTrafficParams: Params | null = null;
|
||||
protected _userTrafficHeaders: Headers_ | null = null;
|
||||
// 未读消息
|
||||
protected _userMailUnreadPage: string | null = "messages.php?action=viewmailbox&box=1&unread=yes";
|
||||
protected _sysMailUnreadPage: string | null = "messages.php?action=viewmailbox&box=-2&unread=yes";
|
||||
protected _mailUnreadParams: Params | null = null;
|
||||
protected _mailUnreadHeaders: Headers_ | null = null;
|
||||
protected _mailContentParams: Params | null = null;
|
||||
protected _mailContentHeaders: Headers_ | null = null;
|
||||
// 做种
|
||||
protected _torrentSeedingPage: string | null = "getusertorrentlistajax.php?userid=";
|
||||
protected _torrentSeedingParams: Params | null = null;
|
||||
protected _torrentSeedingHeaders: Headers_ | null = null;
|
||||
/** 错误信息(MP `err_msg`) */
|
||||
errMsg: string | null = null;
|
||||
/** 本项目扩展:请求失败原因(已脱敏) */
|
||||
httpErrors: string[] = [];
|
||||
|
||||
constructor(opts: SiteUserParserOptions) {
|
||||
this._opts = opts;
|
||||
this.apikey = opts.apikey ?? null;
|
||||
this.token = opts.token ?? null;
|
||||
this._siteName = opts.siteName;
|
||||
this._siteUrl = opts.url;
|
||||
const u = new URL(opts.url);
|
||||
this._siteDomain = u.host;
|
||||
this._baseUrl = `${u.protocol}//${u.host}`;
|
||||
this._siteCookie = opts.cookie ?? null;
|
||||
this._apiUrl = opts.apiUrl ?? null;
|
||||
this._ua = opts.ua ?? null;
|
||||
}
|
||||
|
||||
get schema(): SiteSchemaValue {
|
||||
return (this.constructor as typeof SiteParserBase).schema!;
|
||||
}
|
||||
|
||||
/** MP `site_schema` */
|
||||
siteSchema(): SiteSchemaValue {
|
||||
return this.schema;
|
||||
}
|
||||
|
||||
/** MP `num_filesize` = `size_tools.parse_size` */
|
||||
static numFilesize(text: string | number | null | undefined): number {
|
||||
return parseSizeMp(text);
|
||||
}
|
||||
numFilesize(text: string | number | null | undefined): number {
|
||||
return parseSizeMp(text);
|
||||
}
|
||||
|
||||
/** MP `parse` */
|
||||
async parse(): Promise<void> {
|
||||
if (this.requestMode === "apikey") {
|
||||
if (!this.apikey && !this.token) {
|
||||
this.errMsg = this.errMsg ?? "未设置cookie 或 apikey/token";
|
||||
return;
|
||||
}
|
||||
this._indexHtml = {};
|
||||
} else {
|
||||
this._indexHtml = await this._getPageContent(this._siteUrl);
|
||||
if (!(await this._parseLoggedIn(this._indexHtml))) return;
|
||||
}
|
||||
await this._parseSitePage(this._indexHtml as string);
|
||||
if (this._userBasicPage) {
|
||||
await this._parseUserBaseInfo(
|
||||
await this._getPageContent(
|
||||
urljoin(this._baseUrl, this._userBasicPage),
|
||||
this._userBasicParams,
|
||||
this._userBasicHeaders,
|
||||
this._userBasicMethod,
|
||||
),
|
||||
);
|
||||
} else {
|
||||
await this._parseUserBaseInfo(this._indexHtml as string);
|
||||
}
|
||||
if (this._userDetailPage) {
|
||||
await this._parseUserDetailInfo(
|
||||
await this._getPageContent(
|
||||
urljoin(this._baseUrl, this._userDetailPage),
|
||||
this._userDetailParams,
|
||||
this._userDetailHeaders,
|
||||
),
|
||||
);
|
||||
}
|
||||
if (this._opts.fetchMessages) await this._paseUnreadMsgs();
|
||||
if (this._userTrafficPage) {
|
||||
await this._parseUserTrafficInfo(
|
||||
await this._getPageContent(
|
||||
urljoin(this._baseUrl, this._userTrafficPage),
|
||||
this._userTrafficParams,
|
||||
this._userTrafficHeaders,
|
||||
),
|
||||
);
|
||||
}
|
||||
await this._parseSeedingPages();
|
||||
}
|
||||
|
||||
/** MP `_pase_unread_msgs` */
|
||||
protected async _paseUnreadMsgs(): Promise<void> {
|
||||
const unreadMsgLinks: string[] = [];
|
||||
if (this.messageUnread > 0 || this.messageReadForce) {
|
||||
const links = new Set([this._userMailUnreadPage, this._sysMailUnreadPage]);
|
||||
for (const link of links) {
|
||||
if (!link) continue;
|
||||
const msgLinks: string[] = [];
|
||||
let nextPage = await this._parseMessageUnreadLinks(
|
||||
await this._getPageContent(urljoin(this._baseUrl, link), this._mailUnreadParams, this._mailUnreadHeaders),
|
||||
msgLinks,
|
||||
);
|
||||
let guard = 0;
|
||||
while (nextPage && guard++ < MAX_PAGES) {
|
||||
nextPage = await this._parseMessageUnreadLinks(
|
||||
await this._getPageContent(
|
||||
urljoin(this._baseUrl, nextPage),
|
||||
this._mailUnreadParams,
|
||||
this._mailUnreadHeaders,
|
||||
),
|
||||
msgLinks,
|
||||
);
|
||||
}
|
||||
unreadMsgLinks.push(...msgLinks);
|
||||
}
|
||||
}
|
||||
if (unreadMsgLinks.length && !this.messageUnread) this.messageUnread = unreadMsgLinks.length;
|
||||
for (const msgLink of unreadMsgLinks) {
|
||||
const [head, date, content] = await this._parseMessageContent(
|
||||
await this._getPageContent(urljoin(this._baseUrl, msgLink), this._mailContentParams, this._mailContentHeaders),
|
||||
);
|
||||
this.messageUnreadContents.push([head, date, content]);
|
||||
}
|
||||
}
|
||||
|
||||
/** MP `_parse_seeding_pages` */
|
||||
protected async _parseSeedingPages(): Promise<void> {
|
||||
if (!this._torrentSeedingPage) return;
|
||||
let nextPage: NextPage = await this._parseUserTorrentSeedingInfo(
|
||||
await this._getPageContent(
|
||||
urljoin(this._baseUrl, this._torrentSeedingPage),
|
||||
this._torrentSeedingParams,
|
||||
this._torrentSeedingHeaders,
|
||||
),
|
||||
);
|
||||
let guard = 0;
|
||||
while (nextPage !== null && nextPage !== undefined && nextPage !== false && guard++ < MAX_PAGES) {
|
||||
nextPage = await this._parseUserTorrentSeedingInfo(
|
||||
await this._getPageContent(
|
||||
urljoin(urljoin(this._baseUrl, this._torrentSeedingPage), nextPage),
|
||||
this._torrentSeedingParams,
|
||||
this._torrentSeedingHeaders,
|
||||
),
|
||||
true,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** MP `_prepare_html_text`:去掉 `\d+px` 与 `#\d+` 干扰 */
|
||||
static prepareHtmlText(htmlText: string): string {
|
||||
return String(htmlText ?? "")
|
||||
.replace(/\d+px/g, "")
|
||||
.replace(/#\d+/g, "");
|
||||
}
|
||||
protected _prepareHtmlText(htmlText: string): string {
|
||||
return SiteParserBase.prepareHtmlText(htmlText);
|
||||
}
|
||||
|
||||
/** MP `_get_page_content` */
|
||||
protected async _getPageContent(
|
||||
url: string,
|
||||
params?: Params | null,
|
||||
headers?: Headers_ | null,
|
||||
method?: string | null,
|
||||
): Promise<string> {
|
||||
let reqHeaders: Headers_ | null = null;
|
||||
if (this._ua || headers || this._additionHeaders) {
|
||||
reqHeaders = this.requestMode === "apikey" ? {} : { "User-Agent": `${this._ua}` };
|
||||
if (headers) Object.assign(reqHeaders, headers);
|
||||
else reqHeaders["Content-Type"] = "application/x-www-form-urlencoded; charset=UTF-8";
|
||||
if (this._additionHeaders) Object.assign(reqHeaders, this._additionHeaders);
|
||||
}
|
||||
const h = new Headers();
|
||||
for (const [k, v] of Object.entries(reqHeaders ?? {})) if (v != null) h.set(k, String(v));
|
||||
if (this.requestMode !== "apikey" && this._siteCookie) h.set("cookie", this._siteCookie);
|
||||
|
||||
let init: RequestInit;
|
||||
if (method === "post" || (params && Object.keys(params).length > 0)) {
|
||||
if (reqHeaders?.["Content-Type"] === "application/json") {
|
||||
init = { method: "POST", headers: h, body: JSON.stringify(params ?? {}) };
|
||||
} else {
|
||||
if (!h.has("content-type")) h.set("content-type", "application/x-www-form-urlencoded; charset=UTF-8");
|
||||
const form = new URLSearchParams();
|
||||
for (const [k, v] of Object.entries(params ?? {})) {
|
||||
if (v == null) continue;
|
||||
if (Array.isArray(v)) for (const x of v) form.append(k, String(x));
|
||||
else form.append(k, String(v));
|
||||
}
|
||||
init = { method: "POST", headers: h, body: form.toString() };
|
||||
}
|
||||
} else {
|
||||
init = { method: "GET", headers: h };
|
||||
}
|
||||
|
||||
const fetchImpl = this._opts.fetch ?? globalThis.fetch;
|
||||
const timeout = AbortSignal.timeout(this._opts.timeoutMs ?? 60_000);
|
||||
const signal = this._opts.signal ? AbortSignal.any([this._opts.signal, timeout]) : timeout;
|
||||
let res: Response;
|
||||
try {
|
||||
await this._opts.limiter?.acquire(this._opts.signal);
|
||||
res = await fetchImpl(url, { ...init, signal });
|
||||
} catch (e) {
|
||||
if (this._opts.signal?.aborted) throw e;
|
||||
this.httpErrors.push(redactText(`${init.method} ${url}:${timeout.aborted ? "超时" : (e as Error).message}`));
|
||||
return "";
|
||||
}
|
||||
if (![200, 500, 403].includes(res.status)) {
|
||||
await res.body?.cancel().catch(() => {});
|
||||
this.httpErrors.push(redactText(`${init.method} ${url}:HTTP ${res.status}`));
|
||||
return "";
|
||||
}
|
||||
const bytes = new Uint8Array(await res.arrayBuffer());
|
||||
if (reqHeaders && String(reqHeaders.Accept ?? "").includes("application/json")) {
|
||||
try {
|
||||
return JSON.stringify(JSON.parse(new TextDecoder("utf-8").decode(bytes)));
|
||||
} catch {
|
||||
this.httpErrors.push(redactText(`${init.method} ${url}:API 响应 JSON 解析失败`));
|
||||
return "";
|
||||
}
|
||||
}
|
||||
const text = decode(bytes, detectCharset(res.headers.get("content-type"), bytes));
|
||||
if (underChallenge(text)) {
|
||||
this.httpErrors.push(redactText(`${url}:检测到 Cloudflare,请更新 Cookie 和 UA`));
|
||||
return "";
|
||||
}
|
||||
return text;
|
||||
}
|
||||
|
||||
/** MP `_parse_logged_in` */
|
||||
protected _parseLoggedIn(htmlText: unknown): boolean | Promise<boolean> {
|
||||
const loggedIn = isLoggedIn(String(htmlText ?? ""));
|
||||
if (!loggedIn) this.errMsg = "未检测到已登陆，请检查cookies是否过期";
|
||||
return loggedIn;
|
||||
}
|
||||
|
||||
protected abstract _parseSitePage(htmlText: string): void | Promise<void>;
|
||||
protected abstract _parseUserBaseInfo(htmlText: string): void | Promise<void>;
|
||||
protected abstract _parseUserTrafficInfo(htmlText: string): void | Promise<void>;
|
||||
protected abstract _parseUserTorrentSeedingInfo(htmlText: string, multiPage?: boolean): NextPage | Promise<NextPage>;
|
||||
protected abstract _parseUserDetailInfo(htmlText: string): void | Promise<void>;
|
||||
protected abstract _parseMessageUnreadLinks(htmlText: string, msgLinks: string[]): NextPage | Promise<NextPage>;
|
||||
protected abstract _parseMessageContent(
|
||||
htmlText: string,
|
||||
): [string | null, string | null, string | null] | Promise<[string | null, string | null, string | null]>;
|
||||
|
||||
/** MP `clear` */
|
||||
clear(): void {
|
||||
this._indexHtml = "";
|
||||
this.seedingInfo = [];
|
||||
this.messageUnreadContents = [];
|
||||
}
|
||||
|
||||
/** MP `to_dict`:公开字段(snake_case),不含凭据 */
|
||||
toDict(): Record<string, unknown> {
|
||||
return {
|
||||
schema: this.schema,
|
||||
userid: this.userid,
|
||||
username: this.username,
|
||||
user_level: this.userLevel,
|
||||
join_at: this.joinAt,
|
||||
bonus: this.bonus,
|
||||
upload: this.upload,
|
||||
download: this.download,
|
||||
ratio: this.ratio,
|
||||
seeding: this.seeding,
|
||||
leeching: this.leeching,
|
||||
seeding_size: this.seedingSize,
|
||||
leeching_size: this.leechingSize,
|
||||
uploaded: this.uploaded,
|
||||
completed: this.completed,
|
||||
incomplete: this.incomplete,
|
||||
uploaded_size: this.uploadedSize,
|
||||
completed_size: this.completedSize,
|
||||
incomplete_size: this.incompleteSize,
|
||||
seeding_info: this.seedingInfo.map((x) => [...x]),
|
||||
message_unread: this.messageUnread,
|
||||
message_unread_contents: this.messageUnreadContents.map((x) => [...x]),
|
||||
message_read_force: this.messageReadForce,
|
||||
err_msg: this.errMsg,
|
||||
http_errors: [...this.httpErrors],
|
||||
};
|
||||
}
|
||||
|
||||
/** MP `refresh_userdata` 里构造 `SiteUserData` 的部分 */
|
||||
toUserInfo(): SiteUserInfo {
|
||||
const num = (v: unknown) => {
|
||||
const n = typeof v === "number" ? v : Number(v);
|
||||
return Number.isFinite(n) ? n : 0;
|
||||
};
|
||||
const messages: SiteUnreadMessage[] = this.messageUnreadContents.map(([head, date, content]) => ({
|
||||
head,
|
||||
date,
|
||||
content,
|
||||
}));
|
||||
return {
|
||||
schema: this.schema,
|
||||
domain: extractDomain(this._siteUrl) || null,
|
||||
userId: this.userid == null || this.userid === "" ? null : String(this.userid),
|
||||
username: this.username,
|
||||
userLevel: this.userLevel,
|
||||
joinAt: this.joinAt,
|
||||
upload: num(this.upload),
|
||||
download: num(this.download),
|
||||
ratio: num(this.ratio),
|
||||
bonus: num(this.bonus),
|
||||
seeding: num(this.seeding),
|
||||
seedingSize: num(this.seedingSize),
|
||||
leeching: num(this.leeching),
|
||||
leechingSize: num(this.leechingSize),
|
||||
seedingInfo: this.seedingInfo.map(([a, b]) => [num(a), num(b)] as [number, number]),
|
||||
messageUnread: num(this.messageUnread),
|
||||
messageUnreadContents: messages,
|
||||
errMsg: this.errMsg,
|
||||
raw: this.toDict(),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export type SiteParserClass = (new (
|
||||
opts: SiteUserParserOptions,
|
||||
) => SiteParserBase) & {
|
||||
schema: SiteSchemaValue | null;
|
||||
};
|
||||
@@ -0,0 +1,47 @@
|
||||
import {
|
||||
BITPT_USER_DETAILS,
|
||||
BITPT_USER_INDEX,
|
||||
BITPT_USER_SEEDING_EMPTY,
|
||||
BITPT_USER_SEEDING_P1,
|
||||
createMockFetch,
|
||||
} from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { BitptSiteUserInfo } from "./bitpt.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
const GiB = 1024 ** 3;
|
||||
|
||||
describe("BitptSiteUserInfo(MP bitpt.py)", () => {
|
||||
it("#userinfotop 登录判定 → frmtable 键值 → browse.php myseed 分页", async () => {
|
||||
const m = createMockFetch({
|
||||
"/": { body: BITPT_USER_INDEX },
|
||||
"/userdetails.php?uid=55": { body: BITPT_USER_DETAILS },
|
||||
"/browse.php?p=1": { body: BITPT_USER_SEEDING_P1 },
|
||||
"/browse.php?p=2": { body: BITPT_USER_SEEDING_EMPTY },
|
||||
});
|
||||
const info = await runParser(BitptSiteUserInfo, { fetch: m.fetch, cookie: "bitpt=secret" });
|
||||
expect(info).toMatchObject({
|
||||
schema: "Bitpt",
|
||||
userId: "55",
|
||||
username: "gina",
|
||||
userLevel: "Elite",
|
||||
joinAt: "2021-06-07 08:09:10",
|
||||
upload: 3 * 1024 ** 4,
|
||||
download: 1024 ** 4,
|
||||
ratio: 3,
|
||||
bonus: 2048.5,
|
||||
seeding: 2,
|
||||
seedingSize: 6 * GiB,
|
||||
});
|
||||
const seedReq = m.requests.find((r) => r.url.pathname === "/browse.php")!;
|
||||
expect(seedReq.url.search).toBe("?t=myseed&st=2&d=desc&p=1");
|
||||
expect(JSON.stringify(info)).not.toContain("secret");
|
||||
});
|
||||
|
||||
it("未登录(无 #userinfotop):只请求首页", async () => {
|
||||
const m = createMockFetch({ "/": { body: "<html><body>login</body></html>" } });
|
||||
const info = await runParser(BitptSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info.userId).toBeNull();
|
||||
expect(m.requests).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,153 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/bitpt.py @ e6820a8
|
||||
*
|
||||
* 极速之星(bitpt.cn)站点用户信息解析器(原作者 ThedoRap)。
|
||||
*
|
||||
* 函数对照(`BitptSiteUserInfo`):`_parse_site_page` → `_parseSitePage`、`_parse_logged_in` → `_parseLoggedIn`
|
||||
* (`#userinfotop`)、`_parse_user_base_info` → `_parseUserBaseInfo`(`table.frmtable` 两列键值)、
|
||||
* `_parse_user_torrent_seeding_page_info` → `parseUserTorrentSeedingPageInfo`(`table.torrenttable tr.btr td.r`)、
|
||||
* `parse` → `parse`(基类流程后取 uid → `userdetails.php?uid=` → `browse.php?t=myseed&st=2&d=desc&p=n` 逐页累加)。
|
||||
* BeautifulSoup 以 cheerio 等价实现。
|
||||
*
|
||||
* 与 MP 的偏离:末尾不把字段转字符串(同 zhixing.ts);做种翻页上限 50 页。
|
||||
*/
|
||||
import { load } from "cheerio";
|
||||
import { type NextPage, SiteParserBase, urljoin } from "./base.ts";
|
||||
import { normalizeDatetime, pyFloat, pyInt } from "./common.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
const MAX_PAGES = 50;
|
||||
|
||||
export class BitptSiteUserInfo extends SiteParserBase {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.Bitpt;
|
||||
|
||||
protected _torrentSeedingBase: string | undefined;
|
||||
|
||||
protected _parseSitePage(_htmlText: string): void {
|
||||
this._userBasicPage = "userdetails.php?uid={uid}";
|
||||
this._userDetailPage = null;
|
||||
this._userBasicParams = {};
|
||||
this._userTrafficPage = null;
|
||||
this._sysMailUnreadPage = null;
|
||||
this._userMailUnreadPage = null;
|
||||
this._mailUnreadParams = {};
|
||||
this._torrentSeedingBase = "browse.php";
|
||||
this._torrentSeedingParams = { t: "myseed", st: "2", d: "desc" };
|
||||
this._torrentSeedingHeaders = {};
|
||||
this._additionHeaders = {};
|
||||
}
|
||||
|
||||
protected override _parseLoggedIn(htmlText: unknown): boolean {
|
||||
return load(String(htmlText ?? ""))("#userinfotop").length > 0;
|
||||
}
|
||||
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
if (!htmlText) return;
|
||||
const $ = load(htmlText);
|
||||
const table = $("table.frmtable").first();
|
||||
if (!table.length) return;
|
||||
const info: Record<string, string> = {};
|
||||
table.find("tr").each((_, row) => {
|
||||
const cells = $(row).find("td");
|
||||
if (cells.length === 2) info[cells.eq(0).text().trim()] = cells.eq(1).text().trim();
|
||||
});
|
||||
this.userid = info.UID ?? null;
|
||||
this.username = "用户名" in info ? info.用户名!.split(" ")[0]! : null;
|
||||
this.userLevel = info.用户级别 ?? null;
|
||||
this.joinAt = "注册时间" in info ? (normalizeDatetime(info.注册时间) ?? null) : null;
|
||||
this.upload = "上传流量" in info ? this.numFilesize(info.上传流量) : 0;
|
||||
this.download = "下载流量" in info ? this.numFilesize(info.下载流量) : 0;
|
||||
this.ratio = "共享率" in info ? pyFloat(info.共享率) : 0;
|
||||
const bonusMatch = /累计([\d.]+)/.exec(info.星辰 ?? "");
|
||||
this.bonus = bonusMatch ? pyFloat(bonusMatch[1]) : 0;
|
||||
this.messageUnread = 0;
|
||||
if (this._torrentSeedingBase) {
|
||||
this.seeding = 0;
|
||||
this.seedingSize = 0;
|
||||
} else {
|
||||
const seedingInfo = $("div").filter(
|
||||
(_, el) =>
|
||||
$(el).attr("style") ===
|
||||
"margin:0 auto;width:90%;font-size:14px;margin-top:10px;margin-bottom:10px;text-align:center;",
|
||||
);
|
||||
if (seedingInfo.length) {
|
||||
const links = seedingInfo.first().find("a");
|
||||
const text = links.length > 1 ? links.eq(1).text() : "";
|
||||
const m = /当前上传的种子\((\d+)个, 共([\d.]+ [KMGT]B)\)/.exec(text);
|
||||
this.seeding = m ? pyInt(m[1]) : 0;
|
||||
this.seedingSize = m ? this.numFilesize(m[2]) : 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
protected _parseUserTrafficInfo(_htmlText: string): void {}
|
||||
protected _parseUserDetailInfo(_htmlText: string): void {}
|
||||
|
||||
/** 单页做种:返回 [数量, 大小] */
|
||||
parseUserTorrentSeedingPageInfo(htmlText: string): [number, number] {
|
||||
if (!htmlText) return [0, 0];
|
||||
const $ = load(htmlText);
|
||||
const table = $("table.torrenttable").first();
|
||||
if (!table.length) return [0, 0];
|
||||
const rows = table.find("tr").toArray();
|
||||
if (rows.length <= 1) return [0, 0];
|
||||
let count = 0;
|
||||
let size = 0;
|
||||
for (const row of rows.slice(1)) {
|
||||
if (!$(row).hasClass("btr")) continue;
|
||||
const td = $(row).find("td.r").first();
|
||||
if (!td.length) continue;
|
||||
const a = td.find("a").first();
|
||||
const sizeText = a.length ? a.text().trim() : td.text().trim();
|
||||
if (sizeText) {
|
||||
count += 1;
|
||||
size += this.numFilesize(sizeText);
|
||||
}
|
||||
}
|
||||
return [count, size];
|
||||
}
|
||||
|
||||
protected _parseMessageUnreadLinks(_htmlText: string, _msgLinks: string[]): NextPage {
|
||||
return null;
|
||||
}
|
||||
protected _parseMessageContent(_htmlText: string): [string | null, string | null, string | null] {
|
||||
return [null, null, null];
|
||||
}
|
||||
protected _parseUserTorrentSeedingInfo(_htmlText: string): NextPage {
|
||||
return null;
|
||||
}
|
||||
|
||||
override async parse(): Promise<void> {
|
||||
await super.parse();
|
||||
if (this._indexHtml) {
|
||||
const $ = load(String(this._indexHtml));
|
||||
const link = $("a")
|
||||
.filter((_, el) => /userdetails\.php\?uid=\d+/.test($(el).attr("href") ?? ""))
|
||||
.first();
|
||||
if (link.length) {
|
||||
const m = /uid=(\d+)/.exec(link.attr("href") ?? "");
|
||||
if (m) this.userid = m[1]!;
|
||||
}
|
||||
}
|
||||
if (this.userid && this._userBasicPage) {
|
||||
const basicUrl = this._userBasicPage.replace("{uid}", this.userid);
|
||||
this._parseUserBaseInfo(await this._getPageContent(urljoin(this._baseUrl, basicUrl)));
|
||||
}
|
||||
if (this._torrentSeedingBase) {
|
||||
const baseUrl = urljoin(this._baseUrl, this._torrentSeedingBase);
|
||||
const params: Record<string, string> = {};
|
||||
for (const [k, v] of Object.entries(this._torrentSeedingParams ?? {})) params[k] = String(v);
|
||||
for (let page = 1; page <= MAX_PAGES; page++) {
|
||||
params.p = String(page);
|
||||
const [n, s] = this.parseUserTorrentSeedingPageInfo(
|
||||
await this._getPageContent(`${baseUrl}?${new URLSearchParams(params).toString()}`),
|
||||
);
|
||||
this.seeding += n;
|
||||
this.seedingSize += s;
|
||||
if (n === 0) break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,211 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/foundation/text.py @ e6820a8
|
||||
* @mp-source app/foundation/temporal.py @ e6820a8
|
||||
* @mp-source app/domain/site.py @ e6820a8
|
||||
* @mp-source app/adapters/network/cloudflare.py @ e6820a8
|
||||
*
|
||||
* 解析器公共工具:MoviePilot e6820a8(GPLv3)
|
||||
* - `app/foundation/text.py` `parse_int/parse_float/contains_chinese` → `parseIntMp/parseFloatMp/containsChinese`
|
||||
* - `app/foundation/temporal.py` `normalize_datetime/format_timestamp` → 同名(dateparser 的保守子集)
|
||||
* - `app/domain/site.py` `SiteUtils.is_logged_in` → `isLoggedIn`
|
||||
* - `app/adapters/network/cloudflare.py` `under_challenge` → `underChallenge`
|
||||
* - `app/domain/site.py` `extract_domain` → `extractDomain`(与 mteam/definition.ts `rootDomain` 同义)
|
||||
*/
|
||||
import { load } from "cheerio";
|
||||
import { hasChildElements, parseHtml } from "./dom.ts";
|
||||
|
||||
/** MP `parse_int`:去千分位,失败返回 0 */
|
||||
export function parseIntMp(value: string | number | null | undefined): number {
|
||||
if (value == null) return 0;
|
||||
const s = String(value).trim().replace(/,/g, "");
|
||||
if (!/^[+-]?\d+$/.test(s)) return 0;
|
||||
return Number.parseInt(s, 10);
|
||||
}
|
||||
|
||||
/** MP `parse_float`:去千分位,失败返回 0 */
|
||||
export function parseFloatMp(value: string | number | null | undefined): number {
|
||||
if (value == null) return 0;
|
||||
const s = String(value).trim().replace(/,/g, "");
|
||||
if (s === "") return 0;
|
||||
if (!/^[+-]?(\d+\.?\d*|\.\d+)([eE][+-]?\d+)?$/.test(s) && !/^[+-]?(inf|infinity|nan)$/i.test(s)) return 0;
|
||||
const n = Number(s);
|
||||
return Number.isFinite(n) ? n : 0;
|
||||
}
|
||||
|
||||
/** Python `int(str)`(带 strip);非法抛错,与 MP 一致交给调用方 */
|
||||
export function pyInt(value: unknown): number {
|
||||
if (typeof value === "number") return Math.trunc(value);
|
||||
if (typeof value === "boolean") return value ? 1 : 0;
|
||||
const s = String(value ?? "").trim();
|
||||
if (!/^[+-]?\d+$/.test(s)) throw new Error(`invalid literal for int(): '${s}'`);
|
||||
return Number.parseInt(s, 10);
|
||||
}
|
||||
|
||||
/** Python `float(str)`;非法抛错 */
|
||||
export function pyFloat(value: unknown): number {
|
||||
if (typeof value === "number") return value;
|
||||
const s = String(value ?? "").trim();
|
||||
const n = Number(s);
|
||||
if (s === "" || Number.isNaN(n)) throw new Error(`could not convert string to float: '${s}'`);
|
||||
return n;
|
||||
}
|
||||
|
||||
export function containsChinese(value: string | string[] | null | undefined): boolean {
|
||||
if (!value) return false;
|
||||
const s = Array.isArray(value) ? value.join(" ") : value;
|
||||
return /[一-鿿]/.test(s);
|
||||
}
|
||||
|
||||
const MONTHS: Record<string, number> = {
|
||||
jan: 1,
|
||||
feb: 2,
|
||||
mar: 3,
|
||||
apr: 4,
|
||||
may: 5,
|
||||
jun: 6,
|
||||
jul: 7,
|
||||
aug: 8,
|
||||
sep: 9,
|
||||
sept: 9,
|
||||
oct: 10,
|
||||
nov: 11,
|
||||
dec: 12,
|
||||
january: 1,
|
||||
february: 2,
|
||||
march: 3,
|
||||
april: 4,
|
||||
june: 6,
|
||||
july: 7,
|
||||
august: 8,
|
||||
september: 9,
|
||||
october: 10,
|
||||
november: 11,
|
||||
december: 12,
|
||||
};
|
||||
|
||||
const pad = (n: number | string) => String(Number(n)).padStart(2, "0");
|
||||
|
||||
/**
|
||||
* MP `normalize_datetime`(dateparser.parse 后 strftime("%Y-%m-%d %H:%M:%S"),失败原样返回)。
|
||||
* 这里只实现确定性的绝对时间格式:
|
||||
* - `YYYY-MM-DD[ T]HH:MM[:SS][.fff][Z|±HH:MM]`、`YYYY/MM/DD`、`YYYY年MM月DD日`(时区后缀按 dateparser 行为丢弃,保留墙钟时间)
|
||||
* - `DD Mon YYYY[, HH:MM]`、`Mon DD[,] YYYY [HH:MM]`(英文月份)
|
||||
* 相对时间("3 天前"/"2 hours ago")dateparser 会以当前时间换算,本实现不做(原样返回,标记待核验)。
|
||||
*/
|
||||
export function normalizeDatetime(value: string | null | undefined): string | null | undefined {
|
||||
if (!value) return value;
|
||||
const v = String(value).trim();
|
||||
let m =
|
||||
/^(\d{4})[-/.年](\d{1,2})[-/.月](\d{1,2})日?(?:[ T]+(\d{1,2}):(\d{1,2})(?::(\d{1,2}))?(?:\.\d+)?)?\s*(?:Z|[+-]\d{2}:?\d{2}|UTC|GMT)?$/i.exec(
|
||||
v,
|
||||
);
|
||||
if (m) return `${m[1]}-${pad(m[2]!)}-${pad(m[3]!)} ${pad(m[4] ?? 0)}:${pad(m[5] ?? 0)}:${pad(m[6] ?? 0)}`;
|
||||
m =
|
||||
/^(\d{1,2})(?:st|nd|rd|th)?\s+([A-Za-z]{3,9})\.?,?\s+(\d{4})(?:,?\s+(?:at\s+)?(\d{1,2}):(\d{2})(?::(\d{2}))?\s*(am|pm)?)?$/i.exec(
|
||||
v,
|
||||
);
|
||||
if (m && MONTHS[m[2]!.toLowerCase()]) {
|
||||
const h = ampm(m[4], m[7]);
|
||||
return `${m[3]}-${pad(MONTHS[m[2]!.toLowerCase()]!)}-${pad(m[1]!)} ${pad(h)}:${pad(m[5] ?? 0)}:${pad(m[6] ?? 0)}`;
|
||||
}
|
||||
m =
|
||||
/^([A-Za-z]{3,9})\.?\s+(\d{1,2})(?:st|nd|rd|th)?,?\s+(\d{4})(?:,?\s+(?:at\s+)?(\d{1,2}):(\d{2})(?::(\d{2}))?\s*(am|pm)?)?$/i.exec(
|
||||
v,
|
||||
);
|
||||
if (m && MONTHS[m[1]!.toLowerCase()]) {
|
||||
const h = ampm(m[4], m[7]);
|
||||
return `${m[3]}-${pad(MONTHS[m[1]!.toLowerCase()]!)}-${pad(m[2]!)} ${pad(h)}:${pad(m[5] ?? 0)}:${pad(m[6] ?? 0)}`;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function ampm(h: string | undefined, suffix: string | undefined): number {
|
||||
let n = Number(h ?? 0);
|
||||
if (suffix?.toLowerCase() === "pm" && n < 12) n += 12;
|
||||
if (suffix?.toLowerCase() === "am" && n === 12) n = 0;
|
||||
return n;
|
||||
}
|
||||
|
||||
/**
|
||||
* MP `format_timestamp`:非纯数字字符串原样返回;Unix 秒 → 本地时间 "YYYY-MM-DD HH:MM:SS"。
|
||||
* MP 用进程本地时区;这里用显式 `tzOffsetMinutes`(默认 +480,即 Asia/Shanghai,MP 容器默认 TZ)。
|
||||
*/
|
||||
export function formatTimestamp(timestamp: unknown, tzOffsetMinutes = 480): string | undefined {
|
||||
if (timestamp == null) return undefined;
|
||||
if (typeof timestamp === "string" && !/^\d+$/.test(timestamp)) return timestamp;
|
||||
const n = Number(timestamp);
|
||||
if (!Number.isFinite(n)) return String(timestamp);
|
||||
const d = new Date(Math.trunc(n) * 1000 + tzOffsetMinutes * 60_000);
|
||||
return `${d.getUTCFullYear()}-${pad(d.getUTCMonth() + 1)}-${pad(d.getUTCDate())} ${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())}:${pad(d.getUTCSeconds())}`;
|
||||
}
|
||||
|
||||
/** MP `SiteUtils.is_logged_in` */
|
||||
export function isLoggedIn(htmlText: string): boolean {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return false;
|
||||
if (html.nodes("//input[@type='password']").length) return false;
|
||||
const xpaths = [
|
||||
'//a[contains(@href, "logout") or contains(@data-url, "logout") or contains(@href, "mybonus") or contains(@onclick, "logout") or contains(@href, "usercp") or contains(@lay-on, "logout")]',
|
||||
'//form[contains(@action, "logout")]',
|
||||
'//div[@class="user-info-side"]',
|
||||
'//a[@id="myitem"]',
|
||||
];
|
||||
return xpaths.some((x) => html.nodes(x).length > 0);
|
||||
}
|
||||
|
||||
const CHALLENGE_TITLES = ["Just a moment...", "请稍候…", "DDOS-GUARD"];
|
||||
const CHALLENGE_SELECTORS = [
|
||||
"#cf-challenge-running",
|
||||
".ray_id",
|
||||
".attack-box",
|
||||
"#cf-please-wait",
|
||||
"#challenge-spinner",
|
||||
"#trk_jschal_js",
|
||||
"td.info #js_info",
|
||||
"div.vc div.text-box h2",
|
||||
];
|
||||
|
||||
/** MP `under_challenge`:Cloudflare / DDoS-Guard 挑战页 */
|
||||
export function underChallenge(htmlText: string | null | undefined): boolean {
|
||||
if (!htmlText) return false;
|
||||
const $ = load(htmlText);
|
||||
const title = $("title").text().trim().toLowerCase();
|
||||
if (CHALLENGE_TITLES.some((t) => t.toLowerCase() === title)) return true;
|
||||
return CHALLENGE_SELECTORS.some((s) => $(s).length > 0);
|
||||
}
|
||||
|
||||
const SPECIAL_SITE_DOMAINS = ["u2.dmhy.org", "pt.ecust.pp.ua", "pt.gtkpw.xyz", "pt.gtk.pw"];
|
||||
|
||||
/** MP `site_rules.extract_domain`:特殊域名表 → netloc 超过 3 级原样 → 否则取最后两级 */
|
||||
export function extractDomain(url: string | null | undefined): string {
|
||||
if (!url) return "";
|
||||
for (const d of SPECIAL_SITE_DOMAINS) if (url.includes(d)) return d;
|
||||
let netloc: string;
|
||||
if (!url.startsWith("http")) netloc = url;
|
||||
else {
|
||||
try {
|
||||
netloc = new URL(url).host;
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
if (!netloc) return "";
|
||||
const labels = netloc.split(".");
|
||||
if (labels.length > 3) return netloc;
|
||||
return labels.slice(-2).join(".");
|
||||
}
|
||||
|
||||
/** Python `re.search(...)`: 返回 match 或 null(显式 helper,方便逐行对照) */
|
||||
export function reSearch(pattern: RegExp, text: string | null | undefined): RegExpExecArray | null {
|
||||
if (text == null) return null;
|
||||
const re = new RegExp(pattern.source, pattern.flags.replace("g", ""));
|
||||
return re.exec(text);
|
||||
}
|
||||
|
||||
/** Python `re.findall`(带分组时返回分组数组) */
|
||||
export function reFindall(pattern: RegExp, text: string): string[][] {
|
||||
const re = new RegExp(pattern.source, pattern.flags.includes("g") ? pattern.flags : `${pattern.flags}g`);
|
||||
return [...text.matchAll(re)].map((m) => m.slice(1));
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import { createMockFetch, DISCUZ_USER_INDEX, DISCUZ_USER_SEEDING, DISCUZ_USER_SPACE } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { DiscuzUserInfo } from "./discuz.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
describe("DiscuzUserInfo(MP discuz.py)", () => {
|
||||
it("&uid= 链接定位用户,空间页取等级/积分/流量,做种表按表头定位列", async () => {
|
||||
const m = createMockFetch({
|
||||
"/": { body: DISCUZ_USER_INDEX },
|
||||
"/home.php?uid=42": { body: DISCUZ_USER_SPACE },
|
||||
"/forum.php?mod=torrents": { body: DISCUZ_USER_SEEDING },
|
||||
});
|
||||
const info = await runParser(DiscuzUserInfo, { fetch: m.fetch });
|
||||
expect(info).toMatchObject({
|
||||
schema: "DiscuzX",
|
||||
userId: "42",
|
||||
username: "frank",
|
||||
userLevel: "Lv3",
|
||||
joinAt: "2018-02-03 04:05:00",
|
||||
bonus: 520,
|
||||
upload: 2 * 1024 ** 4,
|
||||
download: 500 * 1024 ** 3,
|
||||
seeding: 1,
|
||||
seedingInfo: [[6, 8 * 1024 ** 3]],
|
||||
});
|
||||
});
|
||||
|
||||
it("未登录(无登出链接):errMsg", async () => {
|
||||
const m = createMockFetch({
|
||||
"/": { body: "<html><body><a href='member.php?mod=logging&action=login'>登录</a></body></html>" },
|
||||
});
|
||||
const info = await runParser(DiscuzUserInfo, { fetch: m.fetch });
|
||||
expect(info.errMsg).toContain("未检测到已登陆");
|
||||
expect(info.userId).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,105 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/discuz.py @ e6820a8
|
||||
*
|
||||
* Discuz!X 站点用户信息解析器。
|
||||
*
|
||||
* 函数对照(`DiscuzUserInfo`):`_parse_user_base_info` → `_parseUserBaseInfo`、`_parse_site_page` → 空、
|
||||
* `_parse_user_detail_info` → `_parseUserDetailInfo`、`_parse_user_torrent_seeding_info` →
|
||||
* `_parseUserTorrentSeedingInfo`、`_parse_user_traffic_info` / `_parse_message_unread_links` / `_parse_message_content` → 空。
|
||||
* XPath 与正则逐字保留(含 MP 把"分享率"括号值写入 bonus 的原行为,随后被"积分"覆盖)。
|
||||
*/
|
||||
import { type NextPage, SiteParserBase } from "./base.ts";
|
||||
import { normalizeDatetime, parseFloatMp, parseIntMp, reSearch } from "./common.ts";
|
||||
import { hasChildElements, parseHtml } from "./dom.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
export class DiscuzUserInfo extends SiteParserBase {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.DiscuzX;
|
||||
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
const html = parseHtml(this._prepareHtmlText(htmlText));
|
||||
if (!html) return;
|
||||
const userInfo = html.nodes('//a[contains(@href, "&uid=")]');
|
||||
if (!userInfo.length) return;
|
||||
const href = userInfo[0]!.attr("href") ?? "";
|
||||
const m = reSearch(/&uid=(\d+)/, href);
|
||||
if (m?.[0].trim()) {
|
||||
this.userid = m[1]!;
|
||||
this._torrentSeedingPage = "forum.php?&mod=torrents&cat_5up=on";
|
||||
this._userDetailPage = href;
|
||||
// lxml `.text`:首个子节点之前的文本
|
||||
const firstChild = userInfo[0]!.node.firstChild;
|
||||
this.username = firstChild && firstChild.nodeType === 3 ? (firstChild.nodeValue ?? "").trim() : "";
|
||||
}
|
||||
}
|
||||
|
||||
protected _parseSitePage(_htmlText: string): void {}
|
||||
|
||||
/** 解析用户额外信息,加入时间,等级 */
|
||||
protected _parseUserDetailInfo(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
const levels = html.strings('//a[contains(@href, "usergroup")]/text()');
|
||||
if (levels.length) this.userLevel = levels[levels.length - 1]!.trim();
|
||||
const joinAt = html.strings('//li[em[text()="注册时间"]]/text()');
|
||||
if (joinAt.length) this.joinAt = normalizeDatetime(joinAt[0]!.trim()) ?? null;
|
||||
const ratioText = html.strings('//li[contains(.//text(), "分享率")]//text()');
|
||||
if (ratioText.length) {
|
||||
const m = reSearch(/\(([\d,.]+)\)/, ratioText[0]);
|
||||
if (m?.[1]?.trim()) this.bonus = parseFloatMp(m[1]);
|
||||
}
|
||||
const bonusText = html.strings('//li[em[text()="积分"]]/text()');
|
||||
if (bonusText.length) this.bonus = parseFloatMp(bonusText[0]!.trim());
|
||||
const upload = html.strings('//li[em[contains(text(),"上传量")]]/text()');
|
||||
if (upload.length) this.upload = this.numFilesize(upload[0]!.trim().split("/").pop());
|
||||
const download = html.strings('//li[em[contains(text(),"下载量")]]/text()');
|
||||
if (download.length) this.download = this.numFilesize(download[0]!.trim().split("/").pop());
|
||||
}
|
||||
|
||||
protected _parseUserTorrentSeedingInfo(htmlText: string): NextPage {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return null;
|
||||
let sizeCol = 3;
|
||||
let seedersCol = 4;
|
||||
if (html.nodes('//tr[position()=1]/td[.//img[@class="size"] and .//img[@alt="size"]]').length) {
|
||||
sizeCol =
|
||||
html.nodes('//tr[position()=1]/td[.//img[@class="size"] and .//img[@alt="size"]]/preceding-sibling::td')
|
||||
.length + 1;
|
||||
}
|
||||
if (html.nodes('//tr[position()=1]/td[.//img[@class="seeders"] and .//img[@alt="seeders"]]').length) {
|
||||
seedersCol =
|
||||
html.nodes('//tr[position()=1]/td[.//img[@class="seeders"] and .//img[@alt="seeders"]]/preceding-sibling::td')
|
||||
.length + 1;
|
||||
}
|
||||
let pageSeeding = 0;
|
||||
let pageSeedingSize = 0;
|
||||
const pageSeedingInfo: Array<[number, number]> = [];
|
||||
const sizes = html.nodes(`//tr[position()>1]/td[${sizeCol}]`);
|
||||
const seeders = html.strings(`//tr[position()>1]/td[${seedersCol}]//text()`);
|
||||
if (sizes.length && seeders.length) {
|
||||
pageSeeding = sizes.length;
|
||||
for (let i = 0; i < sizes.length; i++) {
|
||||
const size = this.numFilesize(sizes[i]!.text().trim());
|
||||
pageSeedingSize += size;
|
||||
pageSeedingInfo.push([parseIntMp(seeders[i]), size]);
|
||||
}
|
||||
}
|
||||
this.seeding += pageSeeding;
|
||||
this.seedingSize += pageSeedingSize;
|
||||
this.seedingInfo.push(...pageSeedingInfo);
|
||||
const next = html.strings('//a[contains(.//text(), "下一页") or contains(.//text(), "下一頁")]/@href');
|
||||
return next.length ? next[next.length - 1]!.trim() : null;
|
||||
}
|
||||
|
||||
protected _parseUserTrafficInfo(_htmlText: string): void {}
|
||||
|
||||
protected _parseMessageUnreadLinks(_htmlText: string, _msgLinks: string[]): NextPage {
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseMessageContent(_htmlText: string): [string | null, string | null, string | null] {
|
||||
return [null, null, null];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/foundation/dom.py @ e6820a8
|
||||
*
|
||||
* lxml `etree.HTML(...).xpath(...)` 的 TS 等价层(站点用户信息解析器专用)。
|
||||
*
|
||||
* MP 解析器(app/modules/indexer/parser/*.py,提交 e6820a8,GPLv3)全部用 lxml XPath 1.0 取值;
|
||||
* 为了让 XPath 表达式能逐字迁移,本层:
|
||||
* 1. 用 htmlparser2(经 cheerio,HTML 模式)解析——与 libxml2 HTML 解析器一样**不插入 <tbody>**,
|
||||
* 标签/属性名小写,实体解码(` ` → U+00A0);
|
||||
* 2. 把 DOM 复制成 @xmldom/xmldom 文档,交给 `xpath` 包求值(XPath 1.0,与 lxml 同语义);
|
||||
* 3. 近似 libxml2 的 "blank" 规则:html/head/table/thead/tbody/tfoot/tr/ul/ol/select 下的纯空白文本节点丢弃
|
||||
* (影响 `//text()[0]` 类取值)。其他 libxml2 容错修复(自动补 <p>/<body> 等)未模拟。
|
||||
*
|
||||
* 结果约定同 lxml:元素 → XNode;`@attr` → 属性值字符串;`text()` → 文本字符串;`string(.)`/`count()` → 标量。
|
||||
*/
|
||||
import { DOMImplementation } from "@xmldom/xmldom";
|
||||
import { load } from "cheerio";
|
||||
import * as xpath from "xpath";
|
||||
|
||||
type XmlNode = Node;
|
||||
|
||||
/** cheerio/domhandler 节点的结构子集(避免直接依赖 domhandler 类型) */
|
||||
interface DhNode {
|
||||
type: string;
|
||||
name?: string;
|
||||
data?: string;
|
||||
attribs?: Record<string, string>;
|
||||
children?: DhNode[];
|
||||
}
|
||||
|
||||
const BLANK_DROP_PARENTS = new Set(["html", "head", "table", "thead", "tbody", "tfoot", "tr", "ul", "ol", "select"]);
|
||||
|
||||
function copyInto(doc: Document, parent: XmlNode, src: DhNode, parentName: string): void {
|
||||
if (src.type === "text") {
|
||||
const data = src.data ?? "";
|
||||
if (data.trim() === "" && BLANK_DROP_PARENTS.has(parentName)) return;
|
||||
parent.appendChild(doc.createTextNode(data));
|
||||
return;
|
||||
}
|
||||
if (src.type === "comment") {
|
||||
parent.appendChild(doc.createComment(src.data ?? ""));
|
||||
return;
|
||||
}
|
||||
if (src.type === "tag" || src.type === "script" || src.type === "style") {
|
||||
const name = (src.name ?? "div").toLowerCase();
|
||||
let el: Element;
|
||||
try {
|
||||
el = doc.createElement(name);
|
||||
} catch {
|
||||
el = doc.createElement("x-invalid");
|
||||
}
|
||||
for (const [k, v] of Object.entries(src.attribs ?? {})) {
|
||||
try {
|
||||
el.setAttribute(k.toLowerCase(), v);
|
||||
} catch {
|
||||
// 非法属性名(如 @click、:class)在 lxml 下同样取不到,跳过
|
||||
}
|
||||
}
|
||||
parent.appendChild(el);
|
||||
for (const c of src.children ?? []) copyInto(doc, el, c, name);
|
||||
return;
|
||||
}
|
||||
// root / directive(doctype) / cdata:仅递归子节点
|
||||
for (const c of src.children ?? []) copyInto(doc, parent, c, parentName);
|
||||
}
|
||||
|
||||
/** 可 XPath 查询的节点包装(对应 lxml `_Element`)。 */
|
||||
export class XNode {
|
||||
constructor(readonly node: XmlNode) {}
|
||||
|
||||
/** lxml `el.xpath(expr)`:节点集 → (XNode|string)[];标量 → [标量] */
|
||||
xpath(expr: string): Array<XNode | string> {
|
||||
const r = xpath.select(expr, this.node);
|
||||
if (Array.isArray(r)) {
|
||||
return r.map((n) => {
|
||||
if (n.nodeType === 1 || n.nodeType === 9) return new XNode(n);
|
||||
if (n.nodeType === 2) return (n as Attr).value;
|
||||
return n.nodeValue ?? "";
|
||||
});
|
||||
}
|
||||
if (r == null) return [];
|
||||
return [String(r)];
|
||||
}
|
||||
|
||||
/** 只取字符串结果(文本/属性);元素结果按 `string(.)` 转文本 */
|
||||
strings(expr: string): string[] {
|
||||
return this.xpath(expr).map((v) => (typeof v === "string" ? v : v.text()));
|
||||
}
|
||||
|
||||
/** 只取元素结果 */
|
||||
nodes(expr: string): XNode[] {
|
||||
return this.xpath(expr).filter((v): v is XNode => v instanceof XNode);
|
||||
}
|
||||
|
||||
/** lxml `el.xpath("string(.)")` */
|
||||
text(): string {
|
||||
return String(xpath.select("string(.)", this.node, true) ?? "");
|
||||
}
|
||||
|
||||
/** 标量求值(string()/count()/boolean()) */
|
||||
evaluate(expr: string): string | number | boolean | null {
|
||||
const r = xpath.select(expr, this.node, true);
|
||||
if (r == null) return null;
|
||||
if (typeof r === "object") return (r as XmlNode).nodeValue ?? new XNode(r as XmlNode).text();
|
||||
return r;
|
||||
}
|
||||
|
||||
get tagName(): string {
|
||||
return (this.node as Element).tagName ?? "";
|
||||
}
|
||||
|
||||
attr(name: string): string | null {
|
||||
const el = this.node as Element;
|
||||
return typeof el.getAttribute === "function" && el.hasAttribute(name) ? el.getAttribute(name) : null;
|
||||
}
|
||||
|
||||
/** lxml `len(el)`:子元素个数 */
|
||||
get childElementCount(): number {
|
||||
let n = 0;
|
||||
const cs = this.node.childNodes;
|
||||
for (let i = 0; i < cs.length; i++) if (cs[i]!.nodeType === 1) n++;
|
||||
return n;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* lxml `etree.HTML(text)`:空文本返回 null(lxml 返回 None);否则返回文档节点。
|
||||
* 与 MP `DomUtils.has_child_elements(html)` 搭配使用 → `hasChildElements`。
|
||||
*/
|
||||
export function parseHtml(text: string | null | undefined): XNode | null {
|
||||
if (text == null || String(text).trim() === "") return null;
|
||||
const $ = load(String(text), { xml: { xmlMode: false, decodeEntities: true } });
|
||||
const doc = new DOMImplementation().createDocument(null, "", null) as unknown as Document;
|
||||
const root = doc.createElement("html");
|
||||
doc.appendChild(root);
|
||||
const dhRoot = $.root()[0] as unknown as DhNode;
|
||||
const kids = dhRoot.children ?? [];
|
||||
// 输入自带 <html> 时展开到根,避免 html/html 嵌套(lxml 会合并)
|
||||
const htmlEl = kids.find((k) => k.type === "tag" && k.name?.toLowerCase() === "html");
|
||||
if (htmlEl) {
|
||||
for (const [k, v] of Object.entries(htmlEl.attribs ?? {})) {
|
||||
try {
|
||||
root.setAttribute(k.toLowerCase(), v);
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
for (const k of kids) {
|
||||
if (k === htmlEl) for (const c of htmlEl.children ?? []) copyInto(doc, root, c, "html");
|
||||
else if (k.type !== "text" || (k.data ?? "").trim() !== "") copyInto(doc, root, k, "html");
|
||||
}
|
||||
} else {
|
||||
for (const k of kids) copyInto(doc, root, k, "html");
|
||||
}
|
||||
return new XNode(root);
|
||||
}
|
||||
|
||||
/** MP `DomUtils.has_child_elements`:存在且至少一个子元素 */
|
||||
export function hasChildElements(el: XNode | null | undefined): el is XNode {
|
||||
return el != null && el.childElementCount > 0;
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import { createMockFetch, FILELIST_USER_DETAIL, FILELIST_USER_INDEX, FILELIST_USER_SEEDING } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { FileListSiteUserInfo } from "./file-list.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
describe("FileListSiteUserInfo(MP file_list.py)", () => {
|
||||
it("详情页流量/分享率/做种/等级/加入日期/积分 + snatchlist 做种人数", async () => {
|
||||
const m = createMockFetch({
|
||||
"/": { body: FILELIST_USER_INDEX },
|
||||
"/userdetails.php": { body: FILELIST_USER_DETAIL },
|
||||
"/snatchlist.php": { body: FILELIST_USER_SEEDING },
|
||||
});
|
||||
const info = await runParser(FileListSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info).toMatchObject({
|
||||
schema: "FileList",
|
||||
userId: "555",
|
||||
username: "erin",
|
||||
upload: 3 * 1024 ** 4,
|
||||
download: 1024 ** 4,
|
||||
ratio: 3,
|
||||
seeding: 42,
|
||||
seedingSize: Math.round(2.5 * 1024 ** 4),
|
||||
userLevel: "Elite User",
|
||||
joinAt: "2017-07-08 09:10:11",
|
||||
bonus: 1000.5,
|
||||
seedingInfo: [[4, 1024 ** 3]],
|
||||
});
|
||||
expect(m.requests.find((r) => r.url.pathname === "/snatchlist.php")!.url.search).toBe(
|
||||
"?id=555&action=torrents&type=seeding",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,95 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/file_list.py @ e6820a8
|
||||
*
|
||||
* FileList 站点用户数据解析器(MP `FileListSiteUserInfo`)。
|
||||
* 函数对照:`_parse_site_page` → `_parseSitePage`、`_parse_user_base_info` → `_parseUserBaseInfo`、
|
||||
* `_parse_user_traffic_info`(空)、`_parse_user_detail_info` → `_parseUserDetailInfo`、
|
||||
* `_parse_user_torrent_seeding_info` → `_parseUserTorrentSeedingInfo`、消息两函数(不支持)。
|
||||
*/
|
||||
import { type NextPage, SiteParserBase } from "./base.ts";
|
||||
import { normalizeDatetime, parseFloatMp, parseIntMp, reSearch } from "./common.ts";
|
||||
import { hasChildElements, parseHtml } from "./dom.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
function at<T>(arr: T[], i: number): T {
|
||||
const v = arr[i];
|
||||
if (v === undefined) throw new Error("IndexError: list index out of range");
|
||||
return v;
|
||||
}
|
||||
|
||||
export class FileListSiteUserInfo extends SiteParserBase {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.FileList;
|
||||
|
||||
protected _parseSitePage(htmlText: string): void {
|
||||
const text = this._prepareHtmlText(htmlText);
|
||||
const m = reSearch(/userdetails.php\?id=(\d+)/, text);
|
||||
if (m?.[0].trim()) {
|
||||
this._userDetailPage = m[0].trim().replace(/^\/+/, "");
|
||||
this.userid = m[1]!;
|
||||
}
|
||||
// MP 此处无条件拼接(userid 为 None 时得到 "id=None")
|
||||
this._torrentSeedingPage = `snatchlist.php?id=${this.userid ?? "None"}&action=torrents&type=seeding`;
|
||||
}
|
||||
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
const html = parseHtml(this._prepareHtmlText(htmlText));
|
||||
if (!html) return;
|
||||
const ret = html.strings(`//a[contains(@href, "userdetails") and contains(@href, "${this.userid}")]//text()`);
|
||||
if (ret.length) this.username = String(ret[0]);
|
||||
}
|
||||
|
||||
/** 上传/下载/分享率 [做种数/魔力值](FileList 在详情页解析) */
|
||||
protected _parseUserTrafficInfo(_htmlText: string): void {}
|
||||
|
||||
protected _parseUserDetailInfo(htmlText: string): void {
|
||||
const html = parseHtml(this._prepareHtmlText(htmlText));
|
||||
if (!html) return;
|
||||
const up = html.strings('//table//tr/td[text()="Uploaded"]/following-sibling::td//text()');
|
||||
if (up.length) this.upload = this.numFilesize(up[0]);
|
||||
const down = html.strings('//table//tr/td[text()="Downloaded"]/following-sibling::td//text()');
|
||||
if (down.length) this.download = this.numFilesize(down[0]);
|
||||
const ratio = html.strings('//table//tr/td[text()="Share ratio"]/following-sibling::td//text()');
|
||||
const shareRatio = ratio.length ? parseFloatMp(ratio[0]) : 0;
|
||||
this.ratio = this.download === 0 ? 0 : shareRatio;
|
||||
|
||||
const seed = html.strings('//table//tr/td[text()="Seed bonus"]/following-sibling::td//text()');
|
||||
if (seed.length) {
|
||||
this.seeding = parseIntMp(at(seed, 1));
|
||||
this.seedingSize = this.numFilesize(at(seed, 3));
|
||||
}
|
||||
const level = html.strings('//table//tr/td[text()="Class"]/following-sibling::td//text()');
|
||||
if (level.length) this.userLevel = level[0]!.trim();
|
||||
const joinAt = html.strings('//table//tr/td[contains(text(), "Join")]/following-sibling::td//text()');
|
||||
if (joinAt.length) this.joinAt = normalizeDatetime(joinAt[0]!.split("(")[0]!.trim()) ?? null;
|
||||
const bonus = html.nodes('//a[contains(@href, "shop.php")]');
|
||||
if (bonus.length) this.bonus = parseFloatMp(bonus[0]!.text().trim());
|
||||
}
|
||||
|
||||
protected _parseUserTorrentSeedingInfo(htmlText: string, _multiPage = false): NextPage {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return null;
|
||||
const sizeCol = 6;
|
||||
const seedersCol = 7;
|
||||
const pageSeedingInfo: Array<[number, number]> = [];
|
||||
const sizes = html.nodes(`//table/tr[position()>1]/td[${sizeCol}]`);
|
||||
const seeders = html.nodes(`//table/tr[position()>1]/td[${seedersCol}]`);
|
||||
if (sizes.length && seeders.length) {
|
||||
for (let i = 0; i < sizes.length; i++) {
|
||||
const size = this.numFilesize(sizes[i]!.text().trim());
|
||||
pageSeedingInfo.push([parseIntMp(seeders[i]?.text().trim()), size]);
|
||||
}
|
||||
}
|
||||
this.seedingInfo.push(...pageSeedingInfo);
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseMessageUnreadLinks(_htmlText: string, _msgLinks: string[]): NextPage {
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseMessageContent(_htmlText: string): [string | null, string | null, string | null] {
|
||||
return [null, null, null];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
import {
|
||||
createMockFetch,
|
||||
GAZELLE_LOGIN_PAGE,
|
||||
GAZELLE_USER_DETAIL,
|
||||
GAZELLE_USER_INDEX,
|
||||
GAZELLE_USER_SEEDING,
|
||||
} from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { GazelleSiteUserInfo } from "./gazelle.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
const GB = 1024 ** 3;
|
||||
|
||||
describe("GazelleSiteUserInfo(MP gazelle.py)", () => {
|
||||
it("首页用户链接/流量/积分 + 详情等级/加入时间 + 做种表", async () => {
|
||||
const m = createMockFetch({
|
||||
"/": { body: GAZELLE_USER_INDEX },
|
||||
"/user.php": { body: GAZELLE_USER_DETAIL },
|
||||
"/index.php": { body: GAZELLE_USER_INDEX },
|
||||
"/torrents.php": { body: GAZELLE_USER_SEEDING },
|
||||
});
|
||||
const info = await runParser(GazelleSiteUserInfo, { fetch: m.fetch, cookie: "session=secret" });
|
||||
expect(info).toMatchObject({
|
||||
schema: "Gazelle",
|
||||
userId: "4242",
|
||||
username: "bob",
|
||||
userLevel: "Power User",
|
||||
joinAt: "2019-05-06 07:08:09",
|
||||
upload: Math.round(120.5 * GB),
|
||||
download: Math.round(40.25 * GB),
|
||||
ratio: Math.round((120.5 / 40.25) * 1000) / 1000,
|
||||
bonus: 1234.5,
|
||||
seeding: 2,
|
||||
seedingSize: 11 * GB,
|
||||
seedingInfo: [
|
||||
[7, 10 * GB],
|
||||
[2, GB],
|
||||
],
|
||||
});
|
||||
expect(m.requests.find((r) => r.url.pathname === "/torrents.php")!.url.search).toBe("?type=seeding&userid=4242");
|
||||
expect(JSON.stringify(info)).not.toContain("secret");
|
||||
});
|
||||
|
||||
it("登录页:errMsg 提示未登录,不再请求", async () => {
|
||||
const m = createMockFetch({ "/": { body: GAZELLE_LOGIN_PAGE } });
|
||||
const info = await runParser(GazelleSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info.errMsg).toContain("未检测到已登陆");
|
||||
expect(m.requests).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,180 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/gazelle.py @ e6820a8
|
||||
*
|
||||
* Gazelle 站点用户数据解析器(MP `GazelleSiteUserInfo`)。
|
||||
* 函数对照:`_parse_logged_in` → `_parseLoggedIn`、`_parse_user_base_info` → `_parseUserBaseInfo`、
|
||||
* `_parse_site_page` → `_parseSitePage`(空)、`_parse_user_detail_info` → `_parseUserDetailInfo`、
|
||||
* `_parse_user_torrent_seeding_info` → `_parseUserTorrentSeedingInfo`、`_parse_user_traffic_info`(空)、
|
||||
* `_parse_message_unread_links` / `_parse_message_content`(不支持)。XPath 与正则逐字保留。
|
||||
*/
|
||||
import { type NextPage, SiteParserBase } from "./base.ts";
|
||||
import { normalizeDatetime, parseFloatMp, parseIntMp, reSearch } from "./common.ts";
|
||||
import { hasChildElements, parseHtml } from "./dom.ts";
|
||||
import { round3 } from "./nexus-php.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
export class GazelleSiteUserInfo extends SiteParserBase {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.Gazelle;
|
||||
|
||||
/** 兼容不展示退出入口的站点首页 */
|
||||
protected override _parseLoggedIn(htmlText: unknown): boolean | Promise<boolean> {
|
||||
const html = parseHtml(String(htmlText ?? ""));
|
||||
if (hasChildElements(html) && html.nodes('//a[contains(@href, "user.php?id=")]').length) return true;
|
||||
return super._parseLoggedIn(htmlText);
|
||||
}
|
||||
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
const html = parseHtml(this._prepareHtmlText(htmlText));
|
||||
if (!html) return;
|
||||
let tmps = html.nodes('//a[contains(@href, "user.php?id=") or contains(@href, "user?id=")]');
|
||||
if (tmps.length) {
|
||||
const m = reSearch(/user(?:\.php)?\?id=(\d+)/, tmps[0]!.attr("href"));
|
||||
if (m?.[0].trim()) {
|
||||
this.userid = m[1]!;
|
||||
this._torrentSeedingPage = `torrents.php?type=seeding&userid=${this.userid}`;
|
||||
this._userDetailPage = `user.php?id=${this.userid}`;
|
||||
// lxml `el.text`:首个子节点为文本时取之
|
||||
const first = tmps[0]!.node.firstChild;
|
||||
this.username = first && first.nodeType === 3 ? (first.nodeValue ?? "").trim() : null;
|
||||
}
|
||||
}
|
||||
|
||||
let vals = html.strings('//*[@id="header-uploaded-value"]/@data-value');
|
||||
if (vals.length) this.upload = this.numFilesize(vals[0]);
|
||||
else {
|
||||
vals = html.strings('//li[@id="stats_seeding"]/span/text()');
|
||||
if (vals.length) this.upload = this.numFilesize(vals[0]);
|
||||
}
|
||||
vals = html.strings('//*[@id="header-downloaded-value"]/@data-value');
|
||||
if (vals.length) this.download = this.numFilesize(vals[0]);
|
||||
else {
|
||||
vals = html.strings('//li[@id="stats_leeching"]/span/text()');
|
||||
if (vals.length) this.download = this.numFilesize(vals[0]);
|
||||
}
|
||||
this.ratio = this.download <= 0.0 ? 0.0 : round3(this.upload / this.download);
|
||||
|
||||
vals = html.strings('//a[contains(@href, "bonus")]/@data-tooltip');
|
||||
if (vals.length) {
|
||||
const m = reSearch(/([\d,.]+)/, vals[0]);
|
||||
if (m?.[1]?.trim()) this.bonus = parseFloatMp(m[1]);
|
||||
} else {
|
||||
tmps = html.nodes('//a[contains(@href, "bonus")]');
|
||||
if (tmps.length) {
|
||||
const m = reSearch(/([\d,.]+)/, tmps[0]!.text());
|
||||
if (m?.[1]?.trim()) this.bonus = parseFloatMp(m[1]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
protected _parseSitePage(_htmlText: string): void {}
|
||||
|
||||
protected _parseUserDetailInfo(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
// 用户等级
|
||||
let levels = html.strings('//*[@id="class-value"]/@data-value');
|
||||
if (levels.length) this.userLevel = levels[0]!.trim();
|
||||
else {
|
||||
levels = html.strings('//li[contains(text(), "用户等级")]/text()');
|
||||
if (levels.length) {
|
||||
const part = levels[0]!.split(":")[1];
|
||||
if (part === undefined) throw new Error("IndexError: list index out of range");
|
||||
this.userLevel = part.trim();
|
||||
}
|
||||
}
|
||||
// 加入日期
|
||||
let joinAt = html.strings('//*[@id="join-date-value"]/@data-value');
|
||||
if (joinAt.length) this.joinAt = normalizeDatetime(joinAt[0]!.trim()) ?? null;
|
||||
else {
|
||||
joinAt = html.strings(
|
||||
'//div[contains(@class, "box_userinfo_stats")]//li[contains(text(), "加入时间")]/span/text()',
|
||||
);
|
||||
if (joinAt.length) this.joinAt = normalizeDatetime(joinAt[0]!.trim()) ?? null;
|
||||
}
|
||||
// 兼容以文本形式展示上传/下载的站点(如 JPopsuki)
|
||||
if (!this.upload) {
|
||||
const up = html.nodes('//li[starts-with(normalize-space(text()), "Uploaded:")]');
|
||||
if (up.length) {
|
||||
const m = reSearch(/([\d.,]+\s*[GMKT]?i?B)/i, up[0]!.text());
|
||||
if (m) this.upload = this.numFilesize(m[1]);
|
||||
}
|
||||
}
|
||||
if (!this.download) {
|
||||
const down = html.nodes('//li[starts-with(normalize-space(text()), "Downloaded:")]');
|
||||
if (down.length) {
|
||||
const m = reSearch(/([\d.,]+\s*[GMKT]?i?B)/i, down[0]!.text());
|
||||
if (m) this.download = this.numFilesize(m[1]);
|
||||
}
|
||||
}
|
||||
if (!this.ratio && this.upload && this.download) this.ratio = round3(this.upload / this.download);
|
||||
}
|
||||
|
||||
protected _parseUserTorrentSeedingInfo(htmlText: string, multiPage = false): NextPage {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return null;
|
||||
let sizeCol = 3;
|
||||
const headCells = html.nodes('//table[contains(@id, "torrent")]//tr[1]/td');
|
||||
if (headCells.length) sizeCol = headCells.length - 3;
|
||||
const seedersCol = sizeCol + 2;
|
||||
|
||||
let pageSeeding = 0;
|
||||
let pageSeedingSize = 0;
|
||||
const pageSeedingInfo: Array<[number, number]> = [];
|
||||
const semanticRows = html.nodes(
|
||||
'//table[contains(@id, "torrent")]//tr[contains(concat(" ", normalize-space(@class), " "), " TableTorrent-rowTitle ")]',
|
||||
);
|
||||
if (semanticRows.length) {
|
||||
// GPW 等现代 Gazelle 页面以语义 class 标记列,逐行按 class 配对读取
|
||||
for (const row of semanticRows) {
|
||||
const sizeCells = row.nodes(
|
||||
'./td[contains(concat(" ", normalize-space(@class), " "), " TableTorrent-cellStatSize ")]',
|
||||
);
|
||||
const seederCells = row.nodes(
|
||||
'./td[contains(concat(" ", normalize-space(@class), " "), " TableTorrent-cellStatSeeders ")]',
|
||||
);
|
||||
if (!sizeCells.length || !seederCells.length) continue;
|
||||
const size = this.numFilesize(sizeCells[0]!.text().trim());
|
||||
const seeders = parseIntMp(seederCells[0]!.text().trim());
|
||||
pageSeeding += 1;
|
||||
pageSeedingSize += size;
|
||||
pageSeedingInfo.push([seeders, size]);
|
||||
}
|
||||
} else {
|
||||
const sizes = html.nodes(`//table[contains(@id, "torrent")]//tr[position()>1]/td[${sizeCol}]`);
|
||||
const seeders = html.strings(`//table[contains(@id, "torrent")]//tr[position()>1]/td[${seedersCol}]/text()`);
|
||||
if (sizes.length && seeders.length) {
|
||||
pageSeeding = sizes.length;
|
||||
for (let i = 0; i < sizes.length; i++) {
|
||||
const size = this.numFilesize(sizes[i]!.text().trim());
|
||||
pageSeedingSize += size;
|
||||
pageSeedingInfo.push([parseIntMp(seeders[i]), size]);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (multiPage) {
|
||||
this.seeding += pageSeeding;
|
||||
this.seedingSize += pageSeedingSize;
|
||||
this.seedingInfo.push(...pageSeedingInfo);
|
||||
} else {
|
||||
if (!this.seeding) this.seeding = pageSeeding;
|
||||
if (!this.seedingSize) this.seedingSize = pageSeedingSize;
|
||||
if (!this.seedingInfo.length) this.seedingInfo = pageSeedingInfo;
|
||||
}
|
||||
const next = html.strings(
|
||||
'//a[contains(.//text(), "Next") or contains(.//text(), "下一页") or contains(@title, "下一页") or contains(@title, "Next")]/@href',
|
||||
);
|
||||
return next.length ? next[next.length - 1]!.trim() : null;
|
||||
}
|
||||
|
||||
protected _parseUserTrafficInfo(_htmlText: string): void {}
|
||||
|
||||
protected _parseMessageUnreadLinks(_htmlText: string, _msgLinks: string[]): NextPage {
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseMessageContent(_htmlText: string): [string | null, string | null, string | null] {
|
||||
return [null, null, null];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import { createMockFetch, HDDOLBY_USER_DATA, HDDOLBY_USER_ERROR, HDDOLBY_USER_PEERS } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { HDDolbySiteUserInfo } from "./hddolby.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
const API = "https://api.hddolby.com";
|
||||
|
||||
describe("HDDolbySiteUserInfo(MP hddolby.py)", () => {
|
||||
it("api/v1/user/data + api/v1/user/peers(GET,x-api-key)", async () => {
|
||||
const m = createMockFetch({
|
||||
[`GET ${API}/api/v1/user/data`]: { body: HDDOLBY_USER_DATA },
|
||||
[`GET ${API}/api/v1/user/peers`]: { body: HDDOLBY_USER_PEERS },
|
||||
});
|
||||
const info = await runParser(HDDolbySiteUserInfo, {
|
||||
fetch: m.fetch,
|
||||
url: "https://www.hddolby.com/",
|
||||
apikey: "dolby-key",
|
||||
});
|
||||
expect(info).toMatchObject({
|
||||
schema: "HDDolby",
|
||||
userId: "1",
|
||||
username: "ivy",
|
||||
userLevel: "Veteran User",
|
||||
joinAt: "2019-03-03 15:30:36",
|
||||
upload: 852071699418,
|
||||
download: 1885536536,
|
||||
ratio: Math.round((852071699418 / 1885536536) * 100) / 100,
|
||||
bonus: 99774.5,
|
||||
messageUnread: 2,
|
||||
seeding: 2,
|
||||
seedingSize: 1500,
|
||||
seedingInfo: [
|
||||
[4, 1000],
|
||||
[1, 500],
|
||||
],
|
||||
});
|
||||
expect(m.requests[0]!.headers["x-api-key"]).toBe("dolby-key");
|
||||
expect(JSON.stringify(info)).not.toContain("dolby-key");
|
||||
});
|
||||
|
||||
it("业务错误:不填用户数据;无 cookie 时 fetchMessages 也不读网页消息", async () => {
|
||||
const m = createMockFetch({
|
||||
[`GET ${API}/api/v1/user/data`]: { body: HDDOLBY_USER_ERROR },
|
||||
[`GET ${API}/api/v1/user/peers`]: { body: HDDOLBY_USER_ERROR },
|
||||
});
|
||||
const info = await runParser(HDDolbySiteUserInfo, {
|
||||
fetch: m.fetch,
|
||||
url: "https://www.hddolby.com/",
|
||||
apikey: "k",
|
||||
fetchMessages: true,
|
||||
});
|
||||
expect(info.userId).toBeNull();
|
||||
expect(m.requests.map((r) => r.url.pathname)).toEqual(["/api/v1/user/data", "/api/v1/user/peers"]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,133 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/hddolby.py @ e6820a8
|
||||
*
|
||||
* HDDolby 用户数据解析器(MP `HDDolbySiteUserInfo`,apikey 模式)。
|
||||
* 函数对照:`HDDolby_sysRoleList` → `HDDOLBY_SYS_ROLE_LIST`、`_parse_site_page` → `_parseSitePage`(api.<域>)、
|
||||
* `_parse_logged_in`(恒 True)、`_parse_user_base_info` → `_parseUserBaseInfo`(api/v1/user/data)、
|
||||
* `_parse_user_torrent_seeding_info` → `_parseUserTorrentSeedingInfo`(api/v1/user/peers)、
|
||||
* `_pase_unread_msgs` → `_paseUnreadMsgs`(API 只给未读数,正文借 NexusPHP 解析器用 Cookie 读网页)。
|
||||
* 注:`_user_basic_params = {}` 在 Python 中为假值,故 profile/peers 两个接口实际为 GET(与 MP 一致)。
|
||||
*/
|
||||
import { type NextPage, SiteParserBase } from "./base.ts";
|
||||
import { extractDomain, pyFloat, pyInt } from "./common.ts";
|
||||
import { NexusPhpSiteUserInfo } from "./nexus-php.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
type Json = Record<string, unknown>;
|
||||
|
||||
/** 用户级别字典(MP `HDDolby_sysRoleList`) */
|
||||
export const HDDOLBY_SYS_ROLE_LIST: Record<string, string> = {
|
||||
"0": "Peasant",
|
||||
"1": "User",
|
||||
"2": "Power User",
|
||||
"3": "Elite User",
|
||||
"4": "Crazy User",
|
||||
"5": "Insane User",
|
||||
"6": "Veteran User",
|
||||
"7": "Extreme User",
|
||||
"8": "Ultimate User",
|
||||
"9": "Nexus Master",
|
||||
"10": "VIP",
|
||||
"11": "Retiree",
|
||||
"12": "Helper",
|
||||
"13": "Seeder",
|
||||
"14": "Transferrer",
|
||||
"15": "Uploader",
|
||||
"16": "Torrent Manager",
|
||||
"17": "Forum Moderator",
|
||||
"18": "Coder",
|
||||
"19": "Moderator",
|
||||
"20": "Administrator",
|
||||
"21": "Sysop",
|
||||
"22": "Staff Leader",
|
||||
};
|
||||
|
||||
export class HDDolbySiteUserInfo extends SiteParserBase {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.HDDolby;
|
||||
override requestMode = "apikey" as const;
|
||||
|
||||
protected _parseSitePage(_htmlText: string): void {
|
||||
this._baseUrl = `https://api.${extractDomain(this._baseUrl)}`;
|
||||
this._userTrafficPage = null;
|
||||
this._userDetailPage = null;
|
||||
this._userBasicPage = "api/v1/user/data";
|
||||
this._userBasicParams = {};
|
||||
this._userBasicHeaders = { "Content-Type": "application/json", Accept: "application/json, text/plain, */*" };
|
||||
this._sysMailUnreadPage = null;
|
||||
this._userMailUnreadPage = null;
|
||||
this._mailUnreadParams = {};
|
||||
this._torrentSeedingPage = "api/v1/user/peers";
|
||||
this._torrentSeedingParams = {};
|
||||
this._torrentSeedingHeaders = { "Content-Type": "application/json", Accept: "application/json, text/plain, */*" };
|
||||
this._additionHeaders = { "x-api-key": this.apikey ?? "" };
|
||||
}
|
||||
|
||||
protected override _parseLoggedIn(_htmlText: unknown): boolean {
|
||||
return true;
|
||||
}
|
||||
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
if (!htmlText) return;
|
||||
const detail = JSON.parse(htmlText) as Json | null;
|
||||
if (detail?.status !== 0) return;
|
||||
const infos = detail.data as Json[] | null;
|
||||
if (!infos?.length) return;
|
||||
const u = infos[0]!;
|
||||
this.userid = u.id == null ? null : String(u.id);
|
||||
this.username = (u.username as string) ?? null;
|
||||
this.userLevel = HDDOLBY_SYS_ROLE_LIST[String(u.class || "1")] ?? null;
|
||||
this.joinAt = (u.added as string) ?? null;
|
||||
this.upload = pyInt(u.uploaded || "0");
|
||||
this.download = pyInt(u.downloaded || "0");
|
||||
this.ratio = this.download ? Math.round((this.upload / this.download) * 100) / 100 : 0;
|
||||
this.bonus = pyFloat(u.seedbonus || "0");
|
||||
this.messageUnread = pyInt(u.unread_messages || "0");
|
||||
}
|
||||
|
||||
protected _parseUserTrafficInfo(_htmlText: string): void {}
|
||||
|
||||
protected _parseUserDetailInfo(_htmlText: string): void {}
|
||||
|
||||
protected _parseUserTorrentSeedingInfo(htmlText: string, _multiPage = false): NextPage {
|
||||
if (!htmlText) return null;
|
||||
const seedingInfo = JSON.parse(htmlText) as Json | null;
|
||||
if (seedingInfo?.status !== 0) return null;
|
||||
const torrents = (seedingInfo.data ?? []) as Json[];
|
||||
let pageSeedingSize = 0;
|
||||
const pageSeedingInfo: Array<[number, number]> = [];
|
||||
for (const info of torrents) {
|
||||
const size = Number(info.size ?? 0);
|
||||
const seeder = Number(info.seeders || 1);
|
||||
pageSeedingSize += size;
|
||||
pageSeedingInfo.push([seeder, size]);
|
||||
}
|
||||
this.seeding += torrents.length;
|
||||
this.seedingSize += pageSeedingSize;
|
||||
this.seedingInfo.push(...pageSeedingInfo);
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseMessageUnreadLinks(_htmlText: string, _msgLinks: string[]): NextPage {
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseMessageContent(_htmlText: string): [string | null, string | null, string | null] {
|
||||
return [null, null, null];
|
||||
}
|
||||
|
||||
/** API 仅返回未读数量,正文需通过 NexusPHP 网页接口读取(需 Cookie) */
|
||||
protected override async _paseUnreadMsgs(): Promise<void> {
|
||||
if (!this._siteCookie) return;
|
||||
const nexus = new NexusPhpSiteUserInfo({ ...this._opts, cookie: this._siteCookie });
|
||||
nexus.messageUnread = this.messageUnread;
|
||||
nexus.messageReadForce = this.messageUnread > 0;
|
||||
await (nexus as unknown as { _paseUnreadMsgs(): Promise<void> })._paseUnreadMsgs();
|
||||
this.messageUnreadContents = [...nexus.messageUnreadContents];
|
||||
this.httpErrors.push(...nexus.httpErrors);
|
||||
if (this.messageUnreadContents.length && !this.messageUnread) {
|
||||
this.messageUnread = this.messageUnreadContents.length;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
export { SiteParserBase, type SiteParserClass, urljoin } from "./base.ts";
|
||||
export { BitptSiteUserInfo } from "./bitpt.ts";
|
||||
export {
|
||||
containsChinese,
|
||||
extractDomain,
|
||||
formatTimestamp,
|
||||
isLoggedIn,
|
||||
normalizeDatetime,
|
||||
parseFloatMp,
|
||||
parseIntMp,
|
||||
underChallenge,
|
||||
} from "./common.ts";
|
||||
export { DiscuzUserInfo } from "./discuz.ts";
|
||||
export { hasChildElements, parseHtml, XNode } from "./dom.ts";
|
||||
export { FileListSiteUserInfo } from "./file-list.ts";
|
||||
export { GazelleSiteUserInfo } from "./gazelle.ts";
|
||||
export { HDDolbySiteUserInfo } from "./hddolby.ts";
|
||||
export { IptSiteUserInfo } from "./ipt.ts";
|
||||
export { MTorrentSiteUserInfo } from "./mtorrent.ts";
|
||||
export { NexusAudiencesSiteUserInfo } from "./nexus-audiences.ts";
|
||||
export { NexusHhanclubSiteUserInfo } from "./nexus-hhanclub.ts";
|
||||
export { NexusPhpSiteUserInfo } from "./nexus-php.ts";
|
||||
export { NexusProjectSiteUserInfo } from "./nexus-project.ts";
|
||||
export { NexusRabbitSiteUserInfo } from "./nexus-rabbit.ts";
|
||||
export {
|
||||
getUserParserClass,
|
||||
refreshSiteUserData,
|
||||
type SiteUserDataRequest,
|
||||
USER_PARSER_CLASSES,
|
||||
} from "./registry.ts";
|
||||
export { RousiSiteUserInfo } from "./rousi.ts";
|
||||
export { SmallHorseSiteUserInfo } from "./small-horse.ts";
|
||||
export { SunnyPTSiteUserInfo } from "./sunnypt.ts";
|
||||
export { TNodeSiteUserInfo } from "./tnode.ts";
|
||||
export { TorrentLeechSiteUserInfo } from "./torrent-leech.ts";
|
||||
export {
|
||||
SiteSchema,
|
||||
type SiteSchemaValue,
|
||||
type SiteUnreadMessage,
|
||||
type SiteUserInfo,
|
||||
type SiteUserParserOptions,
|
||||
} from "./types.ts";
|
||||
export { Unit3dSiteUserInfo } from "./unit3d.ts";
|
||||
export { YemaSiteUserInfo } from "./yema.ts";
|
||||
export { ZhixingSiteUserInfo } from "./zhixing.ts";
|
||||
@@ -0,0 +1,31 @@
|
||||
import { createMockFetch, IPT_USER_DETAIL, IPT_USER_INDEX } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { IptSiteUserInfo } from "./ipt.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
describe("IptSiteUserInfo(MP ipt.py)", () => {
|
||||
it("统计块:分享率/上传/下载/做种/下载中/积分;详情页等级与加入日期", async () => {
|
||||
const m = createMockFetch({ "/": { body: IPT_USER_INDEX }, "/user.php": { body: IPT_USER_DETAIL } });
|
||||
const info = await runParser(IptSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info).toMatchObject({
|
||||
schema: "IPTorrents",
|
||||
userId: "777",
|
||||
username: "dave",
|
||||
ratio: 2.5,
|
||||
upload: Math.round(1.2 * 1024 ** 4),
|
||||
download: 480 * 1024 ** 3,
|
||||
seeding: 15,
|
||||
leeching: 3,
|
||||
bonus: 4321.5,
|
||||
userLevel: "Power User",
|
||||
joinAt: "2018-01-02 03:04:05",
|
||||
});
|
||||
expect(m.requests.find((r) => r.url.pathname === "/user.php")!.url.search).toBe("?u=777");
|
||||
});
|
||||
|
||||
it("统计块结构缺列时与 MP 一样抛 IndexError", async () => {
|
||||
const broken = IPT_USER_INDEX.replace("<span>480.00 GB</span>", "");
|
||||
const m = createMockFetch({ "/": { body: broken } });
|
||||
await expect(runParser(IptSiteUserInfo, { fetch: m.fetch })).rejects.toThrow(/IndexError/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,101 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/ipt.py @ e6820a8
|
||||
*
|
||||
* IPTorrents 站点用户数据解析器(MP `IptSiteUserInfo`)。
|
||||
* 函数对照:`_parse_user_base_info` → `_parseUserBaseInfo`、`_parse_site_page`(空)、
|
||||
* `_parse_user_detail_info` → `_parseUserDetailInfo`、`_parse_user_torrent_seeding_info` → `_parseUserTorrentSeedingInfo`、
|
||||
* 其余为空实现。XPath 逐字保留;统计块下标越界时与 MP 一样抛 IndexError。
|
||||
*/
|
||||
import { type NextPage, SiteParserBase } from "./base.ts";
|
||||
import { normalizeDatetime, parseFloatMp, parseIntMp, reSearch } from "./common.ts";
|
||||
import { hasChildElements, parseHtml } from "./dom.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
/** Python 列表下标(越界抛 IndexError,保持 MP 失败语义) */
|
||||
function at<T>(arr: T[], i: number): T {
|
||||
const v = i < 0 ? arr[arr.length + i] : arr[i];
|
||||
if (v === undefined) throw new Error("IndexError: list index out of range");
|
||||
return v;
|
||||
}
|
||||
|
||||
export class IptSiteUserInfo extends SiteParserBase {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.Ipt;
|
||||
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
const html = parseHtml(this._prepareHtmlText(htmlText));
|
||||
if (!html) return;
|
||||
const names = html.strings('//a[contains(@href, "/u/")]//text()');
|
||||
const ids = html.strings('//a[contains(@href, "/u/")]/@href');
|
||||
if (names.length) this.username = String(at(names, -1));
|
||||
if (ids.length) {
|
||||
const m = reSearch(/\/u\/(\d+)/, ids[0]);
|
||||
if (m?.[0].trim()) {
|
||||
this.userid = m[1]!;
|
||||
this._userDetailPage = `user.php?u=${this.userid}`;
|
||||
this._torrentSeedingPage = `peers?u=${this.userid}`;
|
||||
}
|
||||
}
|
||||
const stats = html.nodes('//div[@class = "stats"]/div/div');
|
||||
if (stats.length) {
|
||||
const s = stats[0]!;
|
||||
const spans = s.strings("span/text()");
|
||||
const links = s.nodes("a");
|
||||
this.upload = this.numFilesize(String(at(spans, 1)).trim());
|
||||
this.download = this.numFilesize(String(at(spans, 2)).trim());
|
||||
this.seeding = parseIntMp(at(at(links, 2).strings("text()"), 0));
|
||||
this.leeching = parseIntMp(at(at(links, 2).strings("text()"), 1));
|
||||
this.ratio = parseFloatMp(String(at(spans, 0)).trim().replaceAll("-", "0"));
|
||||
this.bonus = parseFloatMp(at(at(links, 3).strings("text()"), 0));
|
||||
}
|
||||
}
|
||||
|
||||
protected _parseSitePage(_htmlText: string): void {}
|
||||
|
||||
protected _parseUserDetailInfo(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
const levels = html.strings('//tr/th[text()="Class"]/following-sibling::td[1]/text()');
|
||||
if (levels.length) this.userLevel = levels[0]!.trim();
|
||||
const joinAt = html.strings('//tr/th[text()="Join date"]/following-sibling::td[1]/text()');
|
||||
if (joinAt.length) this.joinAt = normalizeDatetime(joinAt[0]!.split(" (")[0]) ?? null;
|
||||
}
|
||||
|
||||
protected _parseUserTorrentSeedingInfo(htmlText: string, _multiPage = false): NextPage {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return null;
|
||||
let seedingEndPos = 3;
|
||||
if (html.nodes('//tr/td[text() = "Leechers"]').length) {
|
||||
seedingEndPos = html.nodes('//tr/td[text() = "Leechers"]/../preceding-sibling::tr').length + 1;
|
||||
seedingEndPos = seedingEndPos - 3;
|
||||
}
|
||||
let pageSeeding = 0;
|
||||
let pageSeedingSize = 0;
|
||||
const torrents = html.strings('//tr/td[text() = "Seeders"]/../following-sibling::tr/td[position()=6]/text()');
|
||||
if (torrents.length) {
|
||||
pageSeeding = seedingEndPos;
|
||||
// Python 切片 [:n](n 为负时从尾部截)
|
||||
for (let perSize of torrents.slice(0, seedingEndPos)) {
|
||||
if (perSize.includes("(") && perSize.includes(")")) {
|
||||
perSize = perSize.split("(").at(-1)!;
|
||||
perSize = perSize.split(")")[0]!;
|
||||
}
|
||||
pageSeedingSize += this.numFilesize(perSize);
|
||||
}
|
||||
}
|
||||
this.seeding = pageSeeding;
|
||||
this.seedingSize = pageSeedingSize;
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseUserTrafficInfo(_htmlText: string): void {}
|
||||
|
||||
protected _parseMessageUnreadLinks(_htmlText: string, _msgLinks: string[]): NextPage {
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseMessageContent(_htmlText: string): [string | null, string | null, string | null] {
|
||||
return [null, null, null];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
import {
|
||||
createMockFetch,
|
||||
MTORRENT_USER_KEY_INVALID,
|
||||
MTORRENT_USER_MESSAGES,
|
||||
MTORRENT_USER_PEER_STATUS,
|
||||
MTORRENT_USER_PROFILE,
|
||||
MTORRENT_USER_SEEDING_PAGE1,
|
||||
MTORRENT_USER_SEEDING_PAGE2,
|
||||
} from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { MTorrentSiteUserInfo } from "./mtorrent.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
const API = "https://api.m-team.cc";
|
||||
|
||||
function routes() {
|
||||
return createMockFetch({
|
||||
[`POST ${API}/api/member/profile`]: { body: MTORRENT_USER_PROFILE },
|
||||
[`POST ${API}/api/member/getUserTorrentList`]: (req) => ({
|
||||
body: JSON.parse(req.body ?? "{}").pageNumber === 1 ? MTORRENT_USER_SEEDING_PAGE1 : MTORRENT_USER_SEEDING_PAGE2,
|
||||
}),
|
||||
[`POST ${API}/api/tracker/myPeerStatus`]: { body: MTORRENT_USER_PEER_STATUS },
|
||||
[`POST ${API}/api/msg/search`]: { body: MTORRENT_USER_MESSAGES },
|
||||
[`POST ${API}/api/msg/markRead`]: { body: { code: "0" } },
|
||||
});
|
||||
}
|
||||
|
||||
describe("MTorrentSiteUserInfo(MP mtorrent.py)", () => {
|
||||
it("profile + 做种分页(按 myPeerStatus 总数翻页);默认不读消息、不标记已读", async () => {
|
||||
const m = routes();
|
||||
const info = await runParser(MTorrentSiteUserInfo, {
|
||||
fetch: m.fetch,
|
||||
url: "https://kp.m-team.cc/",
|
||||
apikey: "KEY-secret",
|
||||
});
|
||||
expect(info).toMatchObject({
|
||||
schema: "MTorrent",
|
||||
domain: "m-team.cc",
|
||||
userId: "31337",
|
||||
username: "henry",
|
||||
userLevel: "Veteran User",
|
||||
joinAt: "2020-10-11 12:13:14",
|
||||
upload: 3000,
|
||||
download: 1000,
|
||||
ratio: 3,
|
||||
bonus: 4567.8,
|
||||
seeding: 3,
|
||||
seedingSize: 600,
|
||||
seedingInfo: [
|
||||
[5, 100],
|
||||
[2, 200],
|
||||
[1, 300],
|
||||
],
|
||||
messageUnreadContents: [],
|
||||
});
|
||||
const profile = m.requests[0]!;
|
||||
expect(profile.url.href).toBe(`${API}/api/member/profile`);
|
||||
expect(profile.headers["x-api-key"]).toBe("KEY-secret");
|
||||
expect(m.requests.filter((r) => r.url.pathname === "/api/member/getUserTorrentList")).toHaveLength(2);
|
||||
expect(m.requests.some((r) => r.url.pathname === "/api/msg/markRead")).toBe(false);
|
||||
expect(JSON.stringify(info)).not.toContain("KEY-secret");
|
||||
});
|
||||
|
||||
it("fetchMessages=true:读出未读消息并逐条 markRead(写操作)", async () => {
|
||||
const m = routes();
|
||||
const info = await runParser(MTorrentSiteUserInfo, {
|
||||
fetch: m.fetch,
|
||||
url: "https://kp.m-team.cc/",
|
||||
apikey: "k",
|
||||
fetchMessages: true,
|
||||
});
|
||||
expect(info.messageUnreadContents).toEqual([{ head: "欢迎", date: "2026-09-01 10:00:00", content: "hello" }]);
|
||||
// 与 MP 一致:消息直接写入 contents,不经 msg_links,故 messageUnread 不回填
|
||||
expect(info.messageUnread).toBe(0);
|
||||
const mark = m.requests.filter((r) => r.url.pathname === "/api/msg/markRead");
|
||||
expect(mark).toHaveLength(1);
|
||||
expect(mark[0]!.body).toBe("msgId=m1");
|
||||
});
|
||||
|
||||
it("key 无效(业务 code≠0):不填用户数据;未配置 key 时不发请求", async () => {
|
||||
const m = createMockFetch({ [`POST ${API}/api/member/profile`]: { body: MTORRENT_USER_KEY_INVALID } });
|
||||
const info = await runParser(MTorrentSiteUserInfo, { fetch: m.fetch, url: "https://kp.m-team.cc/", apikey: "bad" });
|
||||
expect(info.userId).toBeNull();
|
||||
expect(info.upload).toBe(0);
|
||||
|
||||
const m2 = createMockFetch({});
|
||||
const info2 = await runParser(MTorrentSiteUserInfo, { fetch: m2.fetch, url: "https://kp.m-team.cc/" });
|
||||
expect(m2.requests).toHaveLength(0);
|
||||
expect(info2.errMsg).toBeTruthy();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,163 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/mtorrent.py @ e6820a8
|
||||
*
|
||||
* M-Team(mTorrent API)用户数据解析器(MP `MTorrentSiteUserInfo`,apikey 模式)。
|
||||
* 函数对照:`MTeam_sysRoleList` → `MTEAM_SYS_ROLE_LIST`、`_parse_site_page` → `_parseSitePage`(改用 api.<域> 接口)、
|
||||
* `_parse_logged_in` → `_parseLoggedIn`(恒 True)、`_parse_user_base_info` → `_parseUserBaseInfo`(api/member/profile)、
|
||||
* `_parse_user_torrent_seeding_info` → `_parseUserTorrentSeedingInfo`(api/member/getUserTorrentList 分页,
|
||||
* 以 api/tracker/myPeerStatus 的 seeder 总数判断是否还有下一页)、
|
||||
* `_parse_message_unread_links` → `_parseMessageUnreadLinks`(api/msg/search,直接读出正文)。
|
||||
*
|
||||
* 注意(写操作):与 MP 一样,读取未读消息后会调用 `api/msg/markRead` 把消息标记为已读。
|
||||
* 该流程只在 `fetchMessages=true`(MP `SITE_MESSAGE`)时执行,默认不触发。
|
||||
* API Key 只放在 `x-api-key` 请求头,不进入结果。
|
||||
*/
|
||||
import { type NextPage, SiteParserBase, urljoin } from "./base.ts";
|
||||
import { extractDomain, pyInt } from "./common.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
type Json = Record<string, unknown>;
|
||||
|
||||
/** 用户级别字典(MP `MTeam_sysRoleList`) */
|
||||
export const MTEAM_SYS_ROLE_LIST: Record<string, string> = {
|
||||
"1": "User",
|
||||
"2": "Power User",
|
||||
"3": "Elite User",
|
||||
"4": "Crazy User",
|
||||
"5": "Insane User",
|
||||
"6": "Veteran User",
|
||||
"7": "Extreme User",
|
||||
"8": "Ultimate User",
|
||||
"9": "Nexus Master",
|
||||
"10": "VIP",
|
||||
"11": "Retiree",
|
||||
"12": "Uploader",
|
||||
"13": "Moderator",
|
||||
"14": "Administrator",
|
||||
"15": "Sysop",
|
||||
"16": "Staff",
|
||||
"17": "Offer memberStaff",
|
||||
"18": "Bet memberStaff",
|
||||
};
|
||||
|
||||
export class MTorrentSiteUserInfo extends SiteParserBase {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.MTorrent;
|
||||
override requestMode = "apikey" as const;
|
||||
|
||||
protected _parseSitePage(_htmlText: string): void {
|
||||
// 更换 api 地址
|
||||
this._baseUrl = `https://api.${extractDomain(this._baseUrl)}`;
|
||||
this._userTrafficPage = null;
|
||||
this._userDetailPage = null;
|
||||
this._userBasicPage = "api/member/profile";
|
||||
this._userBasicParams = { uid: this.userid };
|
||||
this._sysMailUnreadPage = null;
|
||||
this._userMailUnreadPage = "api/msg/search";
|
||||
this._mailUnreadParams = { keyword: "", box: "-2", type: "pageNumber", pageSize: 100 };
|
||||
this._torrentSeedingPage = "api/member/getUserTorrentList";
|
||||
this._torrentSeedingHeaders = {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "application/json, text/plain, */*",
|
||||
};
|
||||
this._additionHeaders = { "x-api-key": this.apikey ?? "" };
|
||||
}
|
||||
|
||||
/** 暂时跳过检测(与 MP 一致) */
|
||||
protected override _parseLoggedIn(_htmlText: unknown): boolean {
|
||||
return true;
|
||||
}
|
||||
|
||||
/** 基础信息:合并了流量与详情解析 */
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
if (!htmlText) return;
|
||||
const detail = JSON.parse(htmlText) as Json | null;
|
||||
if (detail?.code !== "0") return;
|
||||
const u = (detail.data ?? {}) as Json;
|
||||
const memberCount = (u.memberCount ?? {}) as Json;
|
||||
this.userid = u.id == null ? null : String(u.id);
|
||||
this.username = (u.username as string) ?? null;
|
||||
this.userLevel = MTEAM_SYS_ROLE_LIST[String(u.role || "1")] ?? null;
|
||||
this.joinAt = ((u.memberStatus ?? {}) as Json).createdDate as string | null;
|
||||
this.upload = pyInt(memberCount.uploaded || "0");
|
||||
this.download = pyInt(memberCount.downloaded || "0");
|
||||
this.ratio = (memberCount.shareRate as number) || 0;
|
||||
this.bonus = (memberCount.bonus as number) || 0;
|
||||
this.messageReadForce = true;
|
||||
this._torrentSeedingParams = { pageNumber: 1, pageSize: 200, type: "SEEDING", userid: this.userid };
|
||||
}
|
||||
|
||||
protected _parseUserTrafficInfo(_htmlText: string): void {}
|
||||
|
||||
protected _parseUserDetailInfo(_htmlText: string): void {}
|
||||
|
||||
protected async _parseUserTorrentSeedingInfo(htmlText: string, _multiPage = false): Promise<NextPage> {
|
||||
if (!htmlText) return null;
|
||||
const seedingInfo = JSON.parse(htmlText) as Json | null;
|
||||
if (seedingInfo?.code !== "0") return null;
|
||||
const seedingData = seedingInfo.data || {};
|
||||
if (typeof seedingData !== "object" || Array.isArray(seedingData)) return null;
|
||||
const torrents = (seedingData as Json).data || [];
|
||||
if (!Array.isArray(torrents)) return null;
|
||||
let pageSeeding = 0;
|
||||
let pageSeedingSize = 0;
|
||||
const pageSeedingInfo: Array<[number, number]> = [];
|
||||
for (const info of torrents as unknown[]) {
|
||||
if (!info || typeof info !== "object" || Array.isArray(info)) continue;
|
||||
const torrent = (info as Json).torrent;
|
||||
if (!torrent || typeof torrent !== "object" || Array.isArray(torrent)) continue;
|
||||
const size = pyInt((torrent as Json).size || "0");
|
||||
const seeders = pyInt((torrent as Json).source || "0");
|
||||
pageSeeding += 1;
|
||||
pageSeedingSize += size;
|
||||
pageSeedingInfo.push([seeders, size]);
|
||||
}
|
||||
this.seeding += pageSeeding;
|
||||
this.seedingSize += pageSeedingSize;
|
||||
this.seedingInfo.push(...pageSeedingInfo);
|
||||
|
||||
// 查询总做种数
|
||||
let seederCount = 0;
|
||||
try {
|
||||
const result = await this._getPageContent(urljoin(this._baseUrl, "api/tracker/myPeerStatus"), {
|
||||
uid: this.userid,
|
||||
});
|
||||
if (result) {
|
||||
const seederInfo = JSON.parse(result) as Json;
|
||||
seederCount = pyInt(((seederInfo.data ?? {}) as Json).seeder || 0);
|
||||
}
|
||||
} catch {
|
||||
// MP:记录日志后按 0 处理
|
||||
}
|
||||
if (!seederCount) return null;
|
||||
if (this.seeding >= seederCount) return null;
|
||||
// 还有下一页
|
||||
(this._torrentSeedingParams as Json).pageNumber = Number((this._torrentSeedingParams as Json).pageNumber) + 1;
|
||||
return "";
|
||||
}
|
||||
|
||||
/** 解析未读消息链接,这里直接读出详情(并标记已读,见文件头) */
|
||||
protected async _parseMessageUnreadLinks(htmlText: string, _msgLinks: string[]): Promise<NextPage> {
|
||||
if (!htmlText) return null;
|
||||
const messagesInfo = JSON.parse(htmlText) as Json | null;
|
||||
if (messagesInfo?.code !== "0") return null;
|
||||
const messages = (((messagesInfo.data ?? {}) as Json).data ?? []) as Json[];
|
||||
for (const message of messages) {
|
||||
if (!message.unread) continue;
|
||||
const head = message.title as string | null;
|
||||
const date = message.createdDate as string | null;
|
||||
const content = message.context as string | null;
|
||||
if (head && date && content) {
|
||||
this.messageUnreadContents.push([head, date, content]);
|
||||
// 设置已读
|
||||
await this._getPageContent(urljoin(this._baseUrl, "api/msg/markRead"), { msgId: message.id });
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
protected _parseMessageContent(_htmlText: string): [string | null, string | null, string | null] {
|
||||
return [null, null, null];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
import {
|
||||
AUDIENCES_USER_DETAILS,
|
||||
AUDIENCES_USER_INDEX,
|
||||
AUDIENCES_USER_MAILBOX,
|
||||
AUDIENCES_USER_MESSAGE_901,
|
||||
AUDIENCES_USER_SEEDING,
|
||||
createMockFetch,
|
||||
NEXUS_PHP_LOGIN_PAGE,
|
||||
} from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { NexusAudiencesSiteUserInfo } from "./nexus-audiences.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
const GiB = 1024 ** 3;
|
||||
const TiB = 1024 ** 4;
|
||||
|
||||
function routes() {
|
||||
return createMockFetch({
|
||||
"/": { body: AUDIENCES_USER_INDEX },
|
||||
"/userdetails.php": { body: AUDIENCES_USER_DETAILS },
|
||||
"/getusertorrentlistajax.php": { body: AUDIENCES_USER_SEEDING },
|
||||
"/messages.php": { body: AUDIENCES_USER_MAILBOX },
|
||||
"/messageviewer.php?id=901": { body: AUDIENCES_USER_MESSAGE_901 },
|
||||
});
|
||||
}
|
||||
|
||||
describe("NexusAudiencesSiteUserInfo(MP nexus_audiences.py)", () => {
|
||||
it("userbar data-uploader-* + compact metric + Total 行汇总做种", async () => {
|
||||
const m = routes();
|
||||
const info = await runParser(NexusAudiencesSiteUserInfo, { fetch: m.fetch, cookie: "c_secure=abc" });
|
||||
expect(info).toMatchObject({
|
||||
schema: "NexusAudiences",
|
||||
userId: "233",
|
||||
username: "bob",
|
||||
userLevel: "Elite User",
|
||||
joinAt: "2020-05-06 07:08:09",
|
||||
upload: 2 * TiB,
|
||||
download: 100 * GiB,
|
||||
ratio: 20.48,
|
||||
bonus: 8888.5,
|
||||
leeching: 1,
|
||||
seeding: 31,
|
||||
seedingSize: Math.round(3.5 * TiB),
|
||||
messageUnread: 2,
|
||||
messageUnreadContents: [],
|
||||
});
|
||||
const seedReq = m.requests.find((r) => r.url.pathname === "/getusertorrentlistajax.php")!;
|
||||
expect(seedReq.headers.referer).toBe("https://pt.example.com/userdetails.php?id=233");
|
||||
expect(m.requests.some((r) => r.url.pathname === "/messages.php")).toBe(false);
|
||||
});
|
||||
|
||||
it("读取未读私信:首页不带 page、详情解析、详情为空用列表预览兜底、达到未读数停止翻页", async () => {
|
||||
const m = routes();
|
||||
const info = await runParser(NexusAudiencesSiteUserInfo, { fetch: m.fetch, fetchMessages: true });
|
||||
const mailReqs = m.requests.filter((r) => r.url.pathname === "/messages.php");
|
||||
expect(mailReqs.map((r) => r.url.search)).toEqual(["?action=viewmailbox&box=1&unread=yes"]);
|
||||
expect(info.messageUnreadContents).toEqual([
|
||||
{ head: "种子被删除", date: "2026-09-01 10:00:00", content: "你的种子 X 因重复已被删除。" },
|
||||
{ head: "欢迎", date: "2026-09-02 11:00:00", content: "欢迎加入观众" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("未登录:只请求首页", async () => {
|
||||
const m = createMockFetch({ "/": { body: NEXUS_PHP_LOGIN_PAGE } });
|
||||
const info = await runParser(NexusAudiencesSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info.userId).toBeNull();
|
||||
expect(m.requests).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("收件箱入口解析:总数/未读、角标、仅未读标记", () => {
|
||||
expect(NexusAudiencesSiteUserInfo.extractInboxUnreadPair("收件箱 1,749 / 172")).toBe(172);
|
||||
expect(NexusAudiencesSiteUserInfo.extractInboxUnreadBadge(" 5 ")).toBe(5);
|
||||
expect(NexusAudiencesSiteUserInfo.extractInboxUnreadBadge("新")).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,401 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/nexus_audiences.py @ e6820a8
|
||||
*
|
||||
* 观众(Audiences)站点用户信息解析器,继承 NexusPHP 通用解析。
|
||||
*
|
||||
* 函数对照(`NexusAudiencesSiteUserInfo`):
|
||||
* `__init__` → constructor、`_parse_message_unread` → `_parseMessageUnread`、
|
||||
* `_parse_message_unread_links` → `_parseMessageUnreadLinks`、`_parse_message_content` → `_parseMessageContent`、
|
||||
* `_pase_unread_msgs` → `_paseUnreadMsgs`、`__parse_unread_message_list_page` → `parseUnreadMessageListPage`、
|
||||
* `__reset_unread_message_parse_state` → `resetUnreadMessageParseState`、`__filter_new_message_links` → `filterNewMessageLinks`、
|
||||
* `__parse_table_unread_message_links` → `parseTableUnreadMessageLinks`、`__parse_pm_item_unread_message_links` →
|
||||
* `parsePmItemUnreadMessageLinks`、`__cache_pm_item_preview` → `cachePmItemPreview`、`__extract_pm_item_text` /
|
||||
* `__extract_first_text` / `__extract_pm_view_meta` / `__normalize_text` → 同名、`__fill_empty_message_content_from_list` →
|
||||
* `fillEmptyMessageContentFromList`、`__should_fetch_next_unread_page` / `__has_reached_expected_unread_count` /
|
||||
* `__is_empty_message_content` / `__build_unread_mailbox_page` / `__build_next_unread_mailbox_page` → 同名、
|
||||
* `_parse_user_traffic_info` / `_parse_user_detail_info` → 同名(追加 `__parse_userbar_info`)、
|
||||
* `__parse_user_identity` / `__parse_uploader_stats` / `__parse_compact_metric` / `__set_metric_value` /
|
||||
* `__parse_inbox_unread` / `__extract_inbox_unread_pair` / `__extract_inbox_unread_badge` /
|
||||
* `__has_inbox_unread_marker` → 同名、`_parse_seeding_pages` → `_parseSeedingPages`(Total 行汇总)。
|
||||
* XPath 与正则逐字保留。
|
||||
*/
|
||||
import { urljoin } from "./base.ts";
|
||||
import { parseFloatMp, parseIntMp, reSearch } from "./common.ts";
|
||||
import { hasChildElements, parseHtml, type XNode } from "./dom.ts";
|
||||
import { NexusPhpSiteUserInfo } from "./nexus-php.ts";
|
||||
import { SiteSchema, type SiteSchemaValue, type SiteUserParserOptions } from "./types.ts";
|
||||
|
||||
type Msg = [string | null, string | null, string | null];
|
||||
|
||||
const UNKNOWN_UNREAD_COUNT = 99999;
|
||||
const MAX_MAIL_PAGES = 50;
|
||||
|
||||
function normalizeText(text: string | null | undefined): string | null {
|
||||
if (!text) return null;
|
||||
const t = text.replaceAll(" ", " ").replace(/\s+/g, " ").trim();
|
||||
return t || null;
|
||||
}
|
||||
|
||||
export class NexusAudiencesSiteUserInfo extends NexusPhpSiteUserInfo {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.NexusAudiences;
|
||||
|
||||
private nextMailPage = 1;
|
||||
private readonly seenUnreadMessageLinks = new Set<string>();
|
||||
private readonly messageListPreviews = new Map<string, Msg>();
|
||||
|
||||
constructor(opts: SiteUserParserOptions) {
|
||||
super(opts);
|
||||
this._userMailUnreadPage = NexusAudiencesSiteUserInfo.buildUnreadMailboxPage(1);
|
||||
this._sysMailUnreadPage = null;
|
||||
}
|
||||
|
||||
/** 解析新版顶部用户栏中的未读消息数 */
|
||||
protected override _parseMessageUnread(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) {
|
||||
super._parseMessageUnread(htmlText);
|
||||
return;
|
||||
}
|
||||
const messageTools = html.nodes(
|
||||
'//a[contains(@class, "site-userbar__compact-tool") and contains(@href, "messages.php") ' +
|
||||
'and (contains(@class, "site-userbar__compact-tool--has-unread") ' +
|
||||
'or .//*[contains(@class, "site-userbar__compact-tool-badge--unread")])]' +
|
||||
'|//a[contains(@href, "messages.php") ' +
|
||||
'and (contains(@title, "收件箱") or contains(@aria-label, "收件箱"))]',
|
||||
);
|
||||
for (const link of messageTools) {
|
||||
const unread = NexusAudiencesSiteUserInfo.parseInboxUnread(link);
|
||||
if (unread !== null) {
|
||||
this.messageUnread = unread;
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (messageTools.length) return;
|
||||
super._parseMessageUnread(htmlText);
|
||||
}
|
||||
|
||||
protected override _parseMessageUnreadLinks(htmlText: string, msgLinks: string[]): string | null {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return null;
|
||||
const messageLinks = NexusAudiencesSiteUserInfo.parseTableUnreadMessageLinks(html);
|
||||
messageLinks.push(...this.parsePmItemUnreadMessageLinks(html));
|
||||
const newLinks = this.filterNewMessageLinks(messageLinks);
|
||||
msgLinks.push(...newLinks);
|
||||
return this.buildNextUnreadMailboxPage(this.shouldFetchNextUnreadPage(newLinks));
|
||||
}
|
||||
|
||||
protected override _parseMessageContent(htmlText: string): Msg {
|
||||
const html = parseHtml(htmlText);
|
||||
if (hasChildElements(html)) {
|
||||
const head = NexusAudiencesSiteUserInfo.extractFirstText(
|
||||
html,
|
||||
'//*[contains(concat(" ", normalize-space(@class), " "), " pm-hero__title ")]',
|
||||
);
|
||||
const date = NexusAudiencesSiteUserInfo.extractPmViewMeta(html, "日期");
|
||||
const content = NexusAudiencesSiteUserInfo.extractFirstText(
|
||||
html,
|
||||
'//*[contains(concat(" ", normalize-space(@class), " "), " pm-view__body ")]',
|
||||
);
|
||||
if (!NexusAudiencesSiteUserInfo.isEmptyMessageContent(head, date, content)) return [head, date, content];
|
||||
}
|
||||
return super._parseMessageContent(htmlText);
|
||||
}
|
||||
|
||||
/** 避免异常分页重复通知和空详情通知 */
|
||||
protected override async _paseUnreadMsgs(): Promise<void> {
|
||||
this.resetUnreadMessageParseState();
|
||||
const unreadMsgLinks: string[] = [];
|
||||
if (this.messageUnread > 0 || this.messageReadForce) {
|
||||
let nextPage = await this.parseUnreadMessageListPage(this._userMailUnreadPage, unreadMsgLinks);
|
||||
let guard = 0;
|
||||
while (nextPage && guard++ < MAX_MAIL_PAGES) {
|
||||
nextPage = await this.parseUnreadMessageListPage(nextPage, unreadMsgLinks);
|
||||
}
|
||||
}
|
||||
if (this.messageUnread === UNKNOWN_UNREAD_COUNT) this.messageUnread = unreadMsgLinks.length;
|
||||
else if (unreadMsgLinks.length && !this.messageUnread) this.messageUnread = unreadMsgLinks.length;
|
||||
for (const msgLink of unreadMsgLinks) {
|
||||
let [head, date, content] = this._parseMessageContent(
|
||||
await this._getPageContent(urljoin(this._baseUrl, msgLink), this._mailContentParams, this._mailContentHeaders),
|
||||
);
|
||||
[head, date, content] = this.fillEmptyMessageContentFromList(msgLink, head, date, content);
|
||||
if (NexusAudiencesSiteUserInfo.isEmptyMessageContent(head, date, content)) continue;
|
||||
this.messageUnreadContents.push([head, date, content]);
|
||||
}
|
||||
}
|
||||
|
||||
private async parseUnreadMessageListPage(link: string | null, unreadMsgLinks: string[]): Promise<string | null> {
|
||||
if (!link) return null;
|
||||
return this._parseMessageUnreadLinks(
|
||||
await this._getPageContent(urljoin(this._baseUrl, link), this._mailUnreadParams, this._mailUnreadHeaders),
|
||||
unreadMsgLinks,
|
||||
);
|
||||
}
|
||||
|
||||
private resetUnreadMessageParseState(): void {
|
||||
this.nextMailPage = 1;
|
||||
this.seenUnreadMessageLinks.clear();
|
||||
this.messageListPreviews.clear();
|
||||
}
|
||||
|
||||
private filterNewMessageLinks(messageLinks: string[]): string[] {
|
||||
const out: string[] = [];
|
||||
for (const link of messageLinks) {
|
||||
const key = urljoin(this._baseUrl, link);
|
||||
if (this.seenUnreadMessageLinks.has(key)) continue;
|
||||
this.seenUnreadMessageLinks.add(key);
|
||||
out.push(link);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
static parseTableUnreadMessageLinks(html: XNode): string[] {
|
||||
return html.strings(
|
||||
'//tr[.//img[contains(concat(" ", normalize-space(@class), " "), " unreadpm ") ' +
|
||||
'or @alt="Unread" or @title="未读"]]/td/a[contains(@href, "viewmessage")]/@href',
|
||||
);
|
||||
}
|
||||
|
||||
private parsePmItemUnreadMessageLinks(html: XNode): string[] {
|
||||
const links: string[] = [];
|
||||
let rows = html.nodes(
|
||||
'//*[contains(concat(" ", normalize-space(@class), " "), " pm-item-row ") ' +
|
||||
'and contains(concat(" ", normalize-space(@class), " "), " is-unread ")]',
|
||||
);
|
||||
if (!rows.length) {
|
||||
rows = html.nodes(
|
||||
'//*[contains(concat(" ", normalize-space(@class), " "), " pm-item-row ") ' +
|
||||
'and .//*[contains(concat(" ", normalize-space(@class), " "), " pm-item__status--unread ") ' +
|
||||
'or @title="未读"]]',
|
||||
);
|
||||
}
|
||||
for (const row of rows) {
|
||||
const rowLinks = row.strings('.//a[contains(@href, "viewmessage")]/@href');
|
||||
if (!rowLinks.length) continue;
|
||||
const link = rowLinks[0]!.trim();
|
||||
if (!link) continue;
|
||||
links.push(link);
|
||||
this.cachePmItemPreview(link, row);
|
||||
}
|
||||
return links;
|
||||
}
|
||||
|
||||
private cachePmItemPreview(link: string, row: XNode): void {
|
||||
const f = (cls: string) =>
|
||||
NexusAudiencesSiteUserInfo.extractPmItemText(
|
||||
row,
|
||||
`.//*[contains(concat(" ", normalize-space(@class), " "), " ${cls} ")]`,
|
||||
);
|
||||
this.messageListPreviews.set(urljoin(this._baseUrl, link), [
|
||||
f("pm-item__subject"),
|
||||
f("pm-item__time"),
|
||||
f("pm-item__preview"),
|
||||
]);
|
||||
}
|
||||
|
||||
static extractPmItemText(row: XNode, xp: string): string | null {
|
||||
const nodes = row.nodes(xp);
|
||||
if (!nodes.length) return null;
|
||||
return normalizeText(nodes[0]!.text());
|
||||
}
|
||||
|
||||
static extractFirstText(html: XNode, xp: string): string | null {
|
||||
const nodes = html.nodes(xp);
|
||||
if (!nodes.length) return null;
|
||||
return normalizeText(nodes[0]!.text());
|
||||
}
|
||||
|
||||
static extractPmViewMeta(html: XNode, label: string): string | null {
|
||||
const values = html.nodes(
|
||||
'//*[contains(concat(" ", normalize-space(@class), " "), " pm-view__meta ") ' +
|
||||
`and .//*[contains(concat(" ", normalize-space(@class), " "), " pm-view__label ") ` +
|
||||
`and normalize-space()="${label}"]]` +
|
||||
'//*[contains(concat(" ", normalize-space(@class), " "), " pm-view__value ")]',
|
||||
);
|
||||
if (!values.length) return null;
|
||||
return normalizeText(values[0]!.text());
|
||||
}
|
||||
|
||||
private fillEmptyMessageContentFromList(
|
||||
msgLink: string,
|
||||
head: string | null,
|
||||
date: string | null,
|
||||
content: string | null,
|
||||
): Msg {
|
||||
const preview = this.messageListPreviews.get(urljoin(this._baseUrl, msgLink));
|
||||
if (!preview) return [head, date, content];
|
||||
return [head || preview[0], date || preview[1], content || preview[2]];
|
||||
}
|
||||
|
||||
private shouldFetchNextUnreadPage(newLinks: string[]): boolean {
|
||||
if (!newLinks.length) return false;
|
||||
return !this.hasReachedExpectedUnreadCount();
|
||||
}
|
||||
|
||||
private hasReachedExpectedUnreadCount(): boolean {
|
||||
return (
|
||||
!this.messageReadForce &&
|
||||
this.messageUnread > 0 &&
|
||||
this.messageUnread !== UNKNOWN_UNREAD_COUNT &&
|
||||
this.seenUnreadMessageLinks.size >= this.messageUnread
|
||||
);
|
||||
}
|
||||
|
||||
static isEmptyMessageContent(head: string | null, date: string | null, content: string | null): boolean {
|
||||
return ![head, date, content].some((x) => x != null && String(x).trim() !== "");
|
||||
}
|
||||
|
||||
static buildUnreadMailboxPage(box: number): string {
|
||||
return `messages.php?action=viewmailbox&box=${box}&unread=yes`;
|
||||
}
|
||||
|
||||
/** 首页不能带 page 参数;之后 page=1,2,... */
|
||||
private buildNextUnreadMailboxPage(hasUnread: boolean): string | null {
|
||||
if (!hasUnread) return null;
|
||||
const next = this.nextMailPage;
|
||||
this.nextMailPage += 1;
|
||||
return `${this._userMailUnreadPage}&page=${next}`;
|
||||
}
|
||||
|
||||
protected override _parseUserTrafficInfo(htmlText: string): void {
|
||||
super._parseUserTrafficInfo(htmlText);
|
||||
this.parseUserbarInfo(htmlText);
|
||||
}
|
||||
|
||||
protected override _parseUserDetailInfo(htmlText: string): void {
|
||||
super._parseUserDetailInfo(htmlText);
|
||||
this.parseUserbarInfo(htmlText);
|
||||
}
|
||||
|
||||
/** 新版顶部用户栏,覆盖 NexusPHP 通用正则的误判 */
|
||||
private parseUserbarInfo(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
for (const node of html.nodes("//*[@data-uploader-url or @data-uploader-stats]")) {
|
||||
this.parseUserIdentity(node);
|
||||
this.parseUploaderStats(node.attr("data-uploader-stats"));
|
||||
}
|
||||
// data-uploader-stats 不包含分享率,需从 compact metric 的 class 中读取
|
||||
this.parseCompactMetric(html, "ratio", "ratio");
|
||||
this.parseCompactMetric(html, "uploaded", "upload");
|
||||
this.parseCompactMetric(html, "downloaded", "download");
|
||||
this.parseCompactMetric(html, "bonus", "bonus");
|
||||
this.parseCompactMetric(html, "active", "active");
|
||||
}
|
||||
|
||||
private parseUserIdentity(node: XNode): void {
|
||||
const userUrl = node.attr("data-uploader-url") ?? "";
|
||||
const m = reSearch(/userdetails\.php\?id=(\d+)/, userUrl);
|
||||
if (m?.[1]?.trim()) this.userid = m[1].trim();
|
||||
const username = node.attr("data-uploader-label");
|
||||
if (username?.trim()) this.username = username.trim();
|
||||
const level = node.attr("data-uploader-badge");
|
||||
if (level?.trim()) this.userLevel = level.trim();
|
||||
}
|
||||
|
||||
private parseUploaderStats(statsText: string | null): void {
|
||||
if (!statsText) return;
|
||||
let stats: unknown;
|
||||
try {
|
||||
stats = JSON.parse(statsText);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!Array.isArray(stats)) return;
|
||||
for (const item of stats) {
|
||||
if (!item || typeof item !== "object" || Array.isArray(item)) continue;
|
||||
const it = item as Record<string, unknown>;
|
||||
const label = String(it.label || "").replace(/^[ ：:]+|[ ：:]+$/g, "");
|
||||
const tone = String(it.tone || "").trim();
|
||||
const value = String(it.value || "").trim();
|
||||
this.setMetricValue(value, label, tone);
|
||||
}
|
||||
}
|
||||
|
||||
private parseCompactMetric(html: XNode, metric: string, field: string): void {
|
||||
let values = html.strings(
|
||||
`//*[contains(concat(" ", normalize-space(@class), " "), " site-userbar__compact-metric--${metric} ")]` +
|
||||
"//span[normalize-space()][last()]/text()",
|
||||
);
|
||||
if (!values.length) {
|
||||
values = html.strings(
|
||||
`//*[contains(concat(" ", normalize-space(@class), " "), " site-userbar__compact-metric--${metric} ")]` +
|
||||
"/text()",
|
||||
);
|
||||
}
|
||||
if (values.length) this.setMetricValue(values[values.length - 1]!.trim(), null, null, field);
|
||||
}
|
||||
|
||||
private setMetricValue(value: string, label?: string | null, tone?: string | null, field?: string | null): void {
|
||||
if (!value) return;
|
||||
const key = field || tone || label || "";
|
||||
if (["uploaded", "上传量", "upload"].includes(key)) this.upload = this.numFilesize(value);
|
||||
else if (["downloaded", "下载量", "download"].includes(key)) this.download = this.numFilesize(value);
|
||||
else if (["bonus", "爆米花"].includes(key)) this.bonus = parseFloatMp(value);
|
||||
else if (key === "ratio") this.ratio = parseFloatMp(value);
|
||||
else if (["active", "活跃"].includes(key)) {
|
||||
const m = reSearch(/↑\s*(\d+)\s*\/\s*↓\s*(\d+)/, value);
|
||||
if (m) {
|
||||
this.seeding = parseIntMp(m[1]);
|
||||
this.leeching = parseIntMp(m[2]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static parseInboxUnread(link: XNode): number | null {
|
||||
for (const t of [link.attr("title"), link.attr("aria-label")]) {
|
||||
const u = NexusAudiencesSiteUserInfo.extractInboxUnreadPair(t);
|
||||
if (u !== null) return u;
|
||||
}
|
||||
for (const t of link.strings('.//*[contains(@class, "site-userbar__compact-tool-badge--unread")]/text()')) {
|
||||
const u = NexusAudiencesSiteUserInfo.extractInboxUnreadBadge(t);
|
||||
if (u !== null) return u;
|
||||
}
|
||||
if (NexusAudiencesSiteUserInfo.hasInboxUnreadMarker(link)) return UNKNOWN_UNREAD_COUNT;
|
||||
return null;
|
||||
}
|
||||
|
||||
/** 总数/未读数格式,如 1749/172 */
|
||||
static extractInboxUnreadPair(text: string | null | undefined): number | null {
|
||||
const t = normalizeText(text);
|
||||
if (!t) return null;
|
||||
const m = reSearch(/(?:收件箱\s*)?(\d[\d,]*)\s*\/\s*(\d[\d,]*)/, t);
|
||||
return m ? parseIntMp(m[2]) : null;
|
||||
}
|
||||
|
||||
static extractInboxUnreadBadge(text: string | null | undefined): number | null {
|
||||
const pair = NexusAudiencesSiteUserInfo.extractInboxUnreadPair(text);
|
||||
if (pair !== null) return pair;
|
||||
const t = normalizeText(text);
|
||||
if (!t) return null;
|
||||
const m = /^(\d[\d,]*)$/.exec(t);
|
||||
return m ? parseIntMp(m[1]) : null;
|
||||
}
|
||||
|
||||
static hasInboxUnreadMarker(link: XNode): boolean {
|
||||
if ((link.attr("class") ?? "").includes("site-userbar__compact-tool--has-unread")) return true;
|
||||
return link.nodes('.//*[contains(@class, "site-userbar__compact-tool-badge--unread")]').length > 0;
|
||||
}
|
||||
|
||||
/** 做种页只读 Total 汇总行 */
|
||||
protected override async _parseSeedingPages(): Promise<void> {
|
||||
if (!this._torrentSeedingPage) return;
|
||||
this._torrentSeedingHeaders = { Referer: urljoin(this._baseUrl, this._userDetailPage ?? "") };
|
||||
const htmlText = await this._getPageContent(
|
||||
urljoin(this._baseUrl, this._torrentSeedingPage),
|
||||
this._torrentSeedingParams,
|
||||
this._torrentSeedingHeaders,
|
||||
);
|
||||
if (!htmlText) return;
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
const totalRow = html.nodes('//table[@class="table table-bordered"]//tr[td[1][normalize-space()="Total"]]');
|
||||
if (!totalRow.length) return;
|
||||
const count = totalRow[0]!.strings("./td[2]/text()");
|
||||
const size = totalRow[0]!.strings("./td[3]/text()");
|
||||
this.seeding = count.length ? parseIntMp(count[0]) : 0;
|
||||
this.seedingSize = size.length ? this.numFilesize(size[0]!.trim()) : 0;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import { createMockFetch, HHCLUB_USER_DETAILS, HHCLUB_USER_INDEX, HHCLUB_USER_INDEX_NO_PANEL } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { NexusHhanclubSiteUserInfo } from "./nexus-hhanclub.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
describe("NexusHhanclubSiteUserInfo(MP nexus_hhanclub.py)", () => {
|
||||
it("#user-info-panel 取上传/下载/分享率,详情取加入日期与 _Name 等级", async () => {
|
||||
const m = createMockFetch({ "/": { body: HHCLUB_USER_INDEX }, "/userdetails.php": { body: HHCLUB_USER_DETAILS } });
|
||||
const info = await runParser(NexusHhanclubSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info).toMatchObject({
|
||||
schema: "NexusHhanclub",
|
||||
userId: "4567",
|
||||
username: "hana",
|
||||
upload: Math.round(1.2 * 1024 ** 4),
|
||||
download: 300 * 1024 ** 3,
|
||||
ratio: 3.5,
|
||||
bonus: 6789,
|
||||
joinAt: "2022-02-02 02:02:02",
|
||||
userLevel: "Power User",
|
||||
});
|
||||
});
|
||||
|
||||
it("面板缺失(改版):与 MP 一样抛 IndexError", async () => {
|
||||
const m = createMockFetch({ "/": { body: HHCLUB_USER_INDEX_NO_PANEL } });
|
||||
await expect(runParser(NexusHhanclubSiteUserInfo, { fetch: m.fetch })).rejects.toThrow(/IndexError/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/nexus_hhanclub.py @ e6820a8
|
||||
*
|
||||
* 憨憨(HHClub)站点用户信息解析器,继承 NexusPHP 通用解析。
|
||||
*
|
||||
* 函数对照(`NexusHhanclubSiteUserInfo`):`_parse_user_traffic_info` → `_parseUserTrafficInfo`、
|
||||
* `_parse_user_detail_info` → `_parseUserDetailInfo`、`_get_user_level` → `_getUserLevel`。XPath 与正则逐字保留。
|
||||
*
|
||||
* 与 MP 一致:`#user-info-panel` 结构缺失时 `xpath(...)[0]` 抛错并中断整站解析(交给 schema 回退)。
|
||||
*/
|
||||
|
||||
import { normalizeDatetime, parseFloatMp, reSearch } from "./common.ts";
|
||||
import { hasChildElements, parseHtml, type XNode } from "./dom.ts";
|
||||
import { NexusPhpSiteUserInfo, round3 } from "./nexus-php.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
function first(values: string[], expr: string): string {
|
||||
if (!values.length) throw new Error(`IndexError: list index out of range (${expr})`);
|
||||
return values[0]!;
|
||||
}
|
||||
|
||||
export class NexusHhanclubSiteUserInfo extends NexusPhpSiteUserInfo {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.NexusHhanclub;
|
||||
|
||||
protected override _parseUserTrafficInfo(htmlText: string): void {
|
||||
super._parseUserTrafficInfo(htmlText);
|
||||
const text = this._prepareHtmlText(htmlText);
|
||||
const html = parseHtml(text);
|
||||
if (!html) throw new Error("AttributeError: 'NoneType' object has no attribute 'xpath'");
|
||||
const x = (e: string) => first(html.strings(e), e);
|
||||
// 上传、下载、分享率
|
||||
const uploadMatch = reSearch(
|
||||
/[_<>/a-zA-Z-="'\s#;]+([\d,.\s]+[KMGTPI]*B)/,
|
||||
x('//*[@id="user-info-panel"]/div[2]/div[2]/div[4]/text()'),
|
||||
);
|
||||
const downloadMatch = reSearch(
|
||||
/[_<>/a-zA-Z-="'\s#;]+([\d,.\s]+[KMGTPI]*B)/,
|
||||
x('//*[@id="user-info-panel"]/div[2]/div[2]/div[5]/text()'),
|
||||
);
|
||||
const ratioMatch = reSearch(
|
||||
/分享率\][:：_<>/a-zA-Z-="'\s#;]+([\d,.\s]+)/,
|
||||
x('//*[@id="user-info-panel"]/div[2]/div[1]/div[1]/div/text()'),
|
||||
);
|
||||
this.upload = uploadMatch ? this.numFilesize(uploadMatch[1]!.trim()) : 0;
|
||||
this.download = downloadMatch ? this.numFilesize(downloadMatch[1]!.trim()) : 0;
|
||||
const calcRatio = this.download <= 0.0 ? 0.0 : round3(this.upload / this.download);
|
||||
this.ratio = ratioMatch?.[1]?.trim() ? parseFloatMp(ratioMatch[1]) : calcRatio;
|
||||
}
|
||||
|
||||
protected override _parseUserDetailInfo(htmlText: string): void {
|
||||
super._parseUserDetailInfo(htmlText);
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
const joinAt = html.strings('//span[contains(text(), "加入日期")]/following-sibling::span/span/@title');
|
||||
if (joinAt.length) this.joinAt = normalizeDatetime(joinAt[0]!.trim()) ?? null;
|
||||
}
|
||||
|
||||
protected override _getUserLevel(html: XNode): void {
|
||||
super._getUserLevel(html);
|
||||
const level = html.strings('//b[contains(@class, "_Name")]/text()');
|
||||
if (level.length) this.userLevel = level[0]!;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
import {
|
||||
createMockFetch,
|
||||
NEXUS_PHP_CF_CHALLENGE,
|
||||
NEXUS_PHP_INDEX,
|
||||
NEXUS_PHP_LOGIN_PAGE,
|
||||
NEXUS_PHP_SEEDING,
|
||||
NEXUS_PHP_USERDETAILS,
|
||||
} from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { NexusPhpSiteUserInfo } from "./nexus-php.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
describe("NexusPhpSiteUserInfo(MP nexus_php.py)", () => {
|
||||
it("首页 + 详情 + 做种页:流量/魔力/等级/加入日期/做种统计", async () => {
|
||||
const m = createMockFetch({
|
||||
"/": { body: NEXUS_PHP_INDEX },
|
||||
"/userdetails.php": { body: NEXUS_PHP_USERDETAILS },
|
||||
"/getusertorrentlistajax.php": { body: NEXUS_PHP_SEEDING },
|
||||
});
|
||||
const info = await runParser(NexusPhpSiteUserInfo, { fetch: m.fetch, cookie: "uid=1; pass=secret" });
|
||||
expect(info).toMatchObject({
|
||||
schema: "NexusPhp",
|
||||
domain: "example.com",
|
||||
userId: "10086",
|
||||
username: "alice",
|
||||
userLevel: "Veteran User",
|
||||
joinAt: "2021-03-04 05:06:07",
|
||||
upload: Math.round(1.5 * 1024 ** 4),
|
||||
download: Math.round(477.3 * 1024 ** 3),
|
||||
ratio: 3.217,
|
||||
bonus: 12345.6,
|
||||
leeching: 2,
|
||||
seeding: 2,
|
||||
seedingSize: 10 * 1024 ** 3 + Math.round(1.5 * 1024 ** 4),
|
||||
seedingInfo: [
|
||||
[5, 10 * 1024 ** 3],
|
||||
[12, Math.round(1.5 * 1024 ** 4)],
|
||||
],
|
||||
messageUnread: 3,
|
||||
errMsg: null,
|
||||
});
|
||||
// 请求顺序:首页 → 用户详情 → 做种(流量页已合并到首页解析)
|
||||
expect(m.requests.map((r) => r.url.pathname)).toEqual(["/", "/userdetails.php", "/getusertorrentlistajax.php"]);
|
||||
expect(m.requests[2]!.url.search).toBe("?userid=10086&type=seeding");
|
||||
expect(m.requests[0]!.headers.cookie).toBe("uid=1; pass=secret");
|
||||
expect(JSON.stringify(info)).not.toContain("secret");
|
||||
});
|
||||
|
||||
it("登录页(含密码框):不再继续请求,errMsg 提示 cookie 过期", async () => {
|
||||
const m = createMockFetch({ "/": { body: NEXUS_PHP_LOGIN_PAGE } });
|
||||
const info = await runParser(NexusPhpSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info.userId).toBeNull();
|
||||
expect(info.errMsg).toContain("未检测到已登陆");
|
||||
expect(m.requests).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("Cloudflare 挑战页按空内容处理并记录原因", async () => {
|
||||
const m = createMockFetch({ "/": { body: NEXUS_PHP_CF_CHALLENGE, status: 403 } });
|
||||
const info = await runParser(NexusPhpSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info.errMsg).toContain("未检测到已登陆");
|
||||
expect(info.raw.http_errors).toEqual([expect.stringContaining("Cloudflare")]);
|
||||
});
|
||||
|
||||
it("_fixup_next_page_url:补 userid/type,已有 userid 原样,无 userid 停止", () => {
|
||||
expect(NexusPhpSiteUserInfo.fixupNextPageUrl("getusertorrentlistajax.php?page=1", "7")).toBe(
|
||||
"getusertorrentlistajax.php?page=1&userid=7&type=seeding",
|
||||
);
|
||||
expect(NexusPhpSiteUserInfo.fixupNextPageUrl("x.php?userid=3&page=2", "7")).toBe("x.php?userid=3&page=2");
|
||||
expect(NexusPhpSiteUserInfo.fixupNextPageUrl("x.php?page=2", null)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,361 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/nexus_php.py @ e6820a8
|
||||
*
|
||||
* NexusPHP 站点用户信息解析器。
|
||||
*
|
||||
* 迁移自 MoviePilot e6820a8(GPLv3)`app/modules/indexer/parser/nexus_php.py` `NexusPhpSiteUserInfo`,
|
||||
* 函数对照:`_parse_site_page` → `_parseSitePage`、`_parse_message_unread` → `_parseMessageUnread`、
|
||||
* `_parse_user_base_info` → `_parseUserBaseInfo`、`_parse_user_traffic_info` → `_parseUserTrafficInfo`、
|
||||
* `_parse_ucoin` → `parseUcoin`、`_parse_user_torrent_seeding_info` → `_parseUserTorrentSeedingInfo`、
|
||||
* `_fixup_next_page_url` → `fixupNextPageUrl`、`_parse_user_detail_info` → `_parseUserDetailInfo`、
|
||||
* `_fixup_torrent_seeding_page` → `_fixupTorrentSeedingPage`、`_get_user_level` → `_getUserLevel`、
|
||||
* `_parse_message_unread_links` → `_parseMessageUnreadLinks`、`_parse_message_content` → `_parseMessageContent`、
|
||||
* `_fixup_traffic_info` → `_fixupTrafficInfo`。XPath 与正则逐字保留。
|
||||
*
|
||||
* 差异:做种表"大小列/做种人数列"条数不一致时 MP 抛 IndexError 中断整站解析,这里缺失的做种人数记 0。
|
||||
*/
|
||||
import { type NextPage, SiteParserBase } from "./base.ts";
|
||||
import { normalizeDatetime, parseFloatMp, parseIntMp, reFindall, reSearch } from "./common.ts";
|
||||
import { hasChildElements, parseHtml, type XNode } from "./dom.ts";
|
||||
import { SiteSchema } from "./types.ts";
|
||||
|
||||
/** Python `round(x, 3)` 的近似 */
|
||||
export function round3(x: number): number {
|
||||
return Math.round(x * 1000) / 1000;
|
||||
}
|
||||
|
||||
export class NexusPhpSiteUserInfo extends SiteParserBase {
|
||||
static override schema: import("./types.ts").SiteSchemaValue | null = SiteSchema.NexusPhp;
|
||||
|
||||
protected _parseSitePage(htmlText: string): void {
|
||||
const text = this._prepareHtmlText(htmlText);
|
||||
const userDetail = reSearch(/userdetails.php\?id=(\d+)/, text);
|
||||
if (userDetail?.[0].trim()) {
|
||||
this._userDetailPage = userDetail[0].trim().replace(/^\/+/, "");
|
||||
this.userid = userDetail[1]!;
|
||||
this._torrentSeedingPage = `getusertorrentlistajax.php?userid=${this.userid}&type=seeding`;
|
||||
} else {
|
||||
const m = reSearch(/(userdetails)/, text);
|
||||
if (m?.[0].trim()) {
|
||||
this._userDetailPage = m[0].trim().replace(/^\/+/, "");
|
||||
this.userid = null;
|
||||
this._torrentSeedingPage = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** 解析未读短消息数量 */
|
||||
protected _parseMessageUnread(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
const messageLabels = html.nodes('//a[@href="messages.php"]/..');
|
||||
messageLabels.push(...html.nodes('//a[contains(@href, "messages.php")]/..'));
|
||||
if (messageLabels.length) {
|
||||
const messageText = messageLabels[0]!.text();
|
||||
const matches = reFindall(/[^Date](信息箱\s*|\((?![^)]*:)|你有 )(\d+)/, messageText);
|
||||
if (matches.length && matches[matches.length - 1]!.length === 2) {
|
||||
this.messageUnread = parseIntMp(matches[matches.length - 1]![1]);
|
||||
} else if (/^\d+$/.test(messageText)) {
|
||||
this.messageUnread = parseIntMp(messageText);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
// 合并解析,减少额外请求调用
|
||||
this._parseUserTrafficInfo(htmlText);
|
||||
this._userTrafficPage = null;
|
||||
this._parseMessageUnread(htmlText);
|
||||
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
let ret = html.strings(`//a[contains(@href, "userdetails") and contains(@href, "${this.userid}")]//b//text()`);
|
||||
if (ret.length) {
|
||||
this.username = String(ret[0]);
|
||||
return;
|
||||
}
|
||||
ret = html.strings(`//a[contains(@href, "userdetails") and contains(@href, "${this.userid}")]//text()`);
|
||||
if (ret.length) this.username = String(ret[0]);
|
||||
ret = html.strings('//a[contains(@href, "userdetails")]//strong//text()');
|
||||
if (ret.length) this.username = String(ret[0]);
|
||||
}
|
||||
|
||||
protected _parseUserTrafficInfo(htmlText: string): void {
|
||||
const text = this._prepareHtmlText(htmlText);
|
||||
const uploadMatch = reSearch(/[^总]上[传傳]量?[:：_<>/a-zA-Z-="'\s#;]+([\d,.\s]+[KMGTPI]*B)/i, text);
|
||||
this.upload = uploadMatch ? this.numFilesize(uploadMatch[1]!.trim()) : 0;
|
||||
const downloadMatch = reSearch(/[^总子影力]下[载載]量?[:：_<>/a-zA-Z-="'\s#;]+([\d,.\s]+[KMGTPI]*B)/i, text);
|
||||
this.download = downloadMatch ? this.numFilesize(downloadMatch[1]!.trim()) : 0;
|
||||
const ratioMatch = reSearch(/分享率[:：_<>/a-zA-Z-="'\s#;]+([\d,.\s]+)/, text);
|
||||
const calcRatio = this.download <= 0.0 ? 0.0 : round3(this.upload / this.download);
|
||||
this.ratio = ratioMatch && ratioMatch[1]!.trim() ? parseFloatMp(ratioMatch[1]) : calcRatio;
|
||||
const leechingMatch = reSearch(/(Torrents leeching|下载中)[一-龥\D\s]+(\d+)[\s\S]+</, text);
|
||||
this.leeching = leechingMatch && leechingMatch[2]!.trim() ? parseIntMp(leechingMatch[2]) : 0;
|
||||
|
||||
const html = parseHtml(text);
|
||||
const [hasUcoin, bonus] = NexusPhpSiteUserInfo.parseUcoin(html);
|
||||
this.bonus = bonus;
|
||||
if (hasUcoin) return;
|
||||
const tmps = html ? html.strings('//a[contains(@href,"mybonus")]/text()') : [];
|
||||
if (tmps.length) {
|
||||
const bonusText = String(tmps[0]).trim();
|
||||
const m = reSearch(/([\d,.]+)/, bonusText);
|
||||
if (m && m[1]!.trim()) {
|
||||
this.bonus = parseFloatMp(m[1]);
|
||||
return;
|
||||
}
|
||||
}
|
||||
let bonusMatch = reSearch(/mybonus.[[\]:：<>/a-zA-Z_\-="'\s#;.(使用魔力值豆]+\s*([\d,.]+)[<()&\s]/, text);
|
||||
if (bonusMatch && bonusMatch[1]!.trim()) {
|
||||
this.bonus = parseFloatMp(bonusMatch[1]);
|
||||
return;
|
||||
}
|
||||
bonusMatch = reSearch(/[魔力值|\]][[\]:：<>/a-zA-Z_\-="'\s#;]+\s*([\d,.]+|"[\d,.]+")[<>()&\s]/s, text);
|
||||
if (bonusMatch && bonusMatch[1]!.trim()) {
|
||||
this.bonus = parseFloatMp(bonusMatch[1]!.replace(/^"+|"+$/g, ""));
|
||||
}
|
||||
}
|
||||
|
||||
/** MP `_parse_ucoin`:金/银/铜币统一换算为铜币 */
|
||||
static parseUcoin(html: XNode | null): [boolean, number] {
|
||||
if (hasChildElements(html)) {
|
||||
let gold: number | null = null;
|
||||
let silver: number | null = null;
|
||||
let copper: number | null = null;
|
||||
const golds = html.strings('//span[@class = "ucoin-symbol ucoin-gold"]//text()');
|
||||
if (golds.length) gold = parseFloatMp(golds[golds.length - 1]);
|
||||
const silvers = html.strings('//span[@class = "ucoin-symbol ucoin-silver"]//text()');
|
||||
if (silvers.length) silver = parseFloatMp(silvers[silvers.length - 1]);
|
||||
const coppers = html.strings('//span[@class = "ucoin-symbol ucoin-copper"]//text()');
|
||||
if (coppers.length) copper = parseFloatMp(coppers[coppers.length - 1]);
|
||||
if (gold || silver || copper) {
|
||||
return [true, (gold || 0) * 100 * 100 + (silver || 0) * 100 + (copper || 0)];
|
||||
}
|
||||
}
|
||||
return [false, 0.0];
|
||||
}
|
||||
|
||||
protected _parseUserTorrentSeedingInfo(htmlText: string, multiPage = false): NextPage {
|
||||
const html = parseHtml(String(htmlText ?? "").replaceAll("\\/", "/"));
|
||||
if (!hasChildElements(html)) return null;
|
||||
|
||||
// 首页存在扩展链接,使用扩展链接
|
||||
const seedingUrlText = html.strings('//a[contains(@href,"torrents.php") and contains(@href,"seeding")]/@href');
|
||||
if (multiPage === false && seedingUrlText.length && seedingUrlText[0]!.trim()) {
|
||||
this._torrentSeedingPage = seedingUrlText[0]!.trim();
|
||||
return this._torrentSeedingPage;
|
||||
}
|
||||
|
||||
let sizeCol = 3;
|
||||
let seedersCol = 4;
|
||||
const sizeColXpath =
|
||||
'//tr[position()=1]/td[(img[@class="size"] and img[@alt="size"]) or (text() = "大小") or (a/img[@class="size" and @alt="size"])]';
|
||||
if (html.nodes(sizeColXpath).length) {
|
||||
sizeCol = html.nodes(`${sizeColXpath}/preceding-sibling::td`).length + 1;
|
||||
}
|
||||
const seedersColXpath =
|
||||
'//tr[position()=1]/td[(img[@class="seeders"] and img[@alt="seeders"]) or (text() = "在做种") or (a/img[@class="seeders" and @alt="seeders"])]';
|
||||
if (html.nodes(seedersColXpath).length) {
|
||||
seedersCol = html.nodes(`${seedersColXpath}/preceding-sibling::td`).length + 1;
|
||||
}
|
||||
|
||||
let pageSeeding = 0;
|
||||
let pageSeedingSize = 0;
|
||||
const pageSeedingInfo: Array<[number, number]> = [];
|
||||
const tableClass = html.nodes('//table[@class="torrents"]').length ? '//table[@class="torrents"]' : "";
|
||||
const seedingSizes = html.nodes(`${tableClass}//tr[position()>1]/td[${sizeCol}]`);
|
||||
let seedingSeeders = html.strings(`${tableClass}//tr[position()>1]/td[${seedersCol}]/b/a/text()`);
|
||||
if (!seedingSeeders.length)
|
||||
seedingSeeders = html.strings(`${tableClass}//tr[position()>1]/td[${seedersCol}]//text()`);
|
||||
if (seedingSizes.length && seedingSeeders.length) {
|
||||
pageSeeding = seedingSizes.length;
|
||||
for (let i = 0; i < seedingSizes.length; i++) {
|
||||
const size = this.numFilesize(seedingSizes[i]!.text().trim());
|
||||
const seeders = parseIntMp(seedingSeeders[i]);
|
||||
pageSeedingSize += size;
|
||||
pageSeedingInfo.push([seeders, size]);
|
||||
}
|
||||
}
|
||||
this.seeding += pageSeeding;
|
||||
this.seedingSize += pageSeedingSize;
|
||||
this.seedingInfo.push(...pageSeedingInfo);
|
||||
|
||||
// 是否存在下页数据
|
||||
let nextPage: string | null = null;
|
||||
const nextPageText = html.strings(
|
||||
'//a[contains(.//text(), "下一页") or contains(.//text(), "下一頁") or contains(.//text(), ">")]/@href',
|
||||
);
|
||||
// 防止识别到详情页
|
||||
while (nextPageText.length) {
|
||||
nextPage = nextPageText.pop()!.trim();
|
||||
if (!nextPage.startsWith("details.php")) break;
|
||||
nextPage = null;
|
||||
}
|
||||
if (nextPage) nextPage = NexusPhpSiteUserInfo.fixupNextPageUrl(nextPage, this.userid);
|
||||
return nextPage;
|
||||
}
|
||||
|
||||
/** MP `_fixup_next_page_url`:补齐 userid/type,无法补齐时停止翻页 */
|
||||
static fixupNextPageUrl(nextPage: string, userid: string | null): string | null {
|
||||
const hashIdx = nextPage.indexOf("#");
|
||||
const fragment = hashIdx >= 0 ? nextPage.slice(hashIdx) : "";
|
||||
const noFrag = hashIdx >= 0 ? nextPage.slice(0, hashIdx) : nextPage;
|
||||
const qIdx = noFrag.indexOf("?");
|
||||
const path = qIdx >= 0 ? noFrag.slice(0, qIdx) : noFrag;
|
||||
const query = qIdx >= 0 ? noFrag.slice(qIdx + 1) : "";
|
||||
// parse_qsl → dict:同名键后者覆盖,保留插入顺序
|
||||
const params = new Map<string, string>();
|
||||
for (const [k, v] of new URLSearchParams(query)) params.set(k, v);
|
||||
if (params.get("userid")) return nextPage;
|
||||
if (!userid) return null;
|
||||
params.set("userid", userid);
|
||||
if (!params.has("type")) params.set("type", "seeding");
|
||||
const qs = new URLSearchParams([...params.entries()]).toString();
|
||||
return `${path}${qs ? `?${qs}` : ""}${fragment}`;
|
||||
}
|
||||
|
||||
protected _parseUserDetailInfo(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
this._getUserLevel(html);
|
||||
this._fixupTrafficInfo(html);
|
||||
|
||||
// 加入日期
|
||||
const joinAtText = html.strings(
|
||||
'//tr/td[text()="加入日期" or text()="注册日期" or *[text()="加入日期"]]/following-sibling::td[1]//text()' +
|
||||
'|//div/b[text()="加入日期"]/../text()' +
|
||||
'|//*[@id="outer"]/table/tr/td/div/div[1]/div[2]/div[3]/span[1]/span/@title',
|
||||
);
|
||||
if (joinAtText.length) {
|
||||
this.joinAt = normalizeDatetime(joinAtText[0]!.split(" (")[0]!.trim()) ?? null;
|
||||
}
|
||||
|
||||
// 做种体积 & 做种数(seeding 页面获取不到的话,此处再获取一次)
|
||||
const seedingSizes = html.nodes(
|
||||
'//tr/td[text()="当前上传"]/following-sibling::td[1]//table[tr[1][td[4 and text()="尺寸"]]]//tr[position()>1]/td[4]',
|
||||
);
|
||||
const seedingSeeders = html.strings(
|
||||
'//tr/td[text()="当前上传"]/following-sibling::td[1]//table[tr[1][td[5 and text()="做种者"]]]//tr[position()>1]/td[5]//text()',
|
||||
);
|
||||
let tmpSeeding = seedingSizes.length;
|
||||
let tmpSeedingSize = 0;
|
||||
const tmpSeedingInfo: Array<[number, number]> = [];
|
||||
for (let i = 0; i < seedingSizes.length; i++) {
|
||||
const size = this.numFilesize(seedingSizes[i]!.text().trim());
|
||||
const seeders = parseIntMp(seedingSeeders[i]);
|
||||
tmpSeedingSize += size;
|
||||
tmpSeedingInfo.push([seeders, size]);
|
||||
}
|
||||
if (!this.seedingSize) this.seedingSize = tmpSeedingSize;
|
||||
if (!this.seeding) this.seeding = tmpSeeding;
|
||||
if (!this.seedingInfo.length) this.seedingInfo = tmpSeedingInfo;
|
||||
|
||||
const stat = html.strings('//tr/td[text()="做种统计"]/following-sibling::td[1]//text()');
|
||||
if (stat.length) {
|
||||
const seedingMatch = reSearch(/总做种数:\s+(\d+)/i, stat[0]);
|
||||
const seedingSizeMatch = reSearch(/总做种体积:\s+([\d,.\s]+[KMGTPI]*B)/i, stat[0]);
|
||||
tmpSeeding = seedingMatch?.[1] ? parseIntMp(seedingMatch[1]) : 0;
|
||||
tmpSeedingSize = seedingSizeMatch ? this.numFilesize(seedingSizeMatch[1]!.trim()) : 0;
|
||||
}
|
||||
if (!this.seedingSize) this.seedingSize = tmpSeedingSize;
|
||||
if (!this.seeding) this.seeding = tmpSeeding;
|
||||
|
||||
this._fixupTorrentSeedingPage(html);
|
||||
}
|
||||
|
||||
/** MP `_fixup_torrent_seeding_page` */
|
||||
protected _fixupTorrentSeedingPage(html: XNode): void {
|
||||
let seedingUrlText = html.strings(
|
||||
'//a[contains(@href,"getusertorrentlist.php") and contains(@href,"seeding")]/@href',
|
||||
);
|
||||
if (seedingUrlText.length) this._torrentSeedingPage = seedingUrlText[0]!.trim();
|
||||
// 从JS调用种获取用户ID
|
||||
seedingUrlText = html.strings(
|
||||
'//a[contains(@href, "javascript: getusertorrentlistajax") and contains(@href,"seeding")]/@href',
|
||||
);
|
||||
const csrfText = html.strings('//meta[@name="x-csrf"]/@content');
|
||||
if (!this._torrentSeedingPage && seedingUrlText.length) {
|
||||
const userJs = reSearch(/javascript: getusertorrentlistajax\(\s*'(\d+)/, seedingUrlText[0]);
|
||||
if (userJs?.[1]?.trim()) {
|
||||
this.userid = userJs[1].trim();
|
||||
this._torrentSeedingPage = `getusertorrentlistajax.php?userid=${this.userid}&type=seeding`;
|
||||
}
|
||||
} else if (seedingUrlText.length && csrfText.length) {
|
||||
if (csrfText[0]!.trim()) {
|
||||
this._torrentSeedingPage = "ajax_getusertorrentlist.php";
|
||||
this._torrentSeedingParams = { userid: this.userid, type: "seeding", csrf: csrfText[0]!.trim() };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** MP `_get_user_level`:图片等级取 title,否则取文本 */
|
||||
protected _getUserLevel(html: XNode): void {
|
||||
let levels = html.strings(
|
||||
'//tr/td[text()="等級" or text()="等级" or *[text()="等级"]]/following-sibling::td[1]/img[1]/@title',
|
||||
);
|
||||
if (levels.length) {
|
||||
this.userLevel = levels[0]!.trim();
|
||||
return;
|
||||
}
|
||||
let nodes = html.nodes(
|
||||
'//tr/td[text()="等級" or text()="等级"]/following-sibling::td[1 and not(img)]' +
|
||||
'|//tr/td[text()="等級" or text()="等级"]/following-sibling::td[1 and img[not(@title)]]',
|
||||
);
|
||||
if (nodes.length) {
|
||||
this.userLevel = nodes[0]!.text().trim();
|
||||
return;
|
||||
}
|
||||
nodes = html.nodes('//tr/td[text()="等級" or text()="等级"]/following-sibling::td[1]');
|
||||
if (nodes.length) {
|
||||
this.userLevel = nodes[0]!.text().trim();
|
||||
return;
|
||||
}
|
||||
levels = html.strings('//a[contains(@href, "userdetails")]/text()');
|
||||
if (!this.userLevel && levels.length) {
|
||||
for (const t of levels) {
|
||||
const m = reSearch(/\[(.*)]/, t);
|
||||
if (m?.[1]?.trim()) {
|
||||
this.userLevel = m[1].trim();
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
protected _parseMessageUnreadLinks(htmlText: string, msgLinks: string[]): NextPage {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return null;
|
||||
msgLinks.push(...html.strings('//tr[not(./td/img[@alt="Read"])]/td/a[contains(@href, "viewmessage")]/@href'));
|
||||
const nextPageText = html.strings('//a[contains(.//text(), "下一页") or contains(.//text(), "下一頁")]/@href');
|
||||
return nextPageText.length ? nextPageText[nextPageText.length - 1]!.trim() : null;
|
||||
}
|
||||
|
||||
protected _parseMessageContent(htmlText: string): [string | null, string | null, string | null] {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return [null, null, null];
|
||||
let head: string | null = null;
|
||||
const messageHead = html.strings('//h1/text()|//div[@class="layui-card-header"]/span[1]/text()');
|
||||
if (messageHead.length) head = messageHead[messageHead.length - 1]!.trim();
|
||||
let date: string | null = null;
|
||||
const messageDate = html.nodes(
|
||||
'//h1/following-sibling::table[.//tr/td[@class="colhead"]]//tr[2]/td[2]|//div[@class="layui-card-header"]/span[2]/span[2]',
|
||||
);
|
||||
if (messageDate.length) date = messageDate[0]!.text().trim();
|
||||
let content: string | null = null;
|
||||
const messageContent = html.nodes(
|
||||
'//h1/following-sibling::table[.//tr/td[@class="colhead"]]//tr[3]/td|//div[contains(@class,"layui-card-body")]',
|
||||
);
|
||||
if (messageContent.length) content = messageContent[0]!.text().trim();
|
||||
return [head, date, content];
|
||||
}
|
||||
|
||||
/** MP `_fixup_traffic_info` */
|
||||
protected _fixupTrafficInfo(html: XNode): void {
|
||||
if (!this.bonus) {
|
||||
const bonusText = html.strings('//tr/td[text()="魔力值" or text()="猫粮"]/following-sibling::td[1]/text()');
|
||||
if (bonusText.length) this.bonus = parseFloatMp(bonusText[0]!.trim());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
import { createMockFetch, NEXUS_PHP_INDEX, NEXUS_PROJECT_USER_INDEX } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { round3 } from "./nexus-php.ts";
|
||||
import { NexusProjectSiteUserInfo } from "./nexus-project.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
describe("NexusProjectSiteUserInfo(MP nexus_project.py)", () => {
|
||||
it("user_detail.php?uid= 变种 + span#user_info 无 B 后缀流量", async () => {
|
||||
const m = createMockFetch({ "/": { body: NEXUS_PROJECT_USER_INDEX } });
|
||||
const info = await runParser(NexusProjectSiteUserInfo, { fetch: m.fetch });
|
||||
const up = Math.round(445.11 * 1024 ** 3);
|
||||
const down = Math.round(29.61 * 1024 ** 3);
|
||||
expect(info).toMatchObject({
|
||||
schema: "NexusProject",
|
||||
userId: "77",
|
||||
username: "carol",
|
||||
upload: up,
|
||||
download: down,
|
||||
ratio: round3(up / down),
|
||||
});
|
||||
expect(m.requests.map((r) => r.url.pathname + r.url.search)).toContain("/viewusertorrents.php?id=77&show=seeding");
|
||||
expect(m.requests.map((r) => r.url.pathname)).toContain("/user_detail.php"); // 与 MP 一致:正则只取 user_detail.php?uid=,不含 p_user/ 前缀
|
||||
});
|
||||
|
||||
it("标准 NexusPHP 首页:回落通用流量解析", async () => {
|
||||
const m = createMockFetch({ "/": { body: NEXUS_PHP_INDEX } });
|
||||
const info = await runParser(NexusProjectSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info).toMatchObject({ userId: "10086", upload: Math.round(1.5 * 1024 ** 4), ratio: 3.217 });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,58 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/nexus_project.py @ e6820a8
|
||||
*
|
||||
* NexusProject 站点用户信息解析器,继承 NexusPHP 通用解析。
|
||||
*
|
||||
* 函数对照(`NexusProjectSiteUserInfo`):`_parse_site_page` → `_parseSitePage`(兼容 `user_detail.php?uid=` 变种)、
|
||||
* `_parse_user_traffic_info` → `_parseUserTrafficInfo`(兼容 `span#user_info` 无 B 后缀的流量文本)。正则逐字保留。
|
||||
*/
|
||||
import { reSearch } from "./common.ts";
|
||||
import { parseHtml } from "./dom.ts";
|
||||
import { NexusPhpSiteUserInfo, round3 } from "./nexus-php.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
export class NexusProjectSiteUserInfo extends NexusPhpSiteUserInfo {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.NexusProject;
|
||||
|
||||
protected override _parseSitePage(htmlText: string): void {
|
||||
const text = this._prepareHtmlText(htmlText);
|
||||
let userDetail = reSearch(/userdetails\.php\?id=(\d+)/, text);
|
||||
if (userDetail?.[0].trim()) {
|
||||
this._userDetailPage = userDetail[0].trim().replace(/^\/+/, "");
|
||||
this.userid = userDetail[1]!;
|
||||
} else {
|
||||
// 兼容部分 NexusProject 变种站点(如 star-space.net)的 p_user/user_detail.php?uid= 用户定位格式
|
||||
userDetail = reSearch(/user_detail\.php\?uid=(\d+)/, text);
|
||||
if (userDetail?.[0].trim()) {
|
||||
this._userDetailPage = userDetail[0].trim().replace(/^\/+/, "");
|
||||
this.userid = userDetail[1]!;
|
||||
const uname = reSearch(/user_detail\.php\?uid=\d+[^>]*?>\s*<[^>]*>([^<]+)</, text);
|
||||
if (uname) this.username = uname[1]!.trim();
|
||||
}
|
||||
}
|
||||
this._torrentSeedingPage = `viewusertorrents.php?id=${this.userid}&show=seeding`;
|
||||
}
|
||||
|
||||
protected override _parseUserTrafficInfo(htmlText: string): void {
|
||||
// 兼容 span#user_info / span#user_info_no_hover:"上传：445.11 G" / "下载：29.61 G"(单位无 B 后缀)
|
||||
try {
|
||||
const html = parseHtml(htmlText);
|
||||
if (html) {
|
||||
const bodyText = html
|
||||
.strings('//span[@id="user_info"]//text() | //span[@id="user_info_no_hover"]//text()')
|
||||
.join(" ");
|
||||
let m = reSearch(/上传[：:]\s*([\d.,]+)\s*([GMKT]?i?B?)/i, bodyText);
|
||||
if (m) this.upload = this.numFilesize(`${m[1]!.replaceAll(",", "")} ${m[2]!.toUpperCase()}B`);
|
||||
m = reSearch(/下载[：:]\s*([\d.,]+)\s*([GMKT]?i?B?)/i, bodyText);
|
||||
if (m) this.download = this.numFilesize(`${m[1]!.replaceAll(",", "")} ${m[2]!.toUpperCase()}B`);
|
||||
if (this.upload && this.download) this.ratio = round3(this.upload / this.download);
|
||||
if (this.upload || this.download) return;
|
||||
}
|
||||
} catch {
|
||||
// 与 MP 一致:吞掉异常回落到通用解析
|
||||
}
|
||||
super._parseUserTrafficInfo(htmlText);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
import {
|
||||
createMockFetch,
|
||||
NEXUS_PHP_LOGIN_PAGE,
|
||||
RABBIT_USER_DETAILS,
|
||||
RABBIT_USER_INDEX,
|
||||
RABBIT_USER_SEEDING_JSON,
|
||||
} from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { NexusRabbitSiteUserInfo } from "./nexus-rabbit.ts";
|
||||
import { runParser } from "./testing.ts";
|
||||
|
||||
describe("NexusRabbitSiteUserInfo(MP nexus_rabbit.py)", () => {
|
||||
it("首页奶糖余额 + 详情 layui-hares + api/general JSON 做种", async () => {
|
||||
const m = createMockFetch({
|
||||
"/": { body: RABBIT_USER_INDEX },
|
||||
"/user.php": { body: RABBIT_USER_DETAILS },
|
||||
"POST /api/general": { body: RABBIT_USER_SEEDING_JSON },
|
||||
});
|
||||
const info = await runParser(NexusRabbitSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info).toMatchObject({
|
||||
schema: "NexusRabbit",
|
||||
userId: "5",
|
||||
username: "dave",
|
||||
userLevel: "Power User",
|
||||
joinAt: "2022-01-02 03:04:05",
|
||||
upload: Math.round(1.5 * 1024 ** 4),
|
||||
download: 500 * 1024 ** 3,
|
||||
ratio: 3.072,
|
||||
bonus: 1024.5,
|
||||
seeding: 2,
|
||||
seedingSize: Math.round(3.5 * 1024 ** 3),
|
||||
seedingInfo: [
|
||||
[3, 1024 ** 3],
|
||||
[7, Math.round(2.5 * 1024 ** 3)],
|
||||
],
|
||||
});
|
||||
const api = m.requests.find((r) => r.url.pathname === "/api/general")!;
|
||||
expect(api.headers["x-requested-with"]).toBe("XMLHttpRequest");
|
||||
expect(JSON.parse(api.body!)).toMatchObject({ action: "userTorrentsList", data: { type: "seeding", id: 5 } });
|
||||
});
|
||||
|
||||
it("未登录:errMsg 且不请求 API", async () => {
|
||||
const m = createMockFetch({ "/": { body: NEXUS_PHP_LOGIN_PAGE } });
|
||||
const info = await runParser(NexusRabbitSiteUserInfo, { fetch: m.fetch });
|
||||
expect(info.errMsg).toContain("未检测到已登陆");
|
||||
expect(m.requests).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,142 @@
|
||||
/**
|
||||
* SPDX-License-Identifier: GPL-3.0-only
|
||||
* 移植自 MoviePilot(https://github.com/jxxghp/MoviePilot,GPLv3),本文件按 GPLv3 条款提供。
|
||||
* @mp-source app/modules/indexer/parser/nexus_rabbit.py @ e6820a8
|
||||
*
|
||||
* NexusRabbit(兔子/layui-hares 模板)站点用户信息解析器。
|
||||
*
|
||||
* 函数对照(`NexusRabbitSiteUserInfo`):`_parse_site_page` → `_parseSitePage`(做种/消息改走 `api/general` JSON)、
|
||||
* `_parse_user_torrent_seeding_info` → `_parseUserTorrentSeedingInfo`、`_parse_message_unread_links` →
|
||||
* `_parseMessageUnreadLinks`(直接读出消息并调用 readMessage 标记已读)、`_parse_user_base_info` → `_parseUserBaseInfo`
|
||||
* (奶糖余额)、`_parse_user_detail_info` → `_parseUserDetailInfo`、`_parse_message_content` / `_parse_user_traffic_info` → 空实现。
|
||||
*/
|
||||
import { type NextPage, SiteParserBase, urljoin } from "./base.ts";
|
||||
import { normalizeDatetime, parseFloatMp, pyInt, reSearch } from "./common.ts";
|
||||
import { hasChildElements, parseHtml } from "./dom.ts";
|
||||
import { SiteSchema, type SiteSchemaValue } from "./types.ts";
|
||||
|
||||
const JSON_HEADERS = {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "application/json, text/plain, */*",
|
||||
"X-Requested-With": "XMLHttpRequest",
|
||||
};
|
||||
|
||||
function removePrefix(s: string, p: string): string {
|
||||
return s.startsWith(p) ? s.slice(p.length) : s;
|
||||
}
|
||||
|
||||
export class NexusRabbitSiteUserInfo extends SiteParserBase {
|
||||
static override schema: SiteSchemaValue | null = SiteSchema.NexusRabbit;
|
||||
|
||||
protected _parseSitePage(htmlText: string): void {
|
||||
const text = this._prepareHtmlText(htmlText);
|
||||
const userDetail = reSearch(/user.php\?id=(\d+)/, text);
|
||||
if (!userDetail?.[0].trim()) return;
|
||||
this.userid = userDetail[1]!;
|
||||
this._userDetailPage = `user.php?id=${this.userid}`;
|
||||
this._userTrafficPage = null;
|
||||
this._torrentSeedingPage = "api/general";
|
||||
this._torrentSeedingParams = {
|
||||
page: 1,
|
||||
limit: 5000000,
|
||||
action: "userTorrentsList",
|
||||
data: { type: "seeding", id: pyInt(this.userid) },
|
||||
};
|
||||
// X-Requested-With 必须要加上,不然返回的是空数据
|
||||
this._torrentSeedingHeaders = { ...JSON_HEADERS };
|
||||
this._userMailUnreadPage = null;
|
||||
this._sysMailUnreadPage = "api/general";
|
||||
this._mailUnreadParams = { page: 1, limit: 5000000, action: "getMessageIn" };
|
||||
this._mailUnreadHeaders = { ...JSON_HEADERS };
|
||||
}
|
||||
|
||||
protected _parseUserTorrentSeedingInfo(htmlText: string): NextPage {
|
||||
let torrents: Array<Record<string, unknown>>;
|
||||
try {
|
||||
const data = (JSON.parse(htmlText) as { data?: unknown }).data;
|
||||
torrents = (data ?? []) as Array<Record<string, unknown>>;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
let seedingSize = 0;
|
||||
const seedingInfo: Array<[number, number]> = [];
|
||||
for (const t of torrents) {
|
||||
const seeders = pyInt(t.seeders ?? 0);
|
||||
const size = this.numFilesize(t.size as string);
|
||||
seedingSize += size;
|
||||
seedingInfo.push([seeders, size]);
|
||||
}
|
||||
this.seeding = torrents.length;
|
||||
this.seedingSize = seedingSize;
|
||||
this.seedingInfo = seedingInfo;
|
||||
return null;
|
||||
}
|
||||
|
||||
protected async _parseMessageUnreadLinks(htmlText: string, _msgLinks: string[]): Promise<NextPage> {
|
||||
const unreadIds: unknown[] = [];
|
||||
let messages: Array<Record<string, unknown>>;
|
||||
try {
|
||||
messages = ((JSON.parse(htmlText) as { data?: unknown }).data ?? []) as Array<Record<string, unknown>>;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
for (const msg of messages) {
|
||||
const id = msg.id;
|
||||
const unread = msg.unread;
|
||||
if (!(id && unread) || unread === "no") continue;
|
||||
unreadIds.push(id);
|
||||
const head = msg.subject as string | null;
|
||||
const date = msg.added as string | null;
|
||||
const content = msg.msg as string | null;
|
||||
if (head && date && content) this.messageUnreadContents.push([head, date, content]);
|
||||
}
|
||||
this.messageUnread = unreadIds.length;
|
||||
if (unreadIds.length) {
|
||||
await this._getPageContent(
|
||||
urljoin(this._baseUrl, "api/general?loading=true"),
|
||||
{ action: "readMessage", data: { ids: unreadIds } },
|
||||
{ ...JSON_HEADERS },
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** 只有奶糖余额需要在 base 中获取,其它均可以在详情页拿到 */
|
||||
protected _parseUserBaseInfo(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
const bonus = html.strings('//div[contains(text(), "奶糖余额")]/following-sibling::div[1]/text()');
|
||||
if (bonus.length) this.bonus = parseFloatMp(bonus[0]!.trim());
|
||||
}
|
||||
|
||||
protected _parseUserDetailInfo(htmlText: string): void {
|
||||
const html = parseHtml(htmlText);
|
||||
if (!hasChildElements(html)) return;
|
||||
const infos = html.nodes('//div[contains(@class, "layui-hares-user-info-right")]');
|
||||
if (!infos.length) return;
|
||||
const info = infos[0]!;
|
||||
const username = info.strings('.//span[contains(text(), "用户名")]/a/span/text()');
|
||||
if (username.length) this.username = username[0]!.trim();
|
||||
const level = info.strings('.//span[contains(text(), "等级")]/b/text()');
|
||||
if (level.length) this.userLevel = level[0]!.trim();
|
||||
const joinDate = info.strings('.//span[contains(text(), "注册日期")]/text()');
|
||||
if (joinDate.length) {
|
||||
const d = removePrefix(joinDate[0]!.trim().split("\r")[0]!, "注册日期：");
|
||||
this.joinAt = normalizeDatetime(d) ?? null;
|
||||
}
|
||||
const upload = info.strings('.//span[contains(text(), "上传量")]/text()');
|
||||
if (upload.length) this.upload = this.numFilesize(removePrefix(upload[0]!.trim(), "上传量："));
|
||||
const download = info.strings('.//span[contains(text(), "下载量")]/text()');
|
||||
if (download.length) this.download = this.numFilesize(removePrefix(download[0]!.trim(), "下载量："));
|
||||
const ratio = info.strings('.//span[contains(text(), "分享率")]/em/text()');
|
||||
if (ratio.length) this.ratio = parseFloatMp(ratio[0]!.trim());
|
||||
}
|
||||
|
||||
/** 已在 _parseMessageUnreadLinks 内实现 */
|
||||
protected _parseMessageContent(_htmlText: string): [string | null, string | null, string | null] {
|
||||
return [null, null, null];
|
||||
}
|
||||
|
||||
/** 已在 _parseUserDetailInfo 内实现 */
|
||||
protected _parseUserTrafficInfo(_htmlText: string): void {}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
import { createMockFetch, NEXUS_PHP_INDEX, NEXUS_PHP_LOGIN_PAGE, NEXUS_PHP_USERDETAILS } from "@umt/testkit";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { getUserParserClass, refreshSiteUserData, USER_PARSER_CLASSES } from "./registry.ts";
|
||||
import { SiteSchema } from "./types.ts";
|
||||
|
||||
describe("用户信息解析器注册表(MP refresh_userdata)", () => {
|
||||
it("覆盖 MP SiteSchema 全部 20 个值,且一一对应", () => {
|
||||
const schemas = USER_PARSER_CLASSES.map((c) => c.schema);
|
||||
expect(new Set(schemas)).toEqual(new Set(Object.values(SiteSchema)));
|
||||
expect(schemas).toHaveLength(20);
|
||||
expect(getUserParserClass("Small Horse")?.name).toBe("SmallHorseSiteUserInfo");
|
||||
expect(getUserParserClass("nope")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("未知 schema 返回 null", async () => {
|
||||
const m = createMockFetch({});
|
||||
expect(
|
||||
await refreshSiteUserData({ name: "x", url: "https://x.example.com/", schema: "Nope", fetch: m.fetch }),
|
||||
).toBeNull();
|
||||
expect(m.requests).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("声明 schema 解析不出 userid 时按序回退到能解析的 schema(Unit3d 标错 → NexusPhp)", async () => {
|
||||
const m = createMockFetch({
|
||||
"/": { body: NEXUS_PHP_INDEX },
|
||||
"/userdetails.php": { body: NEXUS_PHP_USERDETAILS },
|
||||
});
|
||||
const info = await refreshSiteUserData({
|
||||
name: "demo",
|
||||
url: "https://pt.example.com/",
|
||||
schema: SiteSchema.Unit3d,
|
||||
cookie: "c=1",
|
||||
fetch: m.fetch,
|
||||
});
|
||||
expect(info).toMatchObject({ userId: "10086", username: "alice", domain: "example.com" });
|
||||
expect(info!.schema).not.toBe("Unit3d");
|
||||
});
|
||||
|
||||
it("全部回退失败:返回声明 schema 的结果与 errMsg", async () => {
|
||||
const m = createMockFetch({ "/": { body: NEXUS_PHP_LOGIN_PAGE } });
|
||||
const info = await refreshSiteUserData({
|
||||
name: "demo",
|
||||
url: "https://pt.example.com/",
|
||||
schema: SiteSchema.NexusPhp,
|
||||
fetch: m.fetch,
|
||||
});
|
||||
expect(info).toMatchObject({ schema: "NexusPhp", userId: null });
|
||||
expect(info!.errMsg).toContain("未检测到已登陆");
|
||||
});
|
||||
});
|
||||
Loaded 100 of 166 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user