fix(sites): HHClub parsing, resource list layout and request restraint

HHClub (憨憨) was checked read-only against a real account (one detail
request; list, user data and health came from the product's own fetch).

Parsing fixes:
- list titles carried the site's new-torrent marker "[新]" (58 of 100
  rows; the detail page title has none); stripped with a replace filter
- refreshing a detail overwrote seeders / leechers / snatched / category
  name / H&R with null because the HHClub div detail page lacks them;
  detail values now fall back to the list snapshot (mergeDetailOverList),
  and an unknown detail promotion keeps the list promotion
- level "贵宾" is VIP (umt.vipLevels)
- sanitized minimal fixture HHANCLUB_LIST_PAGE_2026 + tests; README marks
  HHClub as verified 2026-09-29
Size, counts, category ids/names, +08:00 time and subtitle were correct.

Request restraint:
- site definitions may carry umt.defaults; merged with settings.sites
  (longer intervals, fewer pages): new settings defaultSnapshotMaxPages 1,
  defaultHealthCheckIntervalSec 6 h, defaultDetailCacheSec 24 h; default
  burst 2. HHClub: snapshot 60 min page 1 only, health 6 h, search
  concurrency 1, detail cache 24 h, 1 token / 3 s burst 2
- periodic / manual snapshots are capped to that page count; scheduled
  health checks skip sites checked within the interval; detail refresh is
  served from cache inside the window; live searches on one site queue
- every rate-limiter token is counted per site per hour
  (site_request_count, migration 0046); GET /sites/{id}/request-stats;
  site overview and resource list show requests in the last 24 hours

Resource list: two-layer rows. Main line: title + spec tags (episodes,
resolution, source, codec, HDR, group, 国语/中字, site tags) + promotion
with time left + H&R. Second line: subtitle (one line, ellipsis) and
fixed-width category / size / seed-leech-snatched / upload time
(relative, absolute on hover) columns, stale badge, inventory overlay;
folds into a card on phones.

Also aligns identity_candidate.normalized_title order in db types with
the migrations (schema drift test).

Takes effect after the stack is rebuilt (migration 0046).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
rootandClaude Opus 5.5 committed 2026-09-29 15:18:15 +08:00
1 parent 20b63db254
commit 901b01d1a1
43 files changed
+1222 -1315

No files matched your search

@@ -0,0 +1,25 @@
import type { Kysely } from "kysely";
import { dropTables, execDdl } from "../src/migration-helpers.ts";
// 站点请求计数:每次从站点限流器拿到令牌(列表 / 搜索 / 详情 / 下载 / 登录 / 用户数据 / 原站视图)记一次,
// 按小时聚合,api / worker 多进程共用。站点页展示"最近 24 小时请求次数";保留 7 天,写入时顺手清理。
// biome-ignore lint/suspicious/noExplicitAny: 迁移不依赖应用表类型
export async function up(db: Kysely<any>): Promise<void> {
await execDdl(
db,
`
create table site_request_count (
site_id uuid not null references site(id) on delete cascade,
hour timestamptz not null,
count integer not null default 0 check (count >= 0),
primary key (site_id, hour)
);
`,
);
}
// biome-ignore lint/suspicious/noExplicitAny: 同上
export async function down(db: Kysely<any>): Promise<void> {
await execDdl(db, dropTables(["site_request_count"]));
}
+11 -1
View File
@@ -788,7 +788,6 @@ export interface IdentityCandidateTable {
status: Generated<IdentityCandidateStatus>;
reason: IdentityCandidateReason;
query: Generated<Json>;
normalized_title: string | null;
candidates: Generated<Json>;
evidence_version: string;
attempts: Generated<number>;
@@ -800,6 +799,7 @@ export interface IdentityCandidateTable {
resolved_at: Timestamptz | null;
created_at: GeneratedTimestamptz;
updated_at: GeneratedTimestamptz;
normalized_title: string | null;
}
export interface IdentityRedirectTable {
@@ -1385,6 +1385,12 @@ export interface SiteAccountTable {
user_data_at: Timestamptz | null;
}
export interface SiteRequestCountTable {
site_id: string;
hour: Timestamptz;
count: Generated<number>;
}
export interface SiteSnapshotTable {
id: Generated<string>;
site_id: string;
@@ -1691,6 +1697,7 @@ export interface DB {
settings: SettingsTable;
site: SiteTable;
site_account: SiteAccountTable;
site_request_count: SiteRequestCountTable;
site_snapshot: SiteSnapshotTable;
site_user_data: SiteUserDataTable;
slot_deferral: SlotDeferralTable;
@@ -1903,6 +1910,9 @@ export type SiteUpdate = Updateable<SiteTable>;
export type SiteAccountRow = Selectable<SiteAccountTable>;
export type NewSiteAccount = Insertable<SiteAccountTable>;
export type SiteAccountUpdate = Updateable<SiteAccountTable>;
export type SiteRequestCountRow = Selectable<SiteRequestCountTable>;
export type NewSiteRequestCount = Insertable<SiteRequestCountTable>;
export type SiteRequestCountUpdate = Updateable<SiteRequestCountTable>;
export type SiteSnapshotRow = Selectable<SiteSnapshotTable>;
export type NewSiteSnapshot = Insertable<SiteSnapshotTable>;
export type SiteSnapshotUpdate = Updateable<SiteSnapshotTable>;
+4
View File
@@ -119,6 +119,8 @@ const EXPECTED_TABLES = [
"rate_limit_bucket",
// C2-A PG 短期缓存(0042,UNLOGGED)
"kv_cache",
// 站点请求计数(0046,按小时聚合)
"site_request_count",
].sort();
/** 不走 entityTable() 的表:追加型/引擎自有表,没有 uuid id + created_at/updated_at 三件套 */
@@ -147,6 +149,8 @@ const NON_ENTITY_TABLES = [
"rate_limit_bucket",
// 缓存:(region, key) 主键
"kv_cache",
// 站点请求计数:(site_id, hour) 主键
"site_request_count",
];
describe.skipIf(skip)("schema migrations (real PG)", () => {