feat(server): W2-B sources, identity resolution, metadata and site workbench
- Sites / accounts / feeds: registry view (/site-definitions), site CRUD with
enable/disable (history stays queryable, marked site_disabled), accounts and
feed URLs stored as secrets (only secret_ref / redacted urlDisplay exposed),
feed enable/disable/reset/fetch; one shared rate limiter + adapter per site.
- Tasks: ingestFeed (FeedFetchState/FeedCursor on feed_source, observation
feed+guid upsert with digest-change updates, source_release site+id
aggregation across feeds, auth failure -> account invalid + decision, no
retry, gap -> feed.gap_suspected -> budgeted backfill snapshot),
resolveObservation (parseRelease -> local recall via title_alias pg_trgm +
accepted identities -> TMDB -> resolveIdentity; auto binds entity /
provisional units / identities / aliases, otherwise identity_candidate pool
with evidence version and bounded review backoff; manual lock respected),
refreshMetadata (metadata_field + field_override, user locks win, paths
untouched), snapshotSite (site_snapshot coverage records).
- Events: feed.due, feed.gap_suspected, release.observed (self-consumed),
release.resolved, decision.created/updated; scheduler in worker start().
- Identity API: entity detail with redirects, manual mapping correction with
identity_revision, merge with identity_redirect, pool list/assign/dismiss/
retry, content unit numbering lock, aliases, external identities.
- Decisions: list/summary (by entity and kind)/detail/resolve; identity
options executed directly.
- Workbench: /sites/{id}/releases (snapshot first, coalesced live search with
backoff, origin + freshness) and /sites/{id}/releases/{rid} (volatile field
recheck times, refresh keeps data on failure).
- Capabilities: catalog.lookup, release.search, release.inspect,
identity.resolve.
- Migrations 0018-0020: site.adapter api, source_release.resolution_locked,
site_snapshot, identity_candidate, resolution_attempt, identity_redirect,
metadata_field, field_override.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
829ca4f7ac
commit
a24bda952b
39 files changed
+6861
-15
No files matched your search
@@ -29,6 +29,9 @@ PostgreSQL 访问层:Kysely 客户端与连接池、表类型(`src/types.ts`)、
|
||||
| `0012_organization` | organization_record(change_seq 序列 + 触发器)、organization_file_result;回填 asset_operation.organization_record_id |
|
||||
| `0013_tasks_tables` | 并入 @umt/tasks 自有表:tasks_run_meta、tasks_fact、tasks_op_ledger(if not exists,可接管 W1 旧库) |
|
||||
| `0014_agent_run_event` | agent_run_event:Agent 运行事件,主键 (run_id, seq),删 run 级联 |
|
||||
| `0018_sources_snapshot` | site.adapter 增加 `api`;source_release.resolution_locked(人工锁);site_snapshot(快照/补漏/实时补查的覆盖范围) |
|
||||
| `0019_identity_pool` | identity_candidate(待识别池)、resolution_attempt(识别尝试)、identity_redirect(合并跳转) |
|
||||
| `0020_metadata_fields` | metadata_field(来源候选字段)、field_override(用户覆盖与锁定) |
|
||||
|
||||
## 表清单(45 张)
|
||||
|
||||
@@ -59,6 +62,11 @@ PostgreSQL 访问层:Kysely 客户端与连接池、表类型(`src/types.ts`)、
|
||||
| `title_alias` | 原名/译名/简称/站点原标题;`normalized` 上建 gin trgm 索引做模糊召回 |
|
||||
| `external_identity` | 本地对象 ↔ 外部 ID;**`(provider, entity_type, external_id)` 对 accepted 部分唯一**,同一本地对象每个 provider+type 只有一个 accepted;候选(proposed/conflicted)可并存;404/超时只记 `verify_status` |
|
||||
| `identity_revision` | 身份每次变更的修订历史,`(external_identity_id, revision)` 唯一,只追加 |
|
||||
| `identity_candidate` | 待识别池:每条 source_release 至多一条;候选只存 JSON(不建正式作品),保存原因/证据版本/复查时间(有界退避) |
|
||||
| `resolution_attempt` | 每次识别尝试的结果与证据版本,只追加 |
|
||||
| `identity_redirect` | 作品合并后的旧 ID 跳转,`from_entity_id` 唯一 |
|
||||
| `metadata_field` | 各来源的候选字段值(来源/语言/时间/来源修订);刷新只写这里 |
|
||||
| `field_override` | 用户字段覆盖与锁定,优先于自动来源;目标 + 字段唯一 |
|
||||
|
||||
### 来源
|
||||
| 表 | 用途与关键不变量 |
|
||||
@@ -68,6 +76,7 @@ PostgreSQL 访问层:Kysely 客户端与连接池、表类型(`src/types.ts`)、
|
||||
| `feed_source` | RSS/列表/查询入口;真实 URL(含 passkey)只在 `url_secret_ref`,对外只给 `url_display`;按 `next_fetch_at` 调度 |
|
||||
| `source_release` | 站点资源;**`(site_id, site_release_id)` 唯一**,多 Feed 观察汇到同一行;`source_published_at`/`first_seen_at` 分开 |
|
||||
| `observation` | 一次原始观察;**`(feed_id, guid)` 唯一**,`content_digest` 变化即条目更新 |
|
||||
| `site_snapshot` | 一次列表快照 / 补漏 / 实时补查:页码、条目数、状态与错误(新鲜度来源) |
|
||||
|
||||
### 文件
|
||||
| 表 | 用途与关键不变量 |
|
||||
|
||||
Reference in new issue
Block a user