fix(security): W4-E1 approvals admin-session only, shared redact table, TMDB key redaction, prod mock gate

- POST /approvals and PATCH /approvals/{id} require a console admin session
  (EndpointDef auth:"session"); API keys incl. admin scope get 403, so an
  organize.execute key can no longer approve its own plan (S1)
- tests for single-use approval checks: expired, revision/payload hash
  mismatch, consumed, revoked, pending/rejected, plan mismatch; re-approve
  after plan change succeeds once and cannot be replayed (S4)
- one redact key table in @umt/contracts shared by server logger and agent
  redactForModel: rsskey, torrent_pass, authkey, downhash, c_secure_pass,
  passkey variants, x-api-key, cookie headers, nested objects/arrays (S2)
- TMDB network errors: message and cause redacted via redactUrl (S3)
- web Agent ?mock=1 only in DEV builds or when pi.provider is fake (S7)
- EndpointDef auth annotations for /auth/*, asserted against
  PUBLIC_AUTH_PATHS (S10); doc drift fixes (S11)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
rootandClaude Opus 5.5 committed 2026-09-27 22:24:44 +08:00
1 parent 79cdd26457
commit f48e1aac96
25 files changed
+825 -235

No files matched your search

+15 -1
View File
@@ -1,3 +1,17 @@
# @umt/adapters
外部系统适配层:qBittorrent、RSS、站点(观众 NexusPHP / 馒头 API)、TMDB、Emby 兼容服务器与文件系统。对上只暴露端口接口,凭证经 secret reference 注入,日志一律脱敏。
外部系统适配层:qBittorrent、RSS、站点(观众 NexusPHP / 馒头 API)、TMDB、Emby 兼容服务器、浏览器服务与文件系统。对上只暴露端口接口,凭证经 secret reference 注入,日志一律脱敏(键表与规则取 `@umt/contracts` 的 `redact.ts`)。
包入口 `src/index.ts` 逐个 `export *` 下列子模块:
| 子模块 | 主要入口 | 说明 |
|---|---|---|
| [`qbittorrent`](src/qbittorrent/README.md) | `createQbClient` / `QbClient`、`DownloaderRegistry`、`assertSafeToDeleteData`、`mapQbState` | qB WebUI v2:提交、标签接管、暂停/恢复、带安全校验的删数据 |
| [`rss`](src/rss/README.md) | `createRssFetcher`、`applyFeedItems`、`redactUrl`、`buildRssLinkRequest` | RSS 2.0 / RDF / Atom 抓取解析、条件请求、退避熔断、去重 |
| [`sites`](src/sites/README.md) | `createSiteAdapter` / `getSiteAdapter`、`listSiteDefinitions`、`siteFetch` | 观众(NexusPHP 配置驱动 spider)与馒头 M-Team API |
| [`tmdb`](src/tmdb/README.md) | `createTmdbClient` / `TmdbClient`、`matchTmdb`、`matchTmdbMulti`、`TmdbError` | 多语言搜索、详情、季集;缓存限流;错误信息不带 api_key |
| [`emby`](src/emby/README.md) | `createEmbyClient` / `EmbyClient`、`MediaServerRegistry` | Emby / Jellyfin 库存查询,失败返回 `unknown` 而不是"不存在" |
| [`fs`](src/fs/README.md) | `copyWithVerify`、`moveSafe`、`crossDeviceMove`、`checkRoot`、`scanIncremental`、`diskUsage`、`ffprobe` | 校验复制、跨卷移动、库根哨兵、增量扫描 |
| [`browser`](src/browser/README.md) | `BrowserProvider` 合同、`createNoneBrowserProvider`、camofox 适配骨架、`redactBrowserUrl` | 浏览器服务接口(M1 只有接口与骨架) |
测试:`deno task test`(vitest,外部服务全部用 `@umt/testkit` 的 mock)。
+16 -2
View File
@@ -7,6 +7,7 @@
* match/match_multi 的匹配链见 match.ts。
* 调整:网络/鉴权/限流错误以 TmdbError 抛出而非返回 None,调用方不得把"查询失败"当成"没有结果"。
*/
import { redactUrl } from "@umt/contracts";
import { RequestLimiter, TtlLruCache } from "./cache.ts";
export type TmdbErrorKind = "auth" | "network" | "timeout" | "notfound" | "rate" | "protocol";
@@ -122,6 +123,11 @@ export class TmdbClient {
this.fetchImpl = cfg.fetch ?? fetch;
}
/** 错误信息里去掉 api_key / access token(query 参数与明文两种形态都遮) */
private redact(text: string): string {
return redactUrl(text, [this.cfg.apiKey, this.cfg.accessToken]);
}
cacheStats() {
return { size: this.cache.size, hits: this.cache.hits, misses: this.cache.misses, coalesced: this.cache.coalesced };
}
@@ -173,7 +179,12 @@ export class TmdbClient {
const name = (e as { name?: string })?.name;
if (name === "TimeoutError" || name === "AbortError")
throw new TmdbError("timeout", "TMDB request timed out", { path });
throw new TmdbError("network", `TMDB request failed: ${(e as Error)?.message}`, { path, cause: e });
// Deno fetch 的错误信息带完整 URL(含 api_key):message 先脱敏,cause 只留脱敏后的名字与信息,不透传原对象
const msg = this.redact(String((e as Error)?.message ?? e));
throw new TmdbError("network", `TMDB request failed: ${msg}`, {
path,
cause: { name: (e as Error)?.name ?? "Error", message: msg },
});
}
if (res.status === 401) {
await res.body?.cancel();
@@ -199,7 +210,10 @@ export class TmdbClient {
try {
return await res.json();
} catch (e) {
throw new TmdbError("protocol", "TMDB returned invalid JSON", { path, cause: e });
throw new TmdbError("protocol", "TMDB returned invalid JSON", {
path,
cause: { name: (e as Error)?.name ?? "Error", message: this.redact(String((e as Error)?.message ?? e)) },
});
}
}
+31
View File
@@ -142,6 +142,37 @@ describe("TmdbClient 缓存/限流/错误分类", () => {
expect((await dead.details("movie", 1).catch((e) => e)).kind).toBe("network");
});
it("网络/HTTP 错误的 message 与 cause 不含 api_key(验收 S3)", async () => {
const KEY = "tmdbSecretKey0123456789abcdef";
// 真实连接失败:Deno fetch 的错误信息带完整 URL
const dead = createTmdbClient({ baseUrl: "http://127.0.0.1:1/3", apiKey: KEY, retries: 0 });
const e1 = await dead.details("movie", 1).catch((x) => x);
expect(e1.kind).toBe("network");
// 模拟把 URL(含 api_key 参数与编码后的明文)拼进信息的 fetch 实现
const leaky: typeof fetch = async (input) => {
throw new TypeError(`error sending request for url (${String(input)}): key ${encodeURIComponent(KEY)}`);
};
const e2 = await createTmdbClient({ baseUrl: "http://tmdb.local/3", apiKey: KEY, retries: 0, fetch: leaky })
.details("movie", 1)
.catch((x) => x);
expect(e2.kind).toBe("network");
expect(e2.message).toContain("api_key=[REDACTED]");
// HTTP 错误
const { mock } = await setup();
mock.fault("/3/movie/872585", { status: 503 });
const e3 = await createTmdbClient({ baseUrl: `${mock.url}/3`, apiKey: KEY, retries: 0 })
.details("movie", 872585)
.catch((x) => x);
expect(e3.kind).toBe("protocol");
for (const e of [e1, e2, e3]) {
expect(e).toBeInstanceOf(TmdbError);
expect(e.message).not.toContain(KEY);
expect(e.stack ?? "").not.toContain(KEY);
expect(JSON.stringify(e.detail)).not.toContain(KEY);
expect(e.detail.cause instanceof Error).toBe(false);
}
});
it("Bearer token 鉴权可用", async () => {
mock = await createMockTmdb({ accessToken: "v4-token" });
const tmdb = createTmdbClient({ baseUrl: `${mock.url}/3`, accessToken: "v4-token" });
+4 -42
View File
@@ -1,50 +1,12 @@
/**
* 敏感信息守卫:工具输入输出进入模型上下文、事件流、审计前统一经过这里。
* 覆盖 cookie / passkey / token / apiKey / authorization / 含 passkey 等私密参数的 URL / URL 内嵌账号密码。
* 键表见 @umt/contracts 的 redact.ts(cookie / passkey / rsskey / torrent_pass / authkey / downhash / c_secure_pass / token / apiKey …)。
* 宁可多遮不可漏遮:误伤只影响可读性,漏遮会把凭证送给外部模型。
*/
export const REDACTED = "[REDACTED]";
import { isSensitiveKey, REDACTED, redactString } from "@umt/contracts";
const SENSITIVE_KEY_PARTS = [
"cookie",
"passkey",
"authkey",
"torrentpass",
"rsskey",
"apikey",
"authorization",
"secret",
"password",
"passwd",
"credential",
"privatekey",
];
/** 规范化键名(去掉 - _ 并小写)后判断。以 token 结尾算敏感,tokens(用量统计)不算。 */
export function isSensitiveKey(key: string): boolean {
const k = key.toLowerCase().replace(/[-_\s]/g, "");
if (k.endsWith("token")) return true;
if (k === "sign" || k === "signature") return true;
return SENSITIVE_KEY_PARTS.some((part) => k.includes(part));
}
/** URL 查询参数/片段中视为凭证的参数名 */
const SENSITIVE_PARAM =
"passkey|authkey|torrent_pass|rsskey|api[_-]?key|access[_-]?token|refresh[_-]?token|token|auth|sign|signature|credential|secret|password|key|uid_key|downhash";
const PARAM_RE = new RegExp(`([?&;#]|\\b)(${SENSITIVE_PARAM})=([^&;#\\s"'<>]+)`, "gi");
const URL_USERINFO_RE = /\b([a-z][a-z0-9+.-]*:\/\/)([^/\s:@]+):([^/\s@]+)@/gi;
const BEARER_RE = /\b(Bearer|Basic|Token)\s+[A-Za-z0-9._~+/=-]{6,}/g;
const HEADER_RE = /\b(cookie|set-cookie|authorization|x-api-key)\s*:\s*[^\r\n]+/gi;
const KNOWN_KEY_RE = /\b(sk-[A-Za-z0-9_-]{8,}|sk-ant-[A-Za-z0-9_-]{8,}|gh[pousr]_[A-Za-z0-9]{16,})\b/g;
export function redactString(input: string): string {
return input
.replace(HEADER_RE, (_m, name: string) => `${name}: ${REDACTED}`)
.replace(URL_USERINFO_RE, (_m, scheme: string) => `${scheme}${REDACTED}@`)
.replace(PARAM_RE, (_m, lead: string, name: string) => `${lead}${name}=${REDACTED}`)
.replace(BEARER_RE, (_m, scheme: string) => `${scheme} ${REDACTED}`)
.replace(KNOWN_KEY_RE, REDACTED);
}
// 键表与字符串规则只保留一份,在 @umt/contracts(redact.ts);服务端 logger 与这里共用。
export { isSensitiveKey, REDACTED, redactString };
/** 深度脱敏,返回新对象;不修改入参。循环引用替换为 "[Circular]"。 */
export function redactForModel<T>(value: T): T {
+22
View File
@@ -74,3 +74,25 @@ describe("buildSystemPrompt", () => {
expect(p).toContain("- 只给方案,不执行");
});
});
describe("redactForModel:站点凭证键全集(与服务端 logger 同一键表)", () => {
it("rsskey / torrent_pass / authkey / downhash / c_secure_pass / passkey 变体 / x-api-key / cookie", () => {
const input = {
rsskey: "S1",
torrent_pass: "S2",
authKey: "S3",
downhash: "S4",
c_secure_pass: "S5",
"Pass-Key": "S6",
"x-api-key": "S7",
Cookie: "S8",
nested: [{ pass_key: "S9" }],
url: "https://x/rss?rsskey=S10&torrent_pass=S11&authkey=S12&downhash=S13&c_secure_pass=S14&PassKey=S15",
usage: { totalTokens: 10 },
};
const out = JSON.stringify(redactForModel(input));
expect(out).not.toMatch(/S\d+"/);
expect(out).not.toMatch(/=S\d+/);
expect(out).toContain('"totalTokens":10');
});
});
+29 -4
View File
@@ -23,10 +23,18 @@ export const API_PREFIX = "/api/v1";
export type HttpMethod = "get" | "post" | "patch" | "put" | "delete";
/**
* 鉴权方式(不写即按 scope 校验,cookie 会话与 API Key 同样适用):
* - public:免鉴权(必须与服务端 http/auth.ts 的 PUBLIC_AUTH_PATHS 一致,测试断言);
* - session:只接受控制台管理员 cookie 会话,API Key(含 admin scope)一律 403。
*/
export type EndpointAuth = "public" | "session";
export interface EndpointDef {
method: HttpMethod;
path: string;
scope: ApiScope;
auth?: EndpointAuth;
summary: string;
params?: z.ZodObject;
query?: z.ZodType;
@@ -793,16 +801,27 @@ export const Api = {
"审批",
{
row: agent.Approval,
update: agent.ApprovalUpdate,
filters: { status: agent.ApprovalStatus.optional(), planId: Id.optional() },
},
{ read: "read", write: "organize.execute" },
R,
),
// 审批的申请与决定只在控制台管理员会话里做:API Key(哪怕持 organize.execute)不能给自己的计划开审批(验收 S1)
update: ep({
method: "patch",
path: "/approvals/{id}",
scope: "admin",
auth: "session",
summary: "批准/拒绝/撤销单次审批(只接受控制台管理员会话)",
params: IdParam,
body: agent.ApprovalUpdate,
response: agent.Approval,
}),
create: ep({
method: "post",
path: "/approvals",
scope: "organize.execute",
summary: "为计划申请单次审批(绑定 plan id + payload hash + revision + 有效期)",
scope: "admin",
auth: "session",
summary: "为计划申请单次审批(绑定 plan id + payload hash + revision + 有效期;只接受控制台管理员会话)",
body: agent.ApprovalRequest,
response: agent.Approval,
}),
@@ -1040,6 +1059,7 @@ export const Api = {
method: "post",
path: "/auth/setup",
scope: "read",
auth: "public",
summary: "首次初始化管理员",
body: auth.AdminSetupRequest,
response: auth.SessionInfo,
@@ -1048,6 +1068,7 @@ export const Api = {
method: "post",
path: "/auth/login",
scope: "read",
auth: "public",
summary: "管理员登录",
body: auth.LoginRequest,
response: auth.LoginResult,
@@ -1057,6 +1078,7 @@ export const Api = {
method: "get",
path: "/auth/session",
scope: "read",
auth: "public",
summary: "当前登录态(免鉴权)",
response: auth.SessionInfo,
}),
@@ -1064,6 +1086,7 @@ export const Api = {
method: "post",
path: "/auth/password",
scope: "admin",
auth: "session",
summary: "修改密码(吊销其它会话)",
body: auth.PasswordChangeRequest,
response: auth.PasswordChangeResult,
@@ -1072,6 +1095,7 @@ export const Api = {
method: "get",
path: "/auth/totp",
scope: "admin",
auth: "session",
summary: "TOTP 状态",
response: auth.TotpStatus,
}),
@@ -1079,6 +1103,7 @@ export const Api = {
method: "post",
path: "/auth/totp",
scope: "admin",
auth: "session",
summary: "TOTP 绑定/启用/停用",
body: auth.TotpAction,
response: auth.TotpActionResult,
+1
View File
@@ -15,5 +15,6 @@ export * from "./instances.ts";
export * from "./ops.ts";
export * from "./organization.ts";
export * from "./policy.ts";
export * from "./redact.ts";
export * from "./settings.ts";
export * from "./sources.ts";
+87
View File
@@ -0,0 +1,87 @@
/**
* 凭证脱敏的唯一键表与字符串规则(服务端 logger、Agent 模型上下文、适配器错误信息共用)。
* 覆盖 cookie / passkey / rsskey / torrent_pass / authkey / downhash / c_secure_pass / token / apiKey /
* authorization / x-api-key 头 / URL 内嵌账号密码。键名先规范化(小写、去掉 - _ 空白)再判断,
* 所以 PassKey、pass-key、pass_key、X-Api-Key 都能命中。
* 宁可多遮不可漏遮:误伤只影响可读性,漏遮会把凭证送进日志或外部模型。
*/
export const REDACTED = "[REDACTED]";
/** 规范化后的键名只要包含其中一项即视为敏感 */
export const SENSITIVE_KEY_PARTS: readonly string[] = [
"cookie",
"passkey",
"authkey",
"torrentpass",
"securepass",
"rsskey",
"downhash",
"apikey",
"authorization",
"secret",
"password",
"passwd",
"credential",
"privatekey",
];
/** 规范化键名(去掉 - _ 空白并小写)后判断。以 token 结尾算敏感,tokens(用量统计)不算。 */
export function isSensitiveKey(key: string): boolean {
const k = key.toLowerCase().replace(/[-_\s]/g, "");
if (k.endsWith("token")) return true;
if (k === "sign" || k === "signature") return true;
return SENSITIVE_KEY_PARTS.some((part) => k.includes(part));
}
/** URL 查询参数/片段中视为凭证的参数名(正则片段,大小写不敏感;- _ 可选的写成 [_-]?) */
export const SENSITIVE_QUERY_PARAMS: readonly string[] = [
"pass[_-]?key",
"auth[_-]?key",
"torrent[_-]?pass",
"rss[_-]?key",
"c[_-]?secure[_-]?pass",
"down[_-]?hash",
"api[_-]?key",
"access[_-]?token",
"refresh[_-]?token",
"token",
"auth",
"sign",
"signature",
"credential",
"secret",
"password",
"uid[_-]?key",
"key",
];
const PARAM_RE = new RegExp(`([?&;#]|\\b)(${SENSITIVE_QUERY_PARAMS.join("|")})=([^&;#\\s"'<>]+)`, "gi");
const URL_USERINFO_RE = /\b([a-z][a-z0-9+.-]*:\/\/)([^/\s:@]+):([^/\s@]+)@/gi;
const BEARER_RE = /\b(Bearer|Basic|Token)\s+[A-Za-z0-9._~+/=-]{6,}/g;
const HEADER_RE = /\b(cookie|set-cookie|authorization|x-api-key)\s*:\s*[^\r\n]+/gi;
const KNOWN_KEY_RE = /\b(sk-[A-Za-z0-9_-]{8,}|sk-ant-[A-Za-z0-9_-]{8,}|gh[pousr]_[A-Za-z0-9]{16,})\b/g;
/** 文本脱敏:请求头行、URL userinfo、query 凭证参数、Bearer/Basic、已知格式的密钥 */
export function redactString(input: string): string {
return input
.replace(HEADER_RE, (_m, name: string) => `${name}: ${REDACTED}`)
.replace(URL_USERINFO_RE, (_m, scheme: string) => `${scheme}${REDACTED}@`)
.replace(PARAM_RE, (_m, lead: string, name: string) => `${lead}${name}=${REDACTED}`)
.replace(BEARER_RE, (_m, scheme: string) => `${scheme} ${REDACTED}`)
.replace(KNOWN_KEY_RE, REDACTED);
}
/**
* URL(或含 URL 的错误信息)脱敏。`secrets` 为调用方已知的明文凭证,出现在任何位置都会被替换
* (兜底 fetch 实现把 URL 编码后再拼进错误信息之类的情况)。
*/
export function redactUrl(input: string, secrets: readonly (string | undefined | null)[] = []): string {
let out = redactString(input);
for (const s of secrets) {
if (!s || s.length < 4) continue;
out = out.split(s).join(REDACTED);
const enc = encodeURIComponent(s);
if (enc !== s) out = out.split(enc).join(REDACTED);
}
return out;
}
+11 -1
View File
@@ -1,6 +1,16 @@
# @umt/testkit
测试夹具与 mock 服务:mock qB WebUI、mock NexusPHP 站点、mock M-Team API、mock RSS、mock TMDB 与临时目录夹具。只供测试依赖,不进入生产构建。
测试夹具与 mock 服务,只供测试依赖,不进入生产构建:
| 入口 | 模拟对象 |
|---|---|
| `createMockQb` | qBittorrent WebUI v2(见下文"落盘") |
| `createMockNexusPhpSite`、`createMockMteamApi`、`createMockFetch` | 观众(NexusPHP)站点页面、馒头 M-Team API(见 `src/sites/README.md`) |
| `createMockRssServer` | RSS / Atom Feed(见 `src/rss/README.md`) |
| `createMockTmdb` | TMDB v3(api_key 与 Bearer 两种鉴权,`fault()` 注入 429/5xx/延迟) |
| `createMockEmby` | Emby / Jellyfin(`flavor`),`/Users`、`/Items`、`/Shows/{id}/Seasons|Episodes`,夹具 `EMBY_FIXTURE_ITEMS` |
| `createMockCamofox` | camofox-browser(tabs / navigate / snapshot / screenshot,可选 accessKey;快照里故意带密码、Cookie、passkey 供脱敏测试) |
| `createTempTree` | 临时目录树(可写库根哨兵) |
## mock qB 落盘