fix(tasks): align OutboxRelay with outbox_event status/next_attempt_at/idempotency_key

The relay only looked at published_at and ignored the ADR-002 §2.1 columns
added in db 0002_foundation: rows stayed status='pending' forever after
dispatch, failures were retried every tick without backoff, and the
row-level idempotency_key was never used as the engine enqueue key.

- claim status='pending' AND next_attempt_at<=now()
- enqueue with idempotency_key (fallback outbox:<id>)
- success: status=dispatched, published_at, attempts+1
- failure: attempts+1, exponential backoff (cap 5 min), last_error; dead at maxAttempts
- test helper table now mirrors db 0001+0002; tests cover backoff and dead

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
rootandClaude Fable 5.1 committed 2026-09-27 18:59:06 +08:00
1 parent 608e567edf
commit 31729f2bac
4 files changed
+56 -15

No files matched your search

+20 -8
View File
@@ -31,12 +31,16 @@ export interface OutboxRelayOptions {
routes: Record<string, OutboxRouteFn>;
batchSize?: number;
intervalMs?: number;
/** 失败达到该次数后置 status='dead'(默认 10);其间按 2^attempts 秒退避,上限 5 分钟 */
maxAttempts?: number;
logSink?: LogSink;
}
/**
* Outbox 中继:业务事务写 outbox_event,中继在自己的事务里
* `SELECT … FOR UPDATE SKIP LOCKED` 认领未投递行 → enqueue(幂等键 = outbox:<行 id>)→ 标记 published_at。
* `SELECT … FOR UPDATE SKIP LOCKED` 认领 status='pending' 且 next_attempt_at 已到的行
* → enqueue(幂等键 = 行的 idempotency_key,缺省 outbox:<行 id>)→ 标记 dispatched + published_at;
* 失败则 attempts+1、退避 next_attempt_at、记 last_error,超过 maxAttempts 置 dead(ADR-002 §2.1)。
* 多个中继进程并发时互不重复认领;enqueue 成功但标记前崩溃,重投时由引擎幂等键去重(24h),
* 更长的去重由业务唯一约束兜底。
*/
@@ -61,9 +65,9 @@ export class OutboxRelay {
let published = 0;
let failed = 0;
await this.opts.db.transaction().execute(async (trx) => {
const rows = await sql<OutboxEvent>`
SELECT id, type, payload, at FROM outbox_event
WHERE published_at IS NULL AND type = ANY(${types}::text[])
const rows = await sql<OutboxEvent & { idempotency_key: string | null; attempts: number }>`
SELECT id, type, payload, at, idempotency_key, attempts FROM outbox_event
WHERE status = 'pending' AND next_attempt_at <= now() AND type = ANY(${types}::text[])
ORDER BY at, id
LIMIT ${this.opts.batchSize ?? 50}
FOR UPDATE SKIP LOCKED`.execute(trx);
@@ -73,17 +77,25 @@ export class OutboxRelay {
if (route) {
const payloadTrace = (ev.payload as { traceId?: unknown } | null)?.traceId;
await this.opts.engine.enqueue(route.task, route.input, {
idempotencyKey: OutboxRelay.idempotencyKey(ev.id),
idempotencyKey: ev.idempotency_key ?? OutboxRelay.idempotencyKey(ev.id),
traceId: route.traceId ?? (typeof payloadTrace === "string" ? payloadTrace : randomUUID()),
...(route.priority === undefined ? {} : { priority: route.priority }),
});
}
await sql`UPDATE outbox_event SET published_at = now() WHERE id = ${ev.id}`.execute(trx);
await sql`UPDATE outbox_event SET status = 'dispatched', published_at = now(), attempts = attempts + 1,
last_error = NULL WHERE id = ${ev.id}`.execute(trx);
published++;
} catch (err) {
failed++;
// 留在未投递状态,下一轮重试
this.log.error("outbox dispatch failed", { eventId: ev.id, type: ev.type, err });
// 退避后重试;超过上限置 dead(需人工处理)
const attempts = ev.attempts + 1;
const dead = attempts >= (this.opts.maxAttempts ?? 10);
const delayS = Math.min(2 ** attempts, 300);
const lastError = JSON.stringify({ message: err instanceof Error ? err.message : String(err) });
await sql`UPDATE outbox_event SET attempts = ${attempts}, status = ${dead ? "dead" : "pending"},
next_attempt_at = now() + make_interval(secs => ${delayS}), last_error = ${lastError}::jsonb
WHERE id = ${ev.id}`.execute(trx);
this.log.error("outbox dispatch failed", { eventId: ev.id, type: ev.type, attempts, dead, err });
}
}
});