fix(system): retired heartbeat rows no longer alarm on status page
- GET /system/health splits processes into current instances (hosts with
a beat in the last 10 min, newest per host+kind, not stopped) and
retiredProcesses[]; 45s stale and level only consider current ones
- heartbeat writer marks stopped_at on graceful exit (migration 0043),
upsert clears it; pure api role now stops its writer on SIGTERM
- scheduled job system.heartbeat_cleanup (maintain, 1h) deletes rows
without a beat for 24h and samples older than 7 days
- status page: per-role alarms ("io worker 45 秒没有心跳"), collapsible
retired instances; glossary/runbook updated
- bump all members to 0.2.2, CHANGELOG [0.2.2]
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
fa261eb6cd
commit
b4c9b1ead2
32 files changed
+747
-91
No files matched your search
@@ -381,8 +381,10 @@ export const SystemProcess = z.object({
|
||||
lastBeatAt: Timestamp,
|
||||
/** 距上次心跳秒数 */
|
||||
beatAgeSec: z.number().nonnegative(),
|
||||
/** 超过 45s 没心跳 */
|
||||
/** 超过 45s 没心跳(只对当前实例告警) */
|
||||
stale: z.boolean(),
|
||||
/** 进程正常退出时写入;非 null 即已下线 */
|
||||
stoppedAt: Timestamp.nullable(),
|
||||
/** 最近一次采样的事件循环延迟 */
|
||||
eventLoopLagMs: NonNegInt,
|
||||
/** 最近一个心跳周期(15s)内的最大延迟 */
|
||||
@@ -434,7 +436,10 @@ export const SystemHealth = z.object({
|
||||
checkedAt: Timestamp,
|
||||
/** 处理本请求的 api 进程 */
|
||||
api: z.object({ version: z.string(), startedAt: Timestamp, uptimeSec: z.number().nonnegative() }),
|
||||
/** 当前实例:最近 10 分钟内有心跳的 host 上、同 host 同 kind 最新启动且未退出的进程;stale 与告警只看这里 */
|
||||
processes: z.array(SystemProcess),
|
||||
/** 已下线实例:已正常退出、所在 host 10 分钟没有心跳(容器已重建)、或被同 host 新进程替换;不计入 level */
|
||||
retiredProcesses: z.array(SystemProcess),
|
||||
sse: z.object({ connections: NonNegInt }),
|
||||
db: z.object({
|
||||
level: HealthLevel,
|
||||
|
||||
Reference in new issue
Block a user