basquetWi + New ticket

Build heartbeat/dead-man's-switch watchdog for registered scheduled jobs

Done normal bvbin-venus-cc

Root problem (Elazar): Sonnet has no innate sense of time -- if an expected report (e.g. monthly rkhunter CLEAN email) simply never arrives, nothing notices. Build a watchdog that reads the job registry (OPS-103) and for each entry checks systemd's own execution truth (systemctl show <unit> -p LastTriggerUSec/Result, or journalctl for cron.d) against the expected cadence + grace window -- NOT 'did an email arrive', since that conflates job-didn't-run with report-delivery-failed. On overdue/failed: DM nw-venus-cc. On healthy-and-ran: also emit a positive low-noise signal (e.g. rolled into sys-check heartbeat, not a separate email) so absence of a NEW alert isn't the only signal of health. Depends on OPS-103's registry format existing first.

Sub-tickets

No sub-tickets.
+ Add sub-ticket

Activity

  • wi-cli-venus created · 2026-07-12
    parent=#1880
  • wi-cli-venus commented · 2026-07-12
    Correction: this watchdog depends on OPS-101 (the registry format/tooling), which depends on OPS-103 (inventory sweep) for seed data. Fix stray OPS-103 registry-ref typo in original description -> should read OPS-101.
  • wi-cli-venus blocked · 2026-07-12
    waiting on OPS-101 (registry format) to exist before a consumer can be built
  • wi-cli-venus completed · 2026-07-12
    Built + deployed 2026-07-12 (folded into the OPS-101 tool, one invocation per nw sign-off). `jobreg watch` reads systemd LastTriggerUSec + .service Result vs interval+grace, DMs nw-venus-cc on drift/overdue/failed, writes ~/.local/state/jobreg/status for sys-check.sh to fold into heartbeat. Daily via jobreg.timer (08:00, enabled). First run caught 2 real silent failures -> OPS-104, OPS-105.
  • wi-cli-venus unblocked · 2026-08-19
    backfill: blockedReason cleared on a closed row; it described a dependency discharged before the item closed and pre-dated the d6153e2 fix that now clears it automatically (Elazar decision 1b, 2026-08-19)
task
2026-07-12 by wi-cli-venus
2026-08-19
2026-07-12 02:49