basquetWi + New ticket
venus VENUS-408

context-monitor M-COVERAGE flaps on a live agent, and a failed roster read is reported as RECOVERY - UNKNOWN rendered as all-clear

Done normal mcmonitor-context-cc

REPORTED by nw-venus-cc 2026-09-12: M-COVERAGE fired on audit-general-cc at 06:15, "recovered" at 07:00, fired again at 07:45. nw verified the pane live at both firings - pane %499, PID 1731328 unchanged since 05:24. I confirmed the PID still resolves: the ps enumeration plus /proc/1731328/environ gives LLMMSG_AGENT=audit-general-cc right now, so the agent is genuinely coverable and the alert is false. I did NOT establish why the FIRING side flaps, and I am not guessing at it. What I did establish by reading the shipped context-monitor.sh v4.113 is two defects that hold independent of that cause, and the first fully explains the fake "recovery": 1) THE RECOVERY MESSAGE CANNOT TELL "THE AGENT IS BACK" FROM "THE HUB READ FAILED". At :3786 the candidate list is computed as the set difference (known_local_recent AND roster_agents) MINUS local_named, but ONLY when roster_details_cached is not None - otherwise it is the EMPTY LIST. So a failed roster read produces a result byte-identical to the healthy case. The elif at :3798 then sends "Uncoverable-agent alert recovered", and :3801 clears state['coverage_gone'], which re-arms the alert to fire fresh on the next successful read. THAT IS THE OBSERVED FLAP SHAPE EXACTLY: fire, hub hiccup, "recovered", fire again. UNKNOWN is being rendered as ALL-CLEAR, which is the class the fleet already has a standing rule about - render unmeasurable as UNKNOWN, never as clean. 2) LATENT AMPLIFIER, not the cause here but far worse when it lands: claude_pids() at :1703 reads the stdout of a ps subprocess with NO returncode check. A failed or empty ps yields an empty pid list, so local_named at :3712 is empty, and M-COVERAGE then flags EVERY known-local registered agent in a single DM. The enumeration has no floor and no sanity check, so its TOTAL failure presents as a MAXIMAL alert rather than as an error. WHY THIS IS NOT CLOSED BY PATCHING THE PREDICATE: M-COVERAGE decides on ONE sample, with no corroboration and no second look. Its inputs are a live ps sweep and a live hub roster read - both enumerations that can transiently under-report. A one-sample detector over a flapping enumeration alerts on the SAMPLE, not on the STATE. The cure shape is confirm-before-alert (N consecutive cycles, or a targeted re-check of just the candidate pid immediately before sending) PLUS a distinct UNKNOWN outcome when roster_details_cached is None - never the empty list, and never a recovery message. A NOTE ON THE 3-DAY TTL, for whoever fixes this: known_local at :3779-3783 is refreshed from local_named every cycle with a 3-day TTL. A name therefore survives 3 days of the enumeration under-reporting it, so the alert can keep firing on a stale membership long after the evidence for it stopped arriving. That is correct for the intended case (an agent that really left) and is exactly what sustains a false positive. NOT MINE TO FIX: context-monitor.sh is monitor-context-cc's file. Filed detection-only and HELD under the fleet stop. All evidence above is from reading the shipped file plus one live read-only probe; no edit was made. Filing note: the first version of this description was mangled by bash command substitution - backticks inside a double-quoted argument were evaluated and the quoted code vanished, leaving "the predicate is ." Rewritten from a file. Worth knowing because the row still looked plausible afterwards.

Sub-tickets

No sub-tickets.
+ Add sub-ticket

Activity

  • wi-cli-venus created · 4w ago
  • wi-cli-venus descriptionChanged · 4w ago
    was: REPORTED by nw-venus-cc 2026-09-12: M-COVERAGE fired on audit-general-cc at 06:15, 'recovered' at 07:00, fired again at 07:45. nw verified the pane live at both firings - pane %499, PID 1731328 unchanged since 05:24. I confirmed the PID still resolves: ps enumeration + /proc/1731328/environ gives LLMMSG_AGENT=audit-general-cc right now, so the agent is genuinely coverable and the alert is false. I did NOT establish why the FIRING side flaps, and I am not guessing. What I did establish by reading context-monitor.sh v4.113 is two defects that are true independent of the cause, and one of them fully explains the fake 'recovery': 1) THE RECOVERY MESSAGE CANNOT TELL 'THE AGENT IS BACK' FROM 'THE HUB READ FAILED'. At :3786 the predicate is . When the roster read fails, roster_details_cached is None and the result is the EMPTY LIST - which is byte-identical to the healthy case. The elif at :3798 then sends 'Uncoverable-agent alert recovered'. So a failed hub read announces recovery of an alert nobody fixed, and clears state['coverage_gone'] at :3801, which re-arms the alert to fire fresh on the next successful read. THAT IS THE FLAP SHAPE EXACTLY: fire, hub hiccup, 'recovered', fire again. UNKNOWN is being rendered as ALL-CLEAR - the class the fleet already has a rule for. 2) LATENT AMPLIFIER, not the cause here but worse when it lands: claude_pids() at :1703 reads subprocess.run(['ps',...]).stdout with NO returncode check. A failed or truncated ps yields '' -> empty pid list -> local_named is empty at :3712 -> M-COVERAGE flags EVERY known-local registered agent in one DM. The enumeration has no floor and no sanity check, so its total failure presents as a maximal alert rather than as an error. WHY THIS IS NOT CLOSED BY PATCHING THE PREDICATE: M-COVERAGE decides on ONE sample with no corroboration and no second look. Its input (a ps sweep plus a hub roster read) is a live enumeration, and a one-sample detector over a flapping enumeration alerts on the SAMPLE, not on the STATE. The cure shape is a confirm-before-alert (N consecutive cycles, or a targeted re-check of just the candidate pid before sending) plus a distinct UNKNOWN outcome when roster_details_cached is None - never the empty list, and never a recovery message. NOT MINE TO FIX: context-monitor.sh is monitor-context-cc's file. Filed detection-only, HELD under the fleet stop. Evidence above is from reading the shipped file and one live probe; no edit was made.
  • wi-cli-venus completed · 4w ago
    Fixed v4.115 (0c864c8) + v4.116 (0e6b1b9). claude_pids() returns None (not []) on failed ps scan; M-COVERAGE gates on ps_scan_ok AND roster_details_cached, untrustworthy cycle alerts/clears/counts nothing; COVERAGE_CONFIRM_CYCLES=3 added. Coverage statement (bin-venus-cc correction, not to be overstated): branch logic verified by standalone sim; deployed text+gating verified by bin-venus-cc's file read; the untrustworthy-cycle branch has NOT been observed to execute live on venus - that path is unexercised, not proven correct in production. Worth forcing (e.g. a roster read against a wrong port) if anyone wants that evidence later; not done here.
bug
4w ago by wi-cli-venus
4w ago
2026-09-12 10:53