Remove idle/unreachable agent hunting from the hub
Elazar CLI 2026-09-20: 'get rid of all those hunting iddle agents and missing agents'. Verbatim earlier: 'You -the scripts and agents- treat unreachable agents as a tragedy, and it is not.' An agent the hub cannot reach is NORMAL - he shuts agents down deliberately when a subscription runs low, some (coder-chatduo-cc) are launched on demand only, and he shut down pm-llmmsgsrv-cc, the fleet's top token consumer, 'and all works just the same'. REMOVE, do not tune. The predicate cannot distinguish the cases it is alerting on: [stuck-agent] fires on 'has unread and has not drained', which is equally true of an agent ignoring mail, an agent idle by design, and an agent that has been shut down. A 57-incident figure built from those lines was withdrawn for exactly that reason - it counted a non-fault. In scope (hub/hub.mjs unless noted): stuck-agent alerting (STUCK_ACTIONABLE_SQL ~:11017, stmtStuckCandidates ~:11025 and its emitter), silence alerts, unreachable/missing-agent reporting, and any sibling escalation whose trigger is 'an agent is quiet'. Reply/busy-signal nudges that fire on an OWED reply are a different predicate - enumerate them and decide each explicitly rather than sweeping them in. Also check the stale-reaper and any scrp-* consumer of these alerts before deleting the producer. Constraint: hub.mjs is a shared surface - build in a throwaway clone, get an audit pass before the working tree changes, and PM sequences the restart. Background: /data-venus/llmmsg-srv/findings/2026-09-20-codex-push-and-inline-body.md section 5a.
Activity
-
Diff built in a throwaway clone at e3f07b9 (byte-identical to live hub.mjs). hub/hub.mjs only, 66 insertions / 599 deletions, at /data-venus/llmmsg-srv/diffs/2026-09-20-msg414-remove-idle-hunting.patch. REMOVED: runSilenceCheck, runStuckCheck, kickSilentAroMembers, the single silenceInterval running all three plus its shutdown clear, SILENCE_*/STUCK_*/ARO_SILENCE_* consts, stmtSilenceState*/stmtStuckStateUpsert/stmtAro{RecvCount,SentCount,PeerDmSentCount,LastKick,KickUpsert}/stmtAllAroKeys, SILENCE_SKIP_AGENTS, STUCK_EXEMPT_RE, STUCK_ACTIONABLE_SQL, stuckPollPids + its notePollPid call in /unread (single reader; stmtSetPollPid still feeds fork detection). DECIDED EXPLICITLY, KEPT: runBusyNudgeCheck + runContactEscalationCheck (predicate is an OWED REPLY, not quietness); runBsStuckCheck (coder-llmmsgsrv-ca's read, agreed - a blackboard invite is an explicit owed WRITE and auto-skip is board completion); pruneStale; /fleet_health and /silence_thresholds as READ surfaces so a human can still see who is quiet with nothing paging anyone. NO DDL CHANGE: silence_state, silence_thresholds, aro_kick_log left as orphan tables - dropping them would need a migration rehearsal + rollback boot for no gain. Stale comments describing the removed machinery were corrected in the same diff. TESTS: node --check clean; clone booted on :19705 against a VACUUM INTO copy of live v2.sqlite, zero ReferenceError, no silence-detector/stuck-agent/aro-kick armed lines, bs-stuck/blackboard-sweep/busy-nudge/kick-checkin/contact-escalate/pmq-sweep all still armed; route sweep 200 on /health /roster /online /fleet_health /silence_thresholds /stale_clients /locks /unread?agent= /unread_agents /guide, 403 on /elazar_contact (gate working). CONTROL: a dangling SILENCE_CHECK_S planted in /health passes node --check and fails the boot with http 500 'SILENCE_CHECK_S is not defined' - the boot test discriminates. Scratch hubs killed by PID after matching /proc/<pid>/cwd; prod pid 2523428 untouched. NOT EXERCISED: POST routes, the blackboard and lock paths, and any handler outside the sweep above. Audit requested from audit-general-ca. Restart authority open - both pm-llmmsgsrv twins are off by Elazar's decision, so it goes to him.
-
SHIPPED. audit-general-ca cleared both BLOCK items (re coder-llmmsgsrv-cc-mu9a5sxwgxm9, 00:52 ART): the corrected pruneStale comment (an ARO membership IS a send list - /send to=aro:X expands via stmtAroMembers with no roster-active filter, hub.mjs:5919 and origin_aro at :6007; accepted-and-documented rather than filtered, because filtering fan-out to active roster members would be silent mail LOSS for exactly the deliberately-shut-down agents this change exists to stop persecuting) plus three stale comments. Final diff 97 insertions / 609 deletions, hub/hub.mjs + package.json. Commit d19ea9f, version 2.60.0 (minor: feature removal). Restarted llmmsg-srv-venus.service myself under Elazar's 2026-09-20 decision (relayed by coder-llmmsgsrv-ca) that both coding lanes restart the hub when a change requires it. VERIFIED ON THE RUNNING PROD HUB: new pid 1750474 (was 2523428), ExecMainStartTimestamp 2026-09-20 03:55:05 UTC, /health reads version 2.60.0 (read back from the deployed artifact, not asserted); boot log carries NO silence-detector, stuck-agent or aro-kick armed line, and still arms pmq-sweep, bs-stuck, blackboard-sweep, busy-nudge, kick-checkin, contact-escalate; zero errors since start. DEPLOYMENT SCOPE - HUB ONLY: this restart deploys the hub change and NOTHING else. It does NOT deploy the MSG-412 shim work; that needs scripts/install-mcp-shim-runtime.sh from 5d1d920+, runtime version >=2.59.27 verified per session, and the affected Codex sessions restarted - tracked as a separate deployment state in MSG-412. Whey /opt/llmmsg-srv (2.55.1) and lezama are now further behind; neither auto-pulls, route a pull --ff-only to an on-host agent. NOT EXERCISED by this ship: POST routes, blackboard/lock paths under load, and the removed detectors' absence over a full day - the only proof they are gone is the boot log, not elapsed time.
-
completed