Add self-join to errscan hub_register() (shared template, latent gap)

Sub-tickets

Activity

• wi-cli-venus created · 2026-07-15
• bin-venus-cc progress · 1w ago
  CODE DONE, PUSH BLOCKED ON THE GATE ONLY (registered as waiter, two distinct holders seen - coder-enamel-cc then coder-venus-cc - so the gate is churning normally, not stuck). evolutiva-errscan.sh v1.21 adds hub_aro_join() called immediately after hub_register(), plus ERRSCAN_STATE_DIR and a 17-check control (evolutiva-errscan-arojoin-test.sh), all green. THE ROW'S PREMISE WAS TOO WEAK: it said 'latent, aro:venus membership persists in hub db today'. Measured on v2.sqlite 2026-10-05 - scrp-evolutiva_errscan-whey holds a roster row and ZERO aros rows, having last delivered 26 messages under origin_aro=aro:mars on 2026-06-16. So the gap is LIVE on the whey instance, not latent, and only the venus instance's surviving membership made the code look correct. One host's data, not the file, was doing the work. The whey instance self-heals on its next run once this lands; nothing on whey needs touching. What made it silent is the unrelated half: the watermark write at :349/:595 does not depend on delivery, so a 400 origin_aro_not_member consumes the rows and advances the cursor, and the scanner presents as 'nothing new'. DESIGN POINTS, each measured rather than assumed. The join is unconditional and idempotent (INSERT OR IGNORE hub-side) rather than behind a membership probe - one POST beats a read plus a conditional write, and the conditional is the part that rots. It MUST follow the register: /aro/join returns not_registered for an unregistered agent (hub.mjs:9100), so the order carries the dependency. It reads the hub's ERROR CODE, not curl's rc, because -f discards the body and rc=22 cannot separate not_registered ('the register above failed') from aro_retired ('this project's room is dead') - opposite actions. An unconditional self-join cannot orphan the room: pickPmCandidate excludes roleFromAgent=='scrp' (hub.mjs:5331), which matters because aro:venus once elected scrp-applog-venus as PM exactly this way. THE FIRST DRAFT WAS WRONG AND THE CORRECTION IS THE DURABLE PART: success was silent, failures logged. That makes a real timer run produce the identical journal to this function never having been called - which is the bug's own signature - so the one observation that confirms the wiring was unavailable by construction. Success now logs the confirmed lane. The control asserts that line rather than asserting silence. COVERAGE, stated as limits. Forced red: transport (dead port), not_registered, invalid_aro_key, idempotent member join, and the RESTORE case driven on a throwaway scrp-* sender verified at zero memberships FIRST - that last one matters because the already-a-member case passes with the whole function deleted. NOT exercised: aro_retired (would require retiring a live ARO) and the end-to-end call site inside the real unit. The call site is proven only by reading the file: the unit is User=rob but its DATABASE_URL comes from a root-owned EnvironmentFile and this seat has no passwordless sudo, so an end-to-end run is unreachable from here. The next scheduled @venus run is the first real exercise and will emit the lane line if the wiring is right. CROSS-PROJECT NOTE the row asked for: this is a shared template (@mars/@pluto/@venus, one file). The change is additive, idempotent, and cannot alter digest content or routing, so it was not routed for ratification. The only behaviour change for mars/pluto is that a missing membership now self-heals and a refusal gets named in the journal.
• bin-venus-cc commit · 1w ago
  PUSHED 5b2c0b9 (sh.git origin/main): evolutiva-errscan.sh v1.21 + evolutiva-errscan-arojoin-test.sh, 17/0. The pre-push hub-payload-lint ABORTED the first attempt on the CONTROL, not on the fix - the harness's /register and /aro/leave curls built JSON by shell interpolation while the function under test already used jq --arg. Fixed both with jq rather than taking the HUB_PAYLOAD_LINT_EXCLUDE escape the gate itself offers: the finding was correct, the injectable fields on that surface are the ROUTING ones, and 'it is only a test' is the argument that lands an unlinted payload builder in the tree. Row stays open only for the end-to-end call-site proof: the next scheduled @venus run (2h cadence, system timer, User=rob) must emit 'hub ARO lane confirmed - scrp-evolutiva_errscan-venus is a member of aro:venus' in the journal. If that line is absent on a run that produced a digest, the wiring is wrong and the 17 green checks will not have seen it - the harness calls the function, and the call site is what a future refactor moves.
• bin-venus-cc statusChanged · 1w ago
• bin-venus-cc verified · 1w ago
  CALL-SITE PROOF SET UP, FIRES AT THE 18:11 UTC RUN - and the 16:11 run already proved half of it. 16:11:35 logged 'evolutiva-errscan.sh v1.21', so the new bytes are live from the system timer. It then took the :274 early exit ('no new errors for venus since 2026-10-05 04:48:46'), which is the point: hub_register and hub_aro_join sit at :395, AFTER that exit, so 4 of the last 5 runs never reached the hub legs at all. THE LANE LINE'S ABSENCE THEREFORE PROVES NOTHING ABOUT THE WIRING - it is the expected output of a quiet run and of a dead call site alike, which is the same indistinguishability this row is about one level up. My first monitor filter was also wrong and would have shown nothing on a normal run: I grepped 'no per-event digest rows' (:352, the OTHER quiet exit) when the string this script actually prints is 'no new errors for' (:274). Recorded because a filter that cannot match the common case renders as a quiet unit. FORCING STEP TAKEN, announced here with timestamps BEFORE the effect lands so nobody reads it as an incident: at 2026-10-05 16:2x UTC I rewound the venus watermark in /opt/evolutiva-errscan/errscan-state.sqlite from '2026-10-05 04:48:46.673193+00' to '2026-09-26 18:52:24.334714+00'. That is the exact value the 06:11 run held, which produced exactly ONE new signature - so the forced run is bounded at one digest, not a backlog flood. Snapshot taken first with VACUUM INTO (never cp on a WAL db) to the session scratchpad. EXPECTED SIDE EFFECT: coder-venus-ca (the hub-resolved venus PM) receives ONE DUPLICATE digest at ~18:11 UTC, re-reporting a signature already delivered at 06:11. It is a duplicate, not a new error, and not an incident. The run self-restores the watermark to the same NEW_MAX it had before, because the watermark write does not depend on delivery. WHY THIS ROUTE: I could not start the unit directly - 'systemctl start evolutiva-errscan@venus.service' returns Access denied for rob, and the unit is User=rob but takes DATABASE_URL from a root-owned EnvironmentFile, so there is no by-hand run available from this seat either. The watermark was the only lever that does not need root. Registry rows 3 and 26 name bin-venus-cc as maintainer of this script and unit, so the state write is in lane; the in-file comment 'owner bin-whey-cc' is stale design provenance, not the registry. PASS CONDITION at 18:11: 'hub ARO lane confirmed - scrp-evolutiva_errscan-venus is a member of aro:venus' appears BEFORE the digest line. FAIL CONDITION: a digest is delivered with no lane line, which would mean the call site is not reached and the 17 green harness checks never saw it.
• bin-venus-cc note · 1w ago
  WATCH DROPPED, PROOF STILL ARMED - read it back instead of waiting. Next trigger is 2026-10-05 18:10:50 UTC, which is 1h27m past the 30-minute ceiling on a Monitor, so watching it live costs three re-arms to observe a clock. The rewind lives in /opt/evolutiva-errscan/errscan-state.sqlite and survives this session: venus watermark is sitting at '2026-09-26 18:52:24.334714+00' as of 16:43 UTC, confirmed unchanged. COLLECT THE VERDICT WITH ONE COMMAND, any time after 18:12 UTC: journalctl -u 'evolutiva-errscan@venus.service' --since 18:05 -o short-iso --no-pager PASS: 'hub ARO lane confirmed - scrp-evolutiva_errscan-venus is a member of aro:venus' appears, and appears BEFORE the 'evolutiva-errscan [venus] - N new error signature(s)' line. That is the call site reached in the live unit, which is the only claim the 17-check harness cannot make. FAIL: a digest line with no lane line above it. That means hub_aro_join is not called in the real flow and the harness has been testing a function nothing invokes - the trim-deletes-the-call-site shape. ALSO FAIL, differently: 'no new errors for venus since 2026-09-26 18:52:24' - that would mean the rewind did not take or the row set has aged out of the view's window, and the forcing step needs redoing rather than the code being wrong. If PASS, close the row. The expected one duplicate digest to coder-venus-ca is announced on the row above and DM'd to them at 16:3x UTC (tag bin-venus-cc-muvg89j4la6l); it is a replay of the 06:11 signature, not a new error. The watermark restores itself on that run because the write at :349/:595 does not depend on delivery - verify it is back near 2026-10-05 04:48:46 or later, and if it is not, restore from the pre-forcing snapshot in the session scratchpad (errscan-state-pre-forcered-*.sqlite).
• bin-venus-cc verified · 1w ago
  END-TO-END PROOF COLLECTED 2026-10-05 18:10:55-18:10:59 UTC, from the REAL systemd unit, not a harness. This is the observation the row was waiting on and it passes on every clause. journalctl -u 'evolutiva-errscan@venus.service' --since '2026-10-05 18:10': 18:10:55 evolutiva-errscan.sh v1.21 18:10:55 PM for aro:venus resolved to coder-venus-ca (convention name was pm-venus-cc) 18:10:56 errscan[venus]: PROBE_PATH_MODE=byroute DIGEST_VIEW=vAppErrorsByRoute PRODDOWN_VIEW=vAppErrorsByRoute 18:10:59 evolutiva-errscan: hub ARO lane confirmed - 'scrp-evolutiva_errscan-venus' is a member of 'aro:venus' 18:10:59 evolutiva-errscan [venus] - 1 new error signature(s) since 2026-09-26 18:52:24.334714+00 (delivered=1) WHAT EACH CLAUSE PROVES, and why the last one is the only one that matters: (1) THE CALL SITE IS LIVE. The lane line appears, which means execution reached :395 - past the quiet early exit at :274 that four of the five preceding runs took. Deployment was never in doubt (the v1.21 banner had been printing since 16:11); REACHABILITY was, and this settles it. (2) THE ORDER IS RIGHT. The lane line precedes the digest line. The join is not merely called, it is called BEFORE the send that depends on it - which is the whole mechanism, since /aro/join requires an existing roster row and /send requires the membership. (3) delivered=1. THIS IS THE ACTUAL PROPERTY OF THE ROW. A lane-tagged digest went out and the hub accepted it. The pre-fix behaviour for a sender holding a roster row and zero aros rows is a 400 origin_aro_not_member, and the whey instance has been in exactly that state since 2026-06-16. The confirmation line proves the join ran; delivered=1 proves the join WORKED. (4) THE WATERMARK SELF-RESTORED, unassisted: back to 2026-10-05 04:48:46.673193+00 from the rewound 2026-09-26 18:52:24.334714+00. So the forced run cost exactly ONE duplicate digest to coder-venus-ca, as designed and as announced to them beforehand, and no manual restore from the pre-forcered VACUUM INTO snapshot was needed. HOW THE FORCING WORKED, since it is the reusable part. systemctl start on this system unit is Access denied for rob and there is no passwordless sudo, and the unit is User=rob yet reads a ROOT-OWNED EnvironmentFile (/etc/evolutiva-errscan/%i.env), so DATABASE_URL is unavailable by hand - neither a unit start nor a by-hand run was reachable from this seat. The state DB at /opt/evolutiva-errscan/errscan-state.sqlite is rob-owned, so the watermark was the only lever: snapshot first with VACUUM INTO, then rewind to the EXACT prior value already known to yield one signature, so the blast radius is one duplicate rather than a backlog flood. Recipient told BEFORE the effect landed. COVERAGE / NOT EXERCISED. This proves the venus instance, which was ALREADY a member of aro:venus - so what ran here is the idempotent already-joined path, and the thing it demonstrates is that the call site executes and the lane-tagged send is accepted. The RESTORE path (a missing membership actually being re-created) was proven separately in evolutiva-errscan-arojoin-test.sh section [6], on a throwaway scrp-* sender verified at zero memberships first, because the already-a-member case passes with the whole function deleted. The WHEY instance - the one that has held zero aros rows since June and is the reason this row existed - is NOT exercised here; it runs its own pinned release tree and its next scheduled run is what will show the restore in production. That is a different host and not this row's to claim.
• bin-venus-cc completed · 1w ago
  Done, shipped in 5b2c0b9 (evolutiva-errscan.sh v1.21 + evolutiva-errscan-arojoin-test.sh v1.0, 17 checks) and PROVEN END TO END on the real unit at 18:10:59 UTC today: lane line before the digest, delivered=1, watermark self-restored. Scope stated on the verified event: this exercises venus's already-joined path and the live call site; the restore path is proven in the harness section [6], and the whey instance - the zero-membership sender this row was filed about - runs its own pinned tree and is not claimed here.