Fleet deliverable: draft BACKTEST SPEC + TRIAGE-CODIFICATION CHECKLIST template (Pluto-owned)

Sub-tickets

Activity

• wi-cli-venus created · 2026-07-06
• wi-cli-venus assigned · 5w ago
  coder-pluto-cc
• coder-pluto-cc note · 1w ago
  Both artifacts drafted. /data-venus/pluto/specs/2026-10-06-triage-codification-checklist-template.md and /data-venus/pluto/specs/2026-10-06-alert-triage-backtest-spec.md. Not pushed anywhere - these are drafts for the fleet consensus round, and nothing lands in shared/ from Pluto. PREMISE CHECKED BEFORE DRAFTING, because it looked dead: 524 builds the evidence gate for a fly-solo re-enable, while PLUTO-201 is a brief to DELETE the fly-solo doctrine. The doctrine is still live and unamended - evolutiva-pm-commons.md line 98 section Autonomy Fly-Solo Error Remediation, evolutiva-error-alerting-standard.md section 5, and the evolutiva-rulings-detail.md#fly-solo-remediation anchor all present and intact. PLUTO-201 was never executed (it is explicitly hold-for-an-Elazar-launched-team-session, created 2026-06-23, and 524 is NEWER at 2026-07-06). So the gate is the condition on which the doctrine would return, not a contradiction of the kill brief, and it is useful under either outcome. Said here rather than DMd because it is a premise note, not a blocker. DESIGN POSITIONS TAKEN, each one a place the spec as dispatched would have scored a lane green for the wrong reason: 1. A MANDATORY SECURITY-SUBSET FLOOR of 5, and a run that misses it is UNSCORED, not a pass. Zero false-clears over zero security-shaped alerts is exactly the shape of a control that has never been seen to fail. This is the single change most likely to matter, because the dispatched pass bar (zero false-clears on the security subset) is trivially satisfiable by a corpus with no security subset. 2. THREE OUTCOMES, not two - pass / fail / unscored - and the per-project table spells out 'not yet run' in its Outcome column rather than leaving cells blank. An empty row is the most likely way the packet gets misread as a pass. 3. COLDNESS MUST BE PROVEN, NOT ASSERTED. The corpus is prepared by a different seat, the disposition/close/WI fields are withheld rather than marked, and the withheld dispositions' sha256 is published BEFORE the replay so the oracle cannot be adjusted after seeing the answers. 4. FAILING TO MAKE THE AUTH-GATE-FIRST DETERMINATION SCORES AS A FALSE-CLEAR even where the final disposition matches, since on those items the determination IS the deliverable. 5. Benign-precision's denominator is what the lane CLAIMED was benign, so a lane that never closes anything cannot score 100%. KNOWN LIMIT, on the packet's first page and not in an appendix: the oracle is PRIOR TRIAGE, so a class mis-triaged consistently in history scores as a pass in both artifacts. Neither detects a systematically wrong default; only a fresh look at the underlying behaviour does. The checklist's coverage is exactly the historical alert corpus, and its benign entries are the likeliest to be wrong because a benign signature is the one nobody re-examines. Checklist entries are Pluto-measured real signatures with a memory/rule link each, four fields per entry (signature, default, the ONE escalate-if, source), auth-gate-first as a hard rule evaluated BEFORE the table rather than as a row in it. The four-field shape is the fleet-portable part; a sibling app forks the table and keeps the shape. OPEN ROUTING QUESTION for whoever runs the consensus round: this row says the final fleet md routes via bin-whey-cc, but evolutiva-coder-commons.md section Commons maintenance says shared/ is maintained by the MARS coder lane with Venus coder as backup. Not resolving that unilaterally.