mars
MARS-687
Review-depth pilot: keep measuring; verdict issued-but-not-recorded is the open finding
Done normal
cmcoder-mars-ca
Elazar 2026-08-31: keep this OPEN until there is actual data; do not close on the current single window. Mars-only scope - do NOT propose a fleet commons rule yet. BACKGROUND. The review-depth pilot (protocol SSOT: shared/docs/2026-08-12-review-tier-protocol-brainstorm.md section 11) replaced Class-A/Class-S on Mars, opening at 5f1da49 with a nominal ~2-week / 20-push window. Compliance is manual BY DESIGN - that is the thing being measured - and audit's PASS:5f1da49 was explicitly conditional on push-path enforcement before any rollout. No WI existed for it until now; that absence is itself a finding, since the pilot had no ledger home and its window lapsed unmeasured. FIRST REAL MEASUREMENT (2026-08-25..27 window, six commits by coder-mars-ca, reconciled 08-31 under MARS-682). Silent skips: ONE (4713c06 - no retained review request, no evidenced review-depth.sh run). The other five WERE cold-reviewed and did receive deep PASSes at the time. THE ACTUAL FINDING, which is not the skip. Those five verdicts existed ONLY as hub messages and were never written to a WI. From the ledger a verdict that was issued-but-not-recorded is indistinguishable from one never issued. Measured cost: I could not tell, so I re-requested all six and audit re-reviewed all six. Duplicated review work is the price of a verdict with no durable home. PROPOSED MARS-ONLY REMEDY, not to be applied until more data: a verdict is not complete until it lands in bwi against a WI. Corollary for any push-path enforcement built from this pilot - gate on the RECORD, not on the issuance. WHAT WOULD MAKE THIS CLOSEABLE: several more push windows reconciled the same way (git log range vs recorded verdicts), enough to say whether the record gap is systemic or was one agent's habit. Reconcile per window, append the counts here, and do not generalise to the fleet from Mars alone. Related: MARS-682 (the reconciliation), MARS-686 (removal of the unrequested feature that window produced).
Activity
-
MECHANISM FINDING that bears directly on the remedy proposed here, measured 2026-08-31. gitpush.sh --review has been DEFAULT-ON for --only pushes since v4.37 (landed 2026-08-22, three days BEFORE the 08-25..27 window), and Mars is strict-mode so every push is --only. That looks like every Mars push already auto-DMs the auditor. IT DOES NOT. gp_review_default_on fires only in a repo that DECLARES an audit agent, read by gp_review_target from $REPO_ROOT/.gitpush-review as an 'audit=<name>' line. pensanta-websites has NO .gitpush-review at its root, so for Mars the default-on path is a SILENT no-op - deliberately silent, since a banner on every push in every undeclared repo is a check that cannot pass. Every Mars review request is therefore hand-sent and forgetting is unenforced, which is precisely the failure this WI exists to measure. CHEAP CANDIDATE FIX, not applied, needs a decision first: add .gitpush-review with audit=audit-mars-ca at the repo root, converting 'the coder must remember to ask' into 'the tool always asks' with no new code. BLOCKER on doing it unilaterally: pensanta-websites is a MULTI-APP repo, so one root-level audit= line would point every sibling app's pushes at Mars's auditor. Needs the other apps' consent, or a per-app mechanism that does not exist today. Raise on the coders room before adding it. ALSO, from bin-venus-cc (OPS-189 under OPS-173): gitpush already resolves short+full SHA at :1428 and hands them to gp_send_review, which sends over the HUB. So the review REQUEST is exactly as ephemeral as the VERDICT this WI is about - a hub message is not a record. Any reconcile must key on something written to disk or bwi, never on a send. That widens the finding: BOTH ends of the review handshake are currently unrecorded, so 'was review requested' is as unanswerable as 'was a verdict returned'. METHOD NOTE worth keeping: on learning the tool default-sends, my first read was that this WI's one-genuine-skip count must be wrong. Checking the declaration showed the feature is inert here, so the count stands. A default documented as ON can still be OFF for your repo - verify the precondition, never the default.
-
coder-mars-ca
-
Reassigned coder-mars-cc -> coder-mars-ca on 2026-09-19 under Elazar's direct Mars coder+PM handover, relayed by coder-mars-cc at 05:39 ART and explicitly ordered by pmmaster-evolutiva-cc at 05:40 ART. This transfers backlog ownership only; implementation/push authority remains governed by mars-commons.md.
-
Second bounded review-depth reconciliation, 2026-10-06. Denominator: git log --first-parent --since=2026-10-04T00:00Z --until=2026-10-06T00:00Z on Mars origin history, 9 commits (Oct 4: 19fcad4,f5a2613,e0a769d,b3f6927,1f41497; Oct 5: 73c3462,bdc9196,e5ec199,c97cf42). All 9 touch src/package; no docs-only exclusion in this window. Searched the associated BWI histories MARS-792, MARS-784, EVO-106, MARS-794, MARS-796, MARS-732 and parent EVO-103 for SHA-matched audited/review verdict events. DURABLE BWI VERDICT: 2/9 (b3f6927 on MARS-784, 1f41497 on EVO-106). KNOWN HUB VERDICT BUT NO MATCHING BWI VERDICT: bdc9196 audit BLOCK (audit-general-ca-muvrgqtbvbmg), e5ec199 audit PASS (audit-general-ca-muvrnc2gsedn), c97cf42 audit PASS (audit-general-ca-muvvzrh5h0lu). The remaining 4 have no verdict on those WIs; this does NOT prove they were never requested or reviewed elsewhere. Thus the issued-but-not-recorded gap recurred in a distinct window (at least 3 known cases), while 2 prove recording is possible. Current repo still has no .gitpush-review declaration; the historical default-on push path remains silent here. No fleet extrapolation or code change. Next reconciliation should inspect another window and distinguish reviews recorded under alternate WI refs before any final pilot verdict.
-
Second bounded Mars push-window reconciliation recorded; 9 commits, 2 durable verdicts, at least 3 hub-issued but not BWI-recorded. Continuing additional-window measurement per Elazar's keep-open instruction.
-
Third bounded reconciliation (partial 2026-10-06 00:00..13:35Z, origin/main at 2cab867): 11 first-parent commits 4577498,053540b,b299180,8226055,c07c6c5,0fb58f8,e7d3c85,d2bb3b4,c31ea94,44317ef,2cab867. scripts/review-depth.sh classifies all 11 DEEP (4577498 checked from full Mars checkout because temp clone is shallow at its parent). BWI events --limit 80 on associated MARS-736/781/751/748/797/779/800/783 and EVO-115 show ZERO audited/audit-finding events matching these SHAs. At least six had actual hub PASSes: 4577498 (audit-general-ca muw4zgbsq10d), 8226055 (muwbfhn9ffyx), c07c6c5 (muwby4ae4xol), 0fb58f8 (muwddvrq8hxt), e7d3c85 (muwo1liwmvqw), c31ea94 (muwomxnj5765). Some coder-authored commit/verified bodies mention those PASS tags, but none is an auditor-signed audited event. The other five have no verdict on these WIs, NOT proof no review occurred or that no alternate WI holds one. This third window independently reproduces issued-but-not-recorded; no fleet inference or push-gate change. Pilot closure metrics (audit token burn, shallow sample quality) still missing.
-
Decision gate, not a clock: three bounded Mars push windows now show the durable-verdict gap recurring. Latest partial Oct 6 window: 11/11 classified DEEP, zero audited events on associated WIs, at least six known hub PASSes; earlier windows include five and three issued-but-not-recorded. Need PM/Elazar ruling on a Mars-only requirement for auditor-signed bwi audit events and where to enforce/declare it. Root .gitpush-review remains absent and would misroute sibling apps in the multi-app repo; do not add it unilaterally. Pilot close metrics (audit token burn and shallow sample quality) remain unmeasured/zero-denominator.
-
PM ruling 2026-10-06 (pmmaster-evolutiva-cc-muwqfqsie7eq), Mars-only: a Class-A WI cannot close until the auditor writes a bwi audit verdict on that WI naming the shipped SHA. Author/PM may not write the verdict. PASS plus other verification gates permits close; FAIL does not. Class-S keeps doer self-verification. Enforce at WI close, not push: a verdict cannot exist when the hook runs. The review-depth deep/shallow pilot remains a separate measurement; no root .gitpush-review. Three measured windows establish the durable-record gap (latest 11 DEEP, zero audited events on associated WIs, at least six wire-only PASSes). Pilot token burn and shallow sample quality remain unmeasured.
-
PM decision gate resolved; Mars-only close rule documented in mars/shared/md/mars-coder-agents.md. Keep pilot open for audit token-burn and shallow-sample-quality measurements; latest bounded window has shallow denominator zero. No fleet-wide inference.
-
5772dd379f35 on origin/main: Mars-only coder rule now requires auditor-authored bwi audit PASS naming the shipped SHA before a Class-A WI closes; Class-S self-verifies. Gate is at WI closure by policy, not push. Scoped documentation push; hook passed 92 files/1029 tests. This does not modify bwi close mechanically; MARS-687 remains inProgress for unmeasured audit token burn and shallow-sample quality.
-
Read-only close sweep shipped dd319562733f and EXECUTED before push against BWI MARS-1..801 with (no BWI write, no verdict backfill). 581 done rows found; 56 explicitly Class-A in title/description/close text. Of those, 27 name exactly one shipped SHA in close text but have no auditor-authored PASS event matching it; 29 have absent/ambiguous close SHA and need tier/SHA triage; zero met the auditor-authored PASS criterion in this historical explicit subset. 525 done rows have NO tier marker, so 27 is a lower-bound detector output, NOT the complete number of Class-A violations or a compliance rate. MARS-750/9d6c77f is a concrete historical example; MARS-783 excluded after tightening classification because its Class-A mentions concern another row. This is pre-ruling history: do not retro-stamp verdicts. Audit token burn and shallow sample quality remain unmeasured.
-
Correction to immediately preceding verified event: shell formatting stripped two inline literals from its body. The exact read-only sweep command was node scripts/audit-class-a-closes.mjs 801, run from the Mars repo root. The required event type is audited, authored by an audit-* agent; 27 named-SHA explicitly labeled Class-A historical closes lacked a matching auditor-authored PASS. All other counts and limitations in that verified event stand. No verdict was backfilled.
-
PM disposition 2026-10-06 (pmmaster-evolutiva-cc-muwrfehsy9bc) for read-only sweep through MARS-801, command from Mars repo: node scripts/audit-class-a-closes.mjs 801. Four buckets among 581 done rows: 27 explicitly labeled Class-A, one named close SHA, no auditor-authored SHA-matched audited PASS; 29 explicitly labeled Class-A but absent/ambiguous close SHA; 525 no tier marker; 0 covered by the strict authored-PASS criterion in the explicit historical subset. These are HISTORICAL LOWER-BOUND detector buckets, not a violation count or compliance rate. The count is dated: initial loose classifier included MARS-783 because its event mentioned OTHER Class-A work; tightening to row title/description/close text excluded it and changed the numbers. Dispositions: 525 unmarked rows closed as unmeasurable, not re-tiered; 27 named-SHA rows unverifiable-by-construction, with MARS-750/9d6c77f as example, no retroactive PASS or memory reconstruction; 29 ambiguous rows only cheap per-row SHA triage, and any requiring a historical tier judgement joins unmeasurable. No verdict backfill. Forward fix shipped 5772dd3: auditor-authored PASS naming shipped SHA required before Class-A WI close; read-only detector shipped dd319562733f. Pilot audit token burn and shallow-sample quality are separate still-unmeasured work, not closure claims here.
-
Closed 2026-10-06 on PM ruling. Mars-only forward close rule shipped 5772dd379f35: Class-A WI closure requires auditor-authored bwi audit PASS naming shipped SHA; Class-S self-verifies. Read-only detector shipped dd319562733f; executed node scripts/audit-class-a-closes.mjs 801 from Mars repo. Historical 581 done rows split: 27 explicitly Class-A with one named shipped SHA but no matching auditor-authored PASS; 29 explicitly Class-A with missing/ambiguous close SHA; 525 no tier marker; 0 strict authored-PASS covered in the explicit subset. DATED LOWER BOUND, not a violation count or compliance rate: tightening classifier to row/close text excluded MARS-783, whose Class-A mention concerned another row. Disposition: 525 unmeasurable, no retro-tiering; 27 unverifiable-by-construction, MARS-750/9d6c77f worked example, no verdict backfill; 29 cheap SHA triage only, historical tier-judgement cases join unmeasurable. No historical PASS reconstructed from memory. Unmeasured audit token burn and shallow-sample quality moved to blocked MARS-802; no estimate or implicit pilot-quality claim.
audit
6w ago by wi-cli-venus
1w ago
2026-10-06 14:21