Measure the connection-layer fixes, then decide on MARS-690 - reminder DMs start 2026-09-09

Sub-tickets

Activity

• wi-cli-venus created · 5w ago
• wi-cli-venus verified · 5w ago
  CORRECTION to this WI's own measurement plan, 2026-09-05, verified by coder-mars-cc rather than inherited. The line 'pool.totalCount/idleCount/waitingCount settles whether OUR 6 slots were full or the pooler refused us' is FALSE. It is already settled, and the answer is neither: the failure never involved our client pool. Evidence: the three incident errors read '(ECHECKOUTTIMEOUT) unable to check out connection from the pool after 15000ms in Transaction mode'. Our connectionTimeoutMillis is 10000, so the 15000ms budget is not ours. ECHECKOUTTIMEOUT appears NOWHERE in the installed dependency tree (grepped node_modules/.pnpm). pg-pool 3.14.0 emits 'timeout exceeded when trying to connect' and 'Connection terminated due to connection timeout' instead (node_modules/.pnpm/pg-pool@3.14.0_pg@8.22.0/node_modules/pg-pool/index.js:224,276). All three markers are Supavisor's. Our client obtained one of its own 6 slots and reached the pooler; SUPAVISOR could not check out a server connection, because the server was hung - which db-mars-cc measured independently (zero connections accepted 22:04:31Z-23:00:00Z; checkpoint write=85.144s on 32KB). So max:6 was never the binding constraint and pool.waitingCount reads LOW during this failure class. Found by coder02-mars-cc; the discriminator sat in the error text from the first hour and nobody read the number. WHAT THIS CHANGES: - Pool counters stay but are DEMOTED: evidence that the bottleneck is upstream, not a trigger and not the instrument that decides MARS-690. - The circuit breaker is keyed on the ERROR SIGNATURE (5 ECHECKOUTTIMEOUTs in 30s), not a waitingCount threshold. A waitingCount trip would have been INERT for the exact failure it was designed to catch. - statement_timeout is NOT settable from app code on this connection type: Supabase ignores the options startup parameter and a session SET under Supavisor TRANSACTION mode, and DATABASE_URL is the :6543 pooler since MARS-145. It needs one DDL, ALTER ROLE ... SET statement_timeout, dispatched to db-mars-cc 2026-09-05. Until that lands the 'statement_timeout kill count' measurement has nothing to count. - MARS-690 stays PARKED. Nothing here unparks it. The reminder timer (scrp-mars691_reminder-venus) shipped carrying this stale sentence inline, and with a start date of 2026-01-01 instead of 2026-09-09; both reported to bin-venus-cc. General lesson for any future reminder: carry the ref and the action, never restate the reasoning - a confident inline sentence carries no date and goes stale silently the moment the investigation moves.
• wi-cli-venus verified · 5w ago
  Day-2 measurement run, 2026-09-10 10:0xZ, coder-mars-cc. MCP/psql auth is working again; all three appEvents-backed measurements ran. THE WINDOW IS CONTAMINATED AND MUST BE EXTENDED - see below. 1) CIRCUIT BREAKER OPENS: ZERO. Query over the last 7 days of appEvents matching action ILIKE breaker|circuit|checkout|pool|timeout, or category ILIKE db, returns an EMPTY SET. Not 'no trips' - no rows of any of those shapes at all. Denominator stated: appEvents holds 399/510/376/322/258/471/597 rows on 09-01..09-07, so the table is alive and the absence is real for that period. 2) POOL COUNTERS on ECHECKOUTTIMEOUT: nothing to read, same empty set. Consistent with the 09-05 correction on this WI (the 15000ms budget is Supavisor's, not ours; our slots were never the binding constraint) - it is not new evidence either way. 3) get_practica_detail_bundle EXEC-TIME DISTRIBUTION: NOT MEASURABLE TODAY. pg_stat_statements_info.stats_reset = 2026-09-10 09:28:15.676Z - the MARS-700 dashboard restart cleared the counters ~30 minutes ago, over a near-empty early-morning window. Any number pulled now describes the restart, not the route reductions. This is exactly what MARS-702 was filed for: re-run after a full business day. 4) MATERIAL FINDING, and it corrects MARS-700's stated onset. appEvents write volume per hour, 09-08 to 09-09: 09-08 runs 12-31 rows/hour through 22:00Z, then STOPS. All of 09-09 has NINE rows total, never more than 2 in any hour, scattered one-per-hour. WAL-G's first wal-push failure was 2026-09-08 23:46:49Z. So the write path degraded from the hour WAL-G broke - roughly 24 HOURS BEFORE Supabase forced the project read-only at ~09-09 23:23Z. The 23:23:24Z max(createdAt) I reported on MARS-700 is the LAST SURVIVING WRITE, not the onset. A max is not an onset. Caveat stated: appEvents is not a traffic meter, so a low-traffic day is not excluded by this data alone - but 09-06/09-07 were 471/597, so weekday/weekend is not the driver. CONSEQUENCE FOR THIS WI: 09-08 22:00Z through 09-10 09:28Z is a period in which the instrument (appEvents, a WRITE) could not record the events it exists to record, and pg_stat_statements has since been zeroed. Roughly two of the measurement window's days are blind. The window does NOT have enough clean data to decide MARS-690. Do not delete the reminder timer yet; re-measure from a clean baseline starting today. MARS-690 stays PARKED - unchanged.
• wi-cli-venus note · 5w ago
  DB-lane measurement correction, 2026-09-10 ~10:08Z. A direct rolling-7-day appEvents query is NOT empty: 13 rows match ECHECKOUTTIMEOUT, dated Sep 4-9; 0 rows match statement timeout; 0 db-breaker-open rows; poolTotalCount/poolIdleCount/poolWaitingCount are NULL on all 13 checkout rows. These are Supavisor 15s checkout errors and do not supply app-client pool evidence. Absence of breaker rows is not durable proof of zero opens because breaker reporting itself writes appEvents and is blind while DB writes fail. The Sep 8 22:00Z-Sep 10 09:28Z degradation/outage interval is additionally write-telemetry blind.\n\nPersisted get_practica_detail_bundle daily interval means through the last pre-restart snapshot (Sep 8) were 724.70, 439.78, 313.11, 641.18, 293.62, 347.10, 359.12, 283.14, 333.12 ms for Sep 8 back through Aug 31 (daily calls 112-279). pg_stat_statements reset at the Sep 10 restart, so there is no meaningful post-restart execution distribution yet.\n\nDecision: measurement window is incomplete; MARS-690 remains PARKED. Keep the reminder timer until at least one clean business-day window is collected.
• wi-cli-venus verified · 5w ago
  RETRACTION AND REPLACEMENT of my 'EMPTY SET' claim, 2026-09-10, coder-mars-cc. db-mars-ca is right and I was wrong; I have now reproduced its 13 rows myself and can say why my query missed them. MY ERROR: I searched category and action NAMES (ILIKE breaker|circuit|checkout|pool|timeout, category ILIKE db). The rows do not carry the signature in either column - they are category='server', action='unhandled-server-error', and 'ECHECKOUTTIMEOUT' appears ONLY in errorMessage/errorStack. A name-column search over a message-column signature returns a confident empty set. The correct predicate is on errorMessage/errorStack/message, not on category/action. THE 13 ROWS, all user-facing 500s on /mi-cursada/* or /practicas/*: 09-04 22:24:46, 22:28:22, 22:45:37, 22:45:40 | 09-05 22:49:29 | 09-07 21:56:56, 22:09:53 | 09-09 09:27:58, 13:53:26, 19:21:17, 19:28:14, 23:23:15, 23:23:24. NEW CONCLUSION - THE BREAKER IS INERT AGAINST THIS FAILURE'S OBSERVED SHAPE, and that is now a measured statement rather than an absence. Trip condition is 5 failures in 30s (BREAKER_FAILURE_THRESHOLD/BREAKER_FAILURE_WINDOW_MS, src/lib/db.ts:107). The densest cluster in 13 events across 5 days is TWO within 30s (22:45:37/22:45:40 and 23:23:15/23:23:24). Five-in-30s was never approached. Zero breaker opens is therefore the CORRECT output of the rule as written, not a gap in the instrument. It is worse than the raw numbers suggest, and the code says so itself at db.ts:95 - the counter is PER CONTAINER, and several warm containers each keep their own. So 13 events spread over five days and over N warm lambdas give a per-container count near 1. The threshold is not merely untuned; on this arrival distribution it is structurally unreachable. CAVEAT, stated: appEvents is a WRITE and is blind during a write outage, so the true failure rate is a FLOOR, not a count. But the cleanest window - 09-04 22:24-22:45, an ordinary pool incident with no write outage - shows 4 events in 21 minutes, which is nowhere near 5-in-30s. The floor argument does not rescue the threshold. WHAT THIS GATES: the breaker needs either a much lower threshold over a much longer window, or shared cross-container state, or it should be acknowledged as decorative and removed. That is a design decision, not a measurement, and I am not making it unilaterally - it wants Elazar or a PM call. MARS-690 stays PARKED; nothing here unparks it. Also confirmed from db-mars-ca: 0 statement-timeout matches; NULL pool counters on all 13 (consistent with the 09-05 correction - the 15s budget is Supavisor's, our slots were never binding); latest persisted bundle interval mean 724.70ms on 09-08 against a prior 283.14-641.18ms daily range, so the MARS-680 climb has NOT reversed - but pg_stat_statements was zeroed at 09:28:15Z today, so post-restart distribution is unmeasurable until a business day runs (MARS-702).
• wi-cli-venus verified · 5w ago
  FIRST CLEAN MEASUREMENT WINDOW. 2026-09-10T10:00:54Z to 2026-09-11T09:37:58Z, 23.6h, 610 appEvents rows spread across 23 distinct hours - the denominator matters, because appEvents is a WRITE and a silent instrument reads identically to a healthy system. It was live the whole window. 1. ECHECKOUTTIMEOUT: ZERO. Predicate: errorMessage/errorStack/message ILIKE '%ECHECKOUT%' (these rows carry category='server', action='unhandled-server-error', so a category/action search returns a confident empty set - see the MARS-691 retraction). Prior rate was 13 events over 09-04..09-09, with the densest cluster 2-in-30s and one window of 4 in 21 minutes. Zero over a full day of real traffic. 2. Breaker opens: ZERO, which follows from item 1 rather than adding to it. Threshold was retuned 5 -> 2 at a7dd9e5 (MARS-704) on that 2-in-30s measurement; at 5 it was structurally unreachable. It has still never fired. 3. The pool.totalCount/idleCount/waitingCount instrument is STILL UNEXERCISED. This was the question the 2026-09-04 incident could not settle - our own 6 slots full, versus the pooler refusing us. It cannot be answered by a clean window; it needs an ECHECKOUTTIMEOUT to fire with the instrument in place. Record it as open, not as resolved. 4. statement_timeout=10s kills: TWO, both 2026-09-10T19:44Z on /mi-cursada (appEvents 46cf20d7-b643-4ea7-909d-1a0b57e90bc7, 62811e8f-49b7-4c3c-9662-006beebc757e). Per this WI's own framing that is the informative outcome: both fell inside a DB-WIDE stall in which a trivial pg_available_extensions catalog read hit 35330.3ms and a large-objects report 32093.4ms. So the 46.7s class is STALL-LINKED, not routine load. Detail on MARS-702. 5. get_practica_detail_bundle exec time: mean 359.6ms over 235 calls, max 3143.9ms. Against MARS-680's 375.8 -> 461.0 -> 631.9ms climb over 08-25..08-27, the mean is back below the start of that climb and the max is an order of magnitude under the 16.9-37.6s burst. Coalescing shipped at 8155822 after most of this window, so credit is not cleanly attributable. DECISION THIS GATES - MARS-690 stays PARKED and is now OPTIONAL rather than urgent, which is the outcome this WI names for exactly this evidence: hold time is bounded by statement_timeout and saturation is absent, so the serial write loops are tolerable. Downgrading it, not closing it - one clean day is not a week, and item 3 is still blind. REMINDER TIMER: keeping it. It costs one DM a day and the two conditions that would reverse this decision (an ECHECKOUTTIMEOUT firing with pool counts attached, or timeout kills appearing OUTSIDE a stall) can only show up in more days.
• wi-cli-venus verified · 4w ago
  SECOND CLEAN MEASUREMENT WINDOW, 2026-09-12, coder-mars-cc. Window 2026-09-11T09:37:58Z to 2026-09-12T09:57:49Z, 24.3h, contiguous with the first clean window so the two now cover ~48h end to end. Denominator stated because appEvents is a WRITE and a silent instrument reads identically to a healthy system: 383 rows across 24 DISTINCT HOURS - it was live for every hour of the window. 1. ECHECKOUTTIMEOUT: ZERO. Predicate on errorMessage/errorStack/message ILIKE '%ECHECKOUT%' - the message-column predicate, not category/action, per this row's own retraction. Second consecutive zero day. Prior rate was 13 events over 09-04..09-09. 2. Breaker opens: ZERO. Follows from item 1 rather than adding to it. Still never fired since the 5->2 retune at a7dd9e5. 3. statement_timeout kills: ZERO. Materially different from day 1, which had TWO - and both of those fell inside a DB-wide stall. Two clean days with zero kills strengthens the day-1 reading that the 46.7s class is STALL-LINKED rather than routine load. 4. unhandled-server-error rows: ZERO across the whole window. No user-facing 500s of any kind. 5. get_practica_detail_bundle: 364 calls, mean 337.5ms, max 3143.9ms (top statement; a second normalised entry adds 28 calls at 355.4ms). Mean is DOWN from 359.6ms on day 1 and remains below the 375.8ms that started MARS-680's 08-25..08-27 climb to 631.9ms. Max is unchanged at 3143.9ms, which means the tail did not recur in this window rather than that it improved - same counter, not yet reset. 6. THE POOL-COUNTER INSTRUMENT IS STILL UNEXERCISED, and this is the one item two clean days CANNOT advance. poolTotalCount/poolIdleCount/poolWaitingCount only populate on an ECHECKOUTTIMEOUT row, and there have been none. Recording it as OPEN, not resolved - a clean window is structurally incapable of answering it, which is exactly why it must not be allowed to age into looking answered. DECISION: MARS-690 stays PARKED and OPTIONAL, unchanged from day 1 and now on twice the evidence. Nothing here unparks it and nothing here closes it. REMINDER TIMER: KEEPING IT. Two clean days is not the week this WI asked for, and the two conditions that would reverse the decision - an ECHECKOUTTIMEOUT firing with pool counts attached, or a timeout kill appearing OUTSIDE a stall - are both only observable in more days. Deleting the timer now would retire the instrument precisely while its remaining question is unanswerable. NOTE ON A CONFOUND I AM NOT HIDING: this window contains today's comision relabel work (migrations 161/162/163) plus a db:export run, all of which touch the DB. None of it is app request traffic and none writes appEvents error rows, so it does not contaminate items 1-4; it may add a small number of calls to item 5.
• wi-cli-venus verified · 4w ago
  SECOND CLEAN MEASUREMENT WINDOW, DB lane, 2026-09-11T09:37:58Z to 2026-09-12T09:57:49Z - 24.3h, 383 appEvents rows across 24 DISTINCT HOURS, so the instrument was live the whole window and the zeros below are absence of events rather than absence of instrument. PREDICATE PROVEN BEFORE THE ZEROS WERE BELIEVED, and this nearly went wrong exactly as it did on 2026-09-10. My first pass searched detail->>'errorMessage' / detail->>'errorStack' / detail->>'message' and returned a confident ZERO. I ran the control this row's own retraction demands - point the same predicate at 2026-09-04..09-09, where 13 ECHECKOUTTIMEOUT rows are known to exist - and it found 0 of 13. THE SIGNATURE IS NOT IN THE detail JSONB AT ALL: unhandled-server-error rows carry only routeType, renderSource, method, digest, routerKind, routePath there. errorMessage, errorStack and message are TOP-LEVEL COLUMNS on appEvents. Re-run against the columns, the control returns 13 of 13, and only then are the window-2 numbers meaningful. Recording the failed shape as well as the working one: the 2026-09-10 retraction said 'the correct predicate is on errorMessage/errorStack/message' and that sentence is ambiguous between column and jsonb key - it cost me a false zero, and the next reader deserves the disambiguation. MEASUREMENTS, all on the proven predicate: 1. ECHECKOUTTIMEOUT: ZERO. Second consecutive clean day (window 1 also zero). Prior rate was 13 events over 09-04..09-09. 2. statement_timeout kills: ZERO. Window 1 had two, both inside the 09-10 DB-wide stall. So across two clean days there are still ZERO timeout kills OUTSIDE a stall - which is one of the two conditions this row names as capable of reversing the MARS-690 decision, and it has not occurred. 3. Circuit breaker opens: ZERO. Follows from item 1, adds nothing independent. 4. Pool counters (poolTotalCount/idleCount/waitingCount): STILL UNEXERCISED, zero rows carry the key. Unchanged and still open. 5. get_practica_detail_bundle: cumulative since the 09-10 09:28:15Z reset now reads 364 calls, mean 337.5ms, max 3143.9ms. pg_stat_statements has NOT been reset between windows, so this is cumulative and not a window-2 figure - DERIVING the window-2 slice from window 1's 235 calls / 359.6ms gives roughly 129 calls at a ~297ms mean, i.e. window 2 was FASTER than window 1. Max unchanged at 3143.9ms, so no new outlier was added. Against MARS-680's 375.8 -> 461.0 -> 631.9ms climb, both windows sit below the START of that climb. 6. Only TWO error-level rows in the entire window, neither connection-layer: an image-proxy fetch timeout (20003ms on connect-or-ttfb), and one self-check-run reporting '1 of 39 data-layer probes failed'. I read that second one rather than pattern-matching it: the failing probe is getDriftCanary, failing with 42703 'column thisColumnDoesNotExist does not exist'. THAT IS A DELIBERATE CANARY AND ITS FAILURE IS THE HEALTHY STATE - it exists so the probe harness is never a control that has not been seen to fail. Not an alert. MY POSITION ON THE TIMER, and it is a recommendation to the assignee rather than an action I am taking: RETIRE IT. Not because the measurement is complete - item 4 is still blind - but because A DAILY REMINDER CANNOT PRODUCE THE EVIDENCE THAT IS MISSING. The two conditions this row names as decision-reversing (an ECHECKOUTTIMEOUT firing with pool counts attached, or timeout kills appearing outside a stall) are INCIDENT-TRIGGERED, not time-triggered. Measuring a healthy system daily yields a clean result every day, forever, and two clean days already establish the baseline. What the row actually wants is an ALERT on those two signatures, which fires on the event rather than on a schedule - and which would also close the row's own 'free and unbuilt' item, the checkpoint-write alert. A reminder that asks a human to go and look is the weaker instrument for a condition that announces itself. If the assignee disagrees, keeping it costs one DM a day and I will not argue further. DECISION THIS GATES: nothing changes from window 1. MARS-690 stays PARKED and OPTIONAL - hold time is bounded, saturation is absent across two consecutive clean days, and the serial write loops remain tolerable. Still not CLOSED, because item 4 has never been exercised and a decision made on two clean days is a decision about a healthy system.
• wi-cli-venus note · 4w ago
  REMINDER TIMER RETIRED, 2026-09-13, on pmmaster-evolutiva-cc's GO. Ran on venus: systemctl --user disable --now mars-691-reminder.timer. Verified after: is-enabled=disabled, is-active=inactive, zero rows in list-timers. The unit FILE remains (mars-691-reminder.service is static, .timer now disabled) - it is disabled, not deleted, so it can be re-armed without rebuilding if a reason appears. bin-venus-cc owns the script and unit and is being told. THE REASONING, recorded because it generalises past this row: the two conditions that could reverse the MARS-690 decision - an ECHECKOUTTIMEOUT firing with pool counters attached, or statement_timeout kills appearing OUTSIDE a stall - are INCIDENT-TRIGGERED, not time-triggered. A daily reminder to re-measure a healthy system returns clean every day, forever, while costing a DM a day across three agents. That is a recurring per-invocation cost with no possible yield, and per Elazar's multiplier ruling a recurring cost outranks a larger one-off. A clean SECOND 24h window is evidence to ACT on, not evidence to keep sampling. What this row actually wants, and what remains genuinely unbuilt, is an ALERT on those two signatures plus the row's own 'free and unbuilt' item - a postgres_logs alert on 'checkpoint complete' lines where write= exceeds a few seconds, which on 2026-09-04 would have fired before any student saw a 500. An alert fires on the event; a reminder asks a human to go and look, which is the weaker instrument for a condition that announces itself. MEASUREMENT STATE AT RETIREMENT, so nobody reads this as the question being closed: two consecutive clean 24h windows (2026-09-10..09-11 and 09-11..09-12), zero ECHECKOUTTIMEOUT, zero statement_timeout kills, zero breaker opens in both. get_practica_detail_bundle back below the start of the MARS-680 climb. MARS-690 stays PARKED and OPTIONAL - downgraded, not closed. ITEM 4 IS STILL BLIND AND ALWAYS WILL BE UNDER CLEAN CONDITIONS: the pool.totalCount/idleCount/waitingCount instrument has never been exercised, because it requires an ECHECKOUTTIMEOUT to fire with the instrument in place. Retiring the timer does not resolve that and must not be recorded as if it had. Row ownership: this row is coder-mars-cc's and the DB-lane measurements were mine. Under the 2026-09-13 runtime switch my open DB rows moved to db03-mars-ca; MARS-691 and MARS-690 are coder-lane and I did NOT reassign them.
• wi-cli-venus note · 4w ago
  CORRECTION AND COMPLETION of my retirement note, from bin-venus-cc - my own event was incomplete in a way that would have caused the loss it warned about. I WROTE 'the unit FILE remains', SINGULAR. THERE ARE THREE ARTIFACTS, and a 'delete the files' reading of my note would have orphaned the third: (1) ~/.config/systemd/user/mars-691-reminder.timer, now disabled; (2) ~/.config/systemd/user/mars-691-reminder.service, static; (3) ~/.local/state/mars-691-reminder/mars-691-reminder.sh, 6543 bytes, in its own state dir, which is what the unit's ExecStart actually points at. Whoever eventually reclaims or removes this takes ALL THREE. Enumerate a unit's artifacts before removing one; a systemd 'unit' is rarely one file. KEEP THEM, DORMANT - bin-venus-cc's decision and the reason is stronger than caution. THE UNITS ARE UNTRACKED AND HOST-LOCAL, so deleting them destroys the only copy in the fleet: git ls-files | grep mars-691 in sh.git returns nothing, and they live under ~/.config and ~/.local/state. A disabled timer costs zero; an untracked file deleted costs everything it was. My disable is confirmed independently from that side: is-enabled=disabled, is-active=inactive. LANE CORRECTION, recorded so the next person routes it right: ~/.config/systemd/user/ is NW-VENUS-CC's surface under the nw/bin split - bin owns code, nw owns units and timers. I routed to bin-venus-cc because the reminder DM itself named bin-venus-cc as maintainer, which is true of the SCRIPT but not of the UNITS. The disable stands (pmmaster GO'd it), but a future DELETE routes through nw-venus-cc. THE ROW'S FREE ITEM IS RESCUED RATHER THAN LOST - MARS-733, the postgres_logs 'checkpoint complete' write= alert, filed deferred by bin-venus-cc precisely because it would otherwise have died with the row that mentioned it. That is the real result of this cleanup: A RULE WITH NO ROW DOES NOT SURVIVE A HANDOFF. Two constraints were written onto MARS-733 so they are not re-derived, and both are corrections to MY wording: the threshold must NOT be 'a few seconds' copied from how I phrased it - it must be derived from an observed baseline of mars's own checkpoint write= distribution and printed next to the value on every run; and it must be forced RED against a synthetic slow line before anyone accepts it, because a monitor never seen to fail is a green line, not a control.
• wi-cli-venus note · 4w ago
  FIRST REAL BREAKER OPEN, 2026-09-13 19:36:32Z, and item 6 is finally answered - from Vercel runtime logs, not appEvents. One lambda instance (dpl_5zJzdnXdF1KGkDQfpny6jxVQGsTJ, v2.79.75) logged twice: acquisition failed total=4 idle=1 waiting=0, cause 'timeout exceeded when trying to connect', then OPEN for 5000ms. READING: poolMax is 6, so our pool had 2 free slots, 1 idle connection and nobody waiting. OUR POOL WAS NOT THE CONSTRAINT - a NEW connection to upstream (Supavisor) timed out. That is exactly the low-waiting signature db.ts:235 names as upstream saturation. DECISION: MARS-690 stays PARKED - enlarging or retuning our pool would not have prevented this. Impact was one user (Windows/Edge), /perfil and /mi-cursada each failed once inside the 5s cooldown; other DB writes succeeded in the same seconds (outbox emails at 19:36:36Z and 19:36:40Z), and /api/health?db=1 read dbOk=true at 19:37:11Z. RETRACTION, and it is load-bearing: my day-1 and day-2 'breaker opens: ZERO' lines are WITHDRAWN as unsound. They were counted from appEvents, and the db-breaker-open row can never reach appEvents - it is written through the open breaker and shed (runtime log: [logEvent-failed] reason=DbUnavailableError action=db-breaker-open). Those zeros measured the instrument's blindness, not the system. The ECHECKOUTTIMEOUT and statement_timeout zeros are unaffected. Defect filed as MARS-735.
• wi-cli-venus note · 4w ago
  STATEMENT_TIMEOUT KILLS 2026-09-14 12:25:03Z and 12:25:19Z on /practicas, full analysis on MARS-736. For this row: burst 1 fell inside a short stall (pg_cron job 6 started 7.5s late, and a normally fast function was killed alongside), while burst 2 came after recovery and killed only the heavy practica-list query, which normally averages 600-800ms with a 6s max. The evidence points at query weight, not the connection layer. MARS-690 stays PARKED.
• wi-cli-venus assigned · 3w ago
  coder-mars-ca
• wi-cli-venus note · 3w ago
  Reassigned coder-mars-cc -> coder-mars-ca on 2026-09-19 under Elazar's direct Mars coder+PM handover, relayed by coder-mars-cc at 05:39 ART and explicitly ordered by pmmaster-evolutiva-cc at 05:40 ART. This transfers backlog ownership only; implementation/push authority remains governed by mars-commons.md.
• coder-mars-ca verified · 1w ago
  Decision audit 2026-10-06: reminder was already retired 2026-09-13 on PM GO (prior WI events); current Venus user systemd has no mars-691-reminder.timer unit. Original MARS-690 decision remains PARKED/OPTIONAL: MARS-787 records 11 breaker trips, every sampled trip waiting=0 with total below max 6 (one trip total=0), and 10/11 correlate with Supavisor SCRAM handshake timeouts; no evidence of local pool-slot exhaustion that set-based write consolidation would cure. Statement-timeout work is separately measured on MARS-736/743/769; list fix 4577498 has its own Oct 13 gate. This is a synthesis of existing BWI/DB-lane evidence, not a new production DB query. MARS-787 causal layer attribution remains open; no pool/schema/write-path change.
• coder-mars-ca completed · 1w ago
  Measurement/decision task complete; no code SHA/version. Reminder retired 2026-09-13 and unit absent today. MARS-690 remains parked/optional: repeated breaker trips show no local pool contention, while statement timeouts/shared stalls have separate active owners (MARS-736/743/769/787). Current evidence does not justify risky serial-write consolidation. No production DB write or new query performed.