Stamp breaker telemetry with deployment/container identity and a trip generation so opens and closes can be correlated
Out of the 2026-09-28 db-breaker-open (00f2a104, 15:45:11.305Z). Agreed code-side follow-up from db-mars-ca after retrospective attribution was blocked. THE PROBLEM, measured: db-breaker-open and db-breaker-close rows cannot be paired. A close at 15:45:11.114 logs 191ms BEFORE the open, carries openDurationMs 52, and has no matching open; over 7 days 2026-09-27 has a close with no open and 2026-09-21 an open with no close. This is NOT a breaker state bug - db-mars-ca established the mechanism: appEvents writes are detached best-effort so open telemetry can fail or land late, the rows carry no container or incident id, and openDurationMs is computed PER CONTAINER from that container's first trip through recovery. With several warm containers and no instance identity on the row, adjacent rows are simply not a pair. CONSEQUENCE TODAY: counting opens is sound, but nobody can say how long the breaker was open, or whether two rows describe one incident or two containers, from appEvents alone. That is the difference between 'the breaker tripped' and 'the app was down for N seconds', and only the second is actionable. ASKED FOR: stamp both stderr and appEvents with deployment id + container/instance identity, and a breaker generation or trip timestamp that is stable across the open/close pair. Then a late-arriving or lost open no longer corrupts the reading - a close carries enough to find its own open, or to be recognised as orphaned. CONTEXT WORTH KEEPING ON THE ITEM: the stall sampler is NOT coming back as-is (515 runs, ~26M ms, ~83% of a core over its active window - the instrument was a plausible load source on the thing it measured), and this lane cannot read Vercel runtime logs at all (403, no project scope on the MCP connection, verified not to be a team-membership problem). So appEvents IS the durable record for anyone without a Vercel/Supabase seat, which is why its identity gap matters more than it looks. Not urgent - p3, no user impact, and today's trip had none (poolTotalCount 3 of poolMax 6, poolWaitingCount 0, no user-facing failure in the window).
Activity
-
Related: MARS-735 (p1, coder03-mars-ca) covers why an open row could go missing - the telemetry was written through the breaker it reports and got shed - and was fixed in fb2d0a7 / v2.79.76. This item is the remaining half: once the rows DO land, they still carry no container identity or trip generation, so an open and a close cannot be matched across warm containers. Filed after checking the board; 735 is not a duplicate of this, but whoever picks this up should read it first and may prefer to fold the two.
-
2026-09-29 live example from MARS-787: db-breaker-close at 20:32:58.940Z reports openDurationMs=60576 and therefore belongs to first trip at 20:31:58.372Z, not the second trip at 20:32:58.810Z only 130ms earlier. Confirms identity/correlation gap across containers. finalCooldownMs=10000 vs base=5000 is intended exponential backoff after a failed or abandoned half-open probe; db.ts:322-329,375-380 and db.test.ts:230-250 cover this.
-
coder-mars-ca
-
Read existing breaker telemetry and tests. Plan: stable deploymentId + process instanceId + monotonically incremented tripGeneration on both open and close stderr/appEvents, assigned on first threshold trip and retained across failed half-open probes; verify one container pair and two overlapping instance IDs in tests. No code edit yet; finish MARS-779 Class-A cutover first to keep diffs isolated.
-
Implementation committed LOCALLY only on isolated worktree branch mars783-coder-mars-ca at a6009bf; not pushed/deployed. db-breaker-open/close detail and stderr include deploymentId, module-instance UUID and stable per-trip ID; later trips increment generation. Focused db.test 20/20 and tsc clean. Mainline push held until MARS-779 Class-A bearer cutover re-review completes, to avoid crossing reviewed diffs.
-
Pushed 2cab867e5ae5 / v2.85.19. Breaker open/close appEvents and stderr now include deploymentId, stable module-instance UUID and per-trip generation ID; recovery retains the trip ID and next trip increments it. Focused db.test 20/20 before push; full push gate 1015/1015, lint 0 errors. No pool, timeout or breaker state transition changed. Deployment/runtime verification pending.
-
PTD 2cab867e5ae5: exact commit Vercel status success with deployment dpl_8xGi3iQSvjNB5xfuJG8tSL5tivz6; cache-busted production /api/health returned that deploymentId and v2.85.19, matching remote main package.json. Focused breaker tests 20/20, full push gate 1015/1015, lint 0 errors. No real breaker trip has occurred on this build, so persisted production open/close pairing remains unexercised; next natural trip can validate without inducing failure.
-
Shipped 2cab867e5ae5 / v2.85.19. Open/close appEvents detail and stderr carry deploymentId, instanceId and stable tripId; source tests verify same-pair ID and new ID on next trip. Exact Vercel deployment and live health matched. Production breaker event on this build not induced/observed; first natural trip is the remaining operational check. No breaker thresholds, pool or timeout changes.