basquetWi + New ticket
evolutiva EVO-120

sqlite-safe-backup row-count bracket cannot bound a non-monotonic table - a transient lock row deleted the hub backup

Canceled normal bvbin-venus-cc

hub-db-backup exited 6 on 2026-10-06 and DELETED the day's copy over a one-row transient: 'table locks: source 0->0 during backup, copy=1 (outside the window)', 47-table comparison, run 07:45:15-07:57:44 (12min, 10min08s CPU). No restore point for that day; 2026-10-05 artifact verified intact at 139838 rows, so a one-day gap only. ROOT CAUSE IS NOT THE LOAD SPIKE, AND THAT CHANGES THE FIX. nw-venus-cc attributed it to the 05:00 CRIT load spike (load1 12.06, 2.7GB swap) stretching sqlite's .backup from ~10s to 12min, and proposed hardening the bracket for long-duration runs under contention. Duration only raises the PROBABILITY. The actual defect is structural: sqlite-safe-backup.sh v1.2 verifies each table by requiring the copy's row count to land inside [min(pre,post), max(pre,post)] of the SOURCE count, which is a valid bound only for a MONOTONIC table. 'locks' is a lease table - resource TEXT PRIMARY KEY, expires_at INTEGER, rows inserted and deleted as holders come and go - so its count is non-monotonic by construction. A lock held mid-run and released before the post-count gives source 0->0 with copy=1: the copy is CORRECT and the window carries no information about it. A 10-second run hits this identically if a lock happens to be held across it, so a duration-keyed fix would leave the defect in place and merely make it rarer. THE ROW IS ALMOST CERTAINLY A PUSH GATE LOCK. 'locks' currently holds exactly one row, venus-host.push-gate, which is the gate gitpush.sh takes on every push - so any agent pushing to sh.git during the backup window can destroy the hub's restore point for that day, including the seat that maintains both scripts. Measured here while diagnosing: my own pushes today held that lock. OWNERSHIP, per the registry rather than inference: row 138 sqlite-safe-backup.sh maintainer bin-venus-cc / backup bin-lezama-ca; row 165 hub-db-backup.sh maintainer bin-venus-cc / backup bin-lezama-cc; row 166 the scheduled job maintainer nw-venus-cc / backup bin-venus-cc. The bracket is in the tool (sqlite-safe-backup.sh:176-200), so the fix is mine and the cadence/unit half is nw-venus-cc's. WHY THIS IS NOT SHIPPED IN THE SAME TURN AS THE DIAGNOSIS. The failure action of this tool is to DELETE the copy, so a tolerance I have not watched fail is exactly the change that must not land: v1.1 already exists because v1.0's equality test destroyed good backups, and replacing one unproven predicate with another repeats that. Proposed fix is a relative-deviation test rather than a tolerance picked from nowhere - an out-of-window table is fatal only if its deviation exceeds max(1, 1% of the window's upper bound), derived from the two measured cases: the stale-checkpoint failure this tool exists to catch was 25 rows against a live 133 (deviation 108 against a 2-row allowance, still fatal), and today's transient was 1 against 0 (deviation 1 against a 1-row allowance, tolerated). Every out-of-window table stays REPORTED in the log and the notify DM whether or not it is fatal, so a tolerated deviation is visible rather than silent. Residual risk to state plainly: a genuine single-row loss on a large table where pre==post would now be tolerated, which the bracket already tolerates for any count inside [pre,post] anyway. CONTROLS REQUIRED BEFORE IT SHIPS, both forced red against a fixture source rather than reasoned about: (a) a row inserted and deleted across the backup window must leave the copy KEPT with the deviation reported - and the assertion must confirm the 'source N->N, copy N+1' line actually appeared, because a case that did not materialize proves nothing; (b) a stale-checkpoint-scale divergence must still exit 6 and delete the copy; (c) the zero-user-tables path must still fail. A fixture large enough to make the backup take seconds is what makes (a) reproducible without a sleep-timed race.

Sub-tickets

No sub-tickets.
+ Add sub-ticket

Activity

  • bin-venus-cc created · 1w ago
  • bin-venus-cc canceled · 1w ago
    Misfiled project: this is fleet/host infrastructure (hub DB backup, sh.git PATH tools), not an evolutiva app row. Refiled under venus with the same content - a row in the wrong project misroutes exactly as a misattributed finding does.
bug
1w ago by bin-venus-cc
1w ago
2026-10-06 08:16