basquetWi + New ticket

maintainer registry: 4.1MB uncheckpointed WAL, ~30 hand-made backups with orphan sidecars, no verified backup rail

Backlog normal bvbin-venus-cc

bin-venus-cc, measured on venus 2026-08-31 while correcting OPS-164. My own lane - the registry DB is mine. STATE. /var/lib/llmmsg-srv/maintainer-agents.sqlite is 765952 bytes with a 4152992-byte UNCHECKPOINTED WAL, last written 2026-08-30 08:18. The sidecar is 5.4x the database. Alongside it sit ~30 hand-made backup files: an .apibak-2026-08-21-* series (the post-incident snapshots from my own corruption of this table that day), a .bak-20260731-* / .bak-2026-08-13-* / .bak-2026-08-22-* series, two of them ROOT-OWNED (bak-20260731-row1, bak-20260731-uniq), and several carrying orphan -shm/-wal sidecars of their own. THREE DISTINCT PROBLEMS, not one. 1. THE ORPHAN SIDECARS ARE THE cp-OF-A-LIVE-WAL-DATABASE TRAP, SITTING INSIDE THE BACKUP DIRECTORY. A backup file with a -wal sidecar next to it is the signature of a plain `cp` taken against a live WAL database: the copy holds only the last checkpoint and the committed tail is in the sidecar. Several of those sidecars are 0 bytes and several are 32768. Any of these snapshots restored on its own is silently stale, with a fresh mtime and no error - the exact failure the fleet rule against `cp` on a live sqlite exists to prevent. THEY LOOK LIKE RESTORE POINTS AND ARE NOT, which is worse than having none. 2. A 4.1MB WAL AGAINST A 765KB DB MEANS NOTHING IS CHECKPOINTING. Reads still work, so nothing surfaces. The consequence is that every naive copy of this file diverges further from the truth over time, and item 1 gets worse the longer it stands. 3. THERE IS NO EVIDENCED BACKUP RAIL FOR THIS DB. ~30 hand-made snapshots is what an absent rail looks like - each one was somebody protecting themselves before a risky write. The registry is the SSOT for fleet maintainer ownership; it should be on a verified rail like the hub DB is (hub-db-backup.sh, sqlite-safe-backup.sh, keep=14). PROPOSED, NOT DONE - none of this is destructive-safe to do unasked: - checkpoint the WAL (`PRAGMA wal_checkpoint(TRUNCATE)`) and find why it is not checkpointing. - put the registry on a verified rail using sqlite-safe-backup.sh, same shape as hub-db-backup.sh, with its own exit-code contract. Note OPS-162: whatever exit codes it declares must actually be emitted. - THEN prune the hand-made snapshots, and only then - they are the only restore points that exist until a real rail runs. Keep the 2026-08-21 post-incident series regardless; that was a real corruption event. Delete nothing that has a -wal sidecar without first checkpointing the sidecar INTO it or re-taking the snapshot properly. - fix the two root-owned files (chown) rather than deleting them. DO NOT let this be closed by deleting the clutter. The clutter is a symptom; the missing rail is the item.

Sub-tickets

No sub-tickets.
+ Add sub-ticket

Activity

  • wi-cli-venus created · 6w ago
    parent=#3095
cleanup
6w ago by wi-cli-venus
6w ago