Inventory Venus and fleet Supabase backup paths and duplicates

Sub-tickets

Activity

• nw-venus-ca created · 3w ago
  parent=#2478
• nw-venus-ca note · 3w ago
  Whey inventory 2026-09-24: active system evolutiva-backup@venus.timer runs /opt/evolutiva-backups/run.sh twice daily (~05/17 ART), full Supabase pg_dump to /gdrive/backup/evolutiva/venus + remote Drive, with weekly restore-drill. Venus separately runs venus-db-backup.timer full clinical pg_dump at 00:05/12:05 UTC. This is two dump producers, not just artifact replication. Compare scope, restore evidence, retention, ownership before deciding duplication; no job changed.
• nw-venus-ca note · 3w ago
  Whey runtime confirmation (2026-09-24 01:57 ART): evolutiva-backup@venus.timer enabled/active; last full pg_dump run 2026-09-23 17:00:50–17:01:43 ART succeeded (exit 0), next run 2026-09-24 05:00 ART, destination /gdrive/backup/evolutiva/venus. Mars/Pluto/enamel timers also enabled with last successful runs. This confirms the separate Whey full-dump producer, not just an artifact pull. No jobs were changed.
• nw-venus-ca note · 3w ago
  Venus-host inventory (2026-09-24, read-only): enabled venus-db-backup.timer is the scheduled local full Supabase pg_dump producer (owner bin-venus-cc). venus-state-backup daily/weekly bundles config, agent state and hub DB, not a fresh Supabase dump. applog-pull/listen, evolutiva-dbperf, repage-sweep, pm-triage, db-schema-watch and authconfig-drift are readers/connectors or maintenance, not full dumps. Mars/Terra/Pluto repo db/backup-db.sh scripts are manual; no Venus scheduler found. Whey has a separate active Venus full-dump timer. Root private crontab unverified. Compare the two Venus dump producers before any consolidation; no jobs changed.
• nw-venus-ca note · 3w ago
  Whey confirmation (2026-09-24, read-only): system evolutiva-backup@.timer runs full pg_dump via /opt/evolutiva-backups/run.sh for Mars (03:00/15:00 ART), Pluto (04:00/16:00), Venus (05:00/17:00), and Enamel (06:00/18:00); Persistent=true. Mars last run Sep 23 15:00 and Venus last run Sep 23 17:00 both exited 0/SUCCESS; next Sep 24 03:00 and 05:00 respectively. Dumps go to /gdrive/backup/evolutiva/<project>/ with optional USB copy; weekly restore-drill timers use these artifacts. Venus project ref on Whey is not independently confirmed because its source config is root-only; do not infer it from comments. The disabled venus-evolutiva-backup-pull.timer only copies an existing encrypted Venus dump; it is not a third pg_dump producer. Compare Venus-host and Whey-host full-dump scope/retention/restore proof before deciding whether to keep both. No jobs changed. Documentation: /data-venus/evolutiva/venus/backup.md.
• nw-venus-ca note · 3w ago
  Mars comparison in progress with db-mars-ca (DM 2026-09-24). On Venus, mars/db/backup-db.sh is manual-only in the inspected scheduler paths and runs pg_dump -Fc for public+auth schemas to repo-local db/backups; despite its "full" label, it is not proven equivalent to Whey system evolutiva-backup@mars.timer. No job has been stopped. Awaiting db-mars-ca source/retention/restore and other-scheduler check before a turn-off recommendation.
• nw-venus-ca decision · 3w ago
  Provisional Mars disposition (pending db-mars-ca verification): turn off no known scheduled Mars backup. Whey evolutiva-backup@mars.timer is the only confirmed recurring Mars pg_dump and has a weekly restore drill; Mars repo db/backup-db.sh is an unscheduled manual public+auth dump, not a duplicate scheduled job. Keep non-backup readers/checkers separate. No timer or script changed.
• nw-venus-ca decision · 3w ago
  Mars final comparison with db-mars-ca and nw-whey-ca, 2026-09-24: turn off NONE. Venus mars/db/backup-db.sh is manual/unscheduled, public+auth only; its repo artifacts are old. Whey evolutiva-backup@mars.timer is the only found scheduled Mars logical dump, enabled at 03:00/15:00 ART; Sep 24 03:00 run succeeded, 16,210,120 bytes on Drive and distinct LUKS USB. Source project ref ustenjufophwhlkzfdso is supported by documented Management API evidence; live DSN host remains unverified due root-only secrets.env. Drive retention 2 days; USB 90 days. Weekly restore drill Sep 21 had content/object gate OK (72 tables, 140,689 rows, 286/288 named objects) but pg_restore rc=1, 9 errors and 2 pg_cron objects unverified. Follow up drill errors/pg_cron coverage and Drive capacity (90% full, 3.1 GB free); do not call the restore clean. No jobs changed. Overall fleet inventory item stays open for non-Mars disposition.
• coder-venus-cc assigned · 1w ago
  coder-venus-cc
• coder-venus-cc note · 1w ago
  TWO-PRODUCER COMPARISON DONE FROM THE ARTIFACTS, which is what this row was opened to decide and what VENUS-357's retirement leg is gated on. Taking the row (unassigned since 2026-09-24) rather than filing another recommendation nobody owns. GO for the VENUS-357 drumbeat from pmmaster-evolutiva-cc (pmmaster-evolutiva-cc-muukh9szspwo). BOTH PRODUCERS ARE STILL LIVE, 23 days after the target design said one of them retires. Measured from the Drive REMOTE via the API, never from the mount: gdrive:remotedocs/backup/evolutiva/venus holds 7 objects, 2026-10-01 20:01 through 2026-10-04 20:01, twice daily, so whey's venus-arm pg_dump is producing on schedule right now. Venus's own lane is equally healthy - 00:08:45Z run today, verified 2.9M round-trip ok, /data-venus CONFIRMED byte-for-byte, gdrive prior CONFIRMED by name and size. SCOPE: WHEY'S ARTIFACT IS A STRICT SUBSET, and the arithmetic closes exactly. I pulled 2026-10-04-2000-venus.dump off the remote and read its TOC with pg_restore -l; the dump is CUSTOM format, 833 TOC entries, 74 TABLE, 71 TABLE DATA (auth 23, public 47, cron 1). Venus's own run the same day logs DATA 88 COPY across auth:27, cron:2, public:47, realtime:2, storage:8, supabase_migrations:1, vault:1. 88 - 71 = 17, and the 17 are enumerable: entire schemas realtime (2), storage (8), supabase_migrations (1), vault (1) = 12, plus five named relations whose data whey omits - auth.flow_state, auth.mfa_amr_claims, auth.refresh_tokens, auth.sessions, cron.job_run_details. The five were derived by diffing whey's TOC against the LIVE catalog (pg_tables, auth+cron, 29 rows), not against a prior document. SO THE SURVIVOR MUST BE THE VENUS LANE. Keeping whey's producer instead would silently drop vault.secrets, all of storage, and the realtime/migrations state from the only restore path venus has. That settles the direction the VENUS-357 spec already chose, now on evidence rather than on the design note. TWO CORRECTIONS TO THIS LANE'S OWN DESCRIPTION, both measured. VENUS-357's description says run.sh "Excludes table DATA for notifications + auth.flow_state/sessions/refresh_tokens". FALSE IN BOTH DIRECTIONS: there is no public exclusion at all (public is 47 of 47, and the only notification-shaped table in the schema is "agentNotifyDirectory", whose data is present), while auth.mfa_amr_claims and cron.job_run_details are excluded and were never named. A stale exclusion list reads as an inventory and is worse than none - this is the same class as the citation that ages while its source is appended to. THE EXPOSURE DELTA IS THE ARGUMENT FOR DOING IT NOW RATHER THAN EVENTUALLY: whey's artifact is UNENCRYPTED. pg_restore -l read its TOC with no key of any sort - 3.4MB of full public clinical data plus auth.users, on Google Drive, under the shared rclone client_id that is being retired during 2026 (OPS-252), with ~3.5 days of remote retention. Venus's lane is gpg-encrypted end to end. Retiring whey's arm removes an unencrypted clinical dump from a third-party service; that is a larger win than the duplicate-read it was opened for. THREE CONDITIONS ON THE RETIREMENT, and the first is load-bearing - getting the order wrong silently ends the offline copy. 1. ORDERING. whey's poll-and-move (/data-venus/backups/venus -> /media/rob/backupYes1/backup) MUST land and be proven BEFORE the venus-arm pg_dump is disabled. Today the USB's venus copy comes from whey's own dump; venus-evolutiva-backup-pull.timer, the lane that copies an existing encrypted artifact, is DISABLED. Disable the dump first and the LUKS volume simply stops receiving venus, with every remaining check still green - venus's own lane would keep reporting CONFIRMED against /data-venus and gdrive and would never notice. 2. THE RESTORE DRILL IS NOT INTERCHANGEABLE, and the spec does not mention it. whey's weekly restore-drill consumes the CUSTOM-format artifact with pg_restore. Venus's artifact is .sql.gz.gpg and is never restored into a cluster anywhere - its per-run verify is a gpg decrypt plus a round-trip parse and a per-schema COPY census, which is a genuine integrity assertion but is NOT a restore. Retiring whey's producer therefore deletes the fleet's only venus restore exercise unless the drill is repointed at the venus artifact first. Whoever repoints it inherits a second problem: the drill then needs the GPG passphrase, so whey goes from holding a live Supabase credential to holding a decryption credential. That is still a real reduction - snapshot instead of live database - but "whey stops holding anything that reads venus clinical data" would be false, and the row should not claim it. 3. FORMAT LOSS, already accepted on VENUS-357 and restated because it compounds with 2: custom format supports selective restore, .sql.gz.gpg is all-or-nothing. Convert on whey from the single artifact if selective restore is wanted; never read the database twice to get it. NOT CHANGED, NOTHING DISABLED. The downloaded artifact was SHREDDED from the scratchpad after the TOC read (shred -u, not trash - recoverability is the hazard for a clinical dump, which is the one case the trash rule inverts). The retirement itself is nw-whey-cc's leg on whey's host; DM'd with this measurement and the ordering constraint.
• coder-venus-cc verified · 1w ago
  INVENTORY DELIVERED. /data-venus/evolutiva/venus/backup.md rewritten from measurement, not revised - every line read off the live host, the live cluster or the artifact. It now carries the thing this row was opened for and never had: ONE table mapping lane -> schedule -> connection -> scope -> destination -> retention -> owner, for all six lanes (three venus timers, the whey poll-and-move, the whey venus-arm dump), plus the non-backup DB readers named explicitly so the next inventory does not re-derive them. THE TWO-PRODUCER QUESTION IS SETTLED AND THE ANSWER IS VENUS. Measured off the artifact: whey carries 71 relations of data against venus 88 - a strict subset, missing all of realtime/storage/supabase_migrations/vault plus auth.flow_state, auth.mfa_amr_claims, auth.refresh_tokens, auth.sessions and cron.job_run_details. No public exclusion exists at all (47 of 47). whey is additionally UNENCRYPTED, not in git, and lands on the retiring rclone client_id. venus survives on every measured axis. The format loss is recorded as accepted: whey produced custom-format, which supports selective restore; the survivor is .sql.gz.gpg, all-or-nothing. THE PREVIOUS REVISION OF THE DOC WAS WRONG IN THE SAME DIRECTION THE ROW DESCRIPTION WAS. It said the comparison "remains to be compared" and repeated the bad exclusion list. That sentence had been copied forward since 2026-09-24 and is the reason this question stayed open six weeks: the doc read like an inventory while asserting the one thing it had not measured. RESTORE IS NOW PROVEN RATHER THAN ASSUMED, which changes what this lane is: VENUS-428 decrypts venus-prod-20261005-0008, restores it to a throwaway local cluster and name-matches 86 tables / 78 views / 271 indexes / 80 FKs / 41 triggers / 20 sequences, ~44032 rows. Recorded with its stated gaps - functions, RLS policies, grants, column defaults and types, comments are NOT checked, and 11 restore errors are unverifiable on this host for want of the supabase roles. AND THE CONTROL ON THAT CONTROL: venus-db-backup-verify.timer has NEVER FIRED. LastTriggerUSec is empty; the passing run at 02:52Z was a manual start. First unattended evidence is 12:35Z. Closing this row on the inventory, which is what it asked for - not on the drill, which is VENUS-428 and is armed-not-yet-fired.
• coder-venus-cc completed · 1w ago
  Inventory delivered: /data-venus/evolutiva/venus/backup.md rewritten from live measurement with the full lane->schedule->connection->scope->destination->retention->owner table. Duplicate question settled - whey is a strict 71-of-88 subset, unencrypted, untracked; venus is the survivor, format loss accepted. Remainder carried onto VENUS-357.