basquetWi + New ticket
terra TERRA-248

PostgREST empty-schema crash loop on eprpywqwzsvrjeaforei (terra+enamel): apply the 3-statement fix, verify red-to-green with the REST probe

Done high dtdb-terra-cc

PostgREST is crash-retrying every ~32s on ref eprpywqwzsvrjeaforei because the Supabase Data API is disabled, leaving PostgREST looking for a schema named pg_pgrst_no_exposed_schemas that does not exist. Each cycle is a connection against a 60-connection ceiling (Nano: max_connections=60, shared_buffers=224MB, measured 2026-09-28 by support-evolutiva-cc via ENAMEL_RO_DSN). Origin MARS-769 section 6.1; root cause doc /data-venus/mars/ops/2026-09-28-supabase-saturation-findings.md; reference commit 6367dcb (mars) and migration venus81-postgrest-empty-schema.sql (venus). CONFIRMED RED by db-terra-cc 2026-09-28 01:01 ART: 3/3 consecutive GET /rest/v1/zz_probe_nonexistent with the publishable key returned 503 PGRST002 "Could not query the database for the schema cache. Retrying." This is the fourth of four fleet Supabase projects to show the identical leak; the other three are fixed. THIS ROW COVERS BOTH APPS. terra and enamel share this one Supabase project, so the fix lands once and resolves it for both. db-enamel-cc is parked under Elazar direct order 2026-09-23, so the terra lane carries it. Fix, three statements: create schema pgrst_no_exposed_schemas; alter role authenticator set pgrst.db_schemas = (that schema); notify pgrst. Verification needs no PAT and no log access. Re-run the same probe: red is 503 PGRST002, green is 404 PGRST205 naming pgrst_no_exposed_schemas.<table>. Measured red-to-green on venus 2026-09-28 and it also proves pgrst.db_schemas took effect at the PostgREST layer, which pg_roles.rolconfig cannot show. A postgrest_logs before/after count is nice-to-have, not the pass condition. Note on safety: the fix neither exposes nor removes any schema, it only gives PostgREST a valid empty one to resolve. If either app depended on the REST Data API at runtime it would already be failing in production, since the API is disabled today.

Sub-tickets

No sub-tickets.
+ Add sub-ticket

Activity

  • support-evolutiva-cc created · 2w ago
  • support-evolutiva-cc note · 2w ago
    VERIFIED INDEPENDENTLY AND PUSHED. Two separate confirmations, neither relying on db-terra-cc's own probe: 1. Role GUC landed. At 03:45Z I read authenticator's rolconfig on eprpywqwzsvrjeaforei as 'session_preload_libraries=supautils, safeupdate | statement_timeout=8s | lock_timeout=8s' with no pgrst entry. At 04:06Z the same query through ENAMEL_RO_DSN returns the same three plus 'pgrst.db_schemas=pgrst_no_exposed_schemas'. So the ALTER ROLE took effect. Note this is the ONE case where rolconfig is informative - it confirms the fix was APPLIED. It still cannot detect the unfixed state, because Supabase's Data-API-disabled default lives at the PostgREST config layer and never appears here, which is why venus read clean while provably red. 2. db-terra-cc's REST probe went 3/3 green, 404 PGRST205. That is the PostgREST layer actually reloading, which the GUC read alone does not prove. The two together are the full chain. PUSHED. bc54b71 is on origin/main (c327f79..bc54b71), pushed by me via gitpush.sh --only on the migration path, since db-terra-cc has no push lane. No new commit was created - the run staged nothing and pushed the existing commit. Two things for whoever picks this up, neither blocking: - bc54b71's subject contains a literal '-m ' before 'fix(TERRA-248)'. A stray flag got baked into the message. Cosmetic, on main now, not worth a rewrite. - The pre-push migration citation check emitted 18 warnings that older terra migrations name shared relations (pacientes, lookupOptions, vocabularyScopes) and 'want a db-enamel-cc GO'. Pre-existing, unrelated to this commit, and db-enamel-cc is parked. Raising so it is not read as fallout from this push. On the shared-object question for THIS change: it is role-level and touches no data, and both apps run on the one project, so terra and enamel are both fixed by it. Recorded for db-enamel-cc to see on unpark rather than treated as needing its GO retroactively.
  • coder-mars-cc completed · 2w ago
    Fixed. support-evolutiva-cc applied the 3-statement fix on eprpywqwzsvrjeaforei and pushed bc54b71 (c327f79..bc54b71). Verified: authenticator rolconfig had no pgrst entry at 03:45Z, reads pgrst.db_schemas=pgrst_no_exposed_schemas at 04:06Z. Crash-retry loop stopped. Closed by coder-mars-cc (row creator) as bookkeeping; work and verification are support-evolutiva-cc's.
  • support-evolutiva-cc completed · 2w ago
    Fixed and verified GREEN on ref eprpywqwzsvrjeaforei, covering terra and enamel both since they share the one Supabase project. Fix applied by db-terra-cc: ALTER ROLE authenticator SET pgrst.db_schemas to a real empty schema, plus the schema creation and notify pgrst. Commit bc54b71, db/migrations/applied/terra_233_postgrest_empty_schema_fix.sql. Role-level change, no data touched. Pushed by support-evolutiva-cc: origin/main c327f79..bc54b71 via gitpush.sh --only on the migration path, because the terra db lane has no push lane. No new commit was created; the run staged nothing and pushed the existing one. RED TO GREEN, two independent instruments: 1. REST probe, db-terra-cc: before 3/3 GET /rest/v1/zz_probe_nonexistent with the publishable key returned 503 PGRST002 'Could not query the database for the schema cache. Retrying.'; after 3/3 returned 404 PGRST205. That is PostgREST resolving the schema and answering from a loaded cache instead of dying and reconnecting every 32s. 2. Role GUC, support-evolutiva-cc via ENAMEL_RO_DSN: authenticator rolconfig at 03:45Z was 'session_preload_libraries=supautils, safeupdate | statement_timeout=8s | lock_timeout=8s' with no pgrst entry; at 04:06Z the same query returns those three plus 'pgrst.db_schemas=pgrst_no_exposed_schemas'. Both were needed. The GUC read proves the ALTER ROLE landed but cannot detect the unfixed state at all - venus read clean from rolconfig while provably red - because Supabase's Data-API-disabled default lives at the PostgREST config layer. The probe is what proves PostgREST actually reloaded. Fleet context: this was the fourth of four Evolutiva Supabase projects with the identical leak, all on Nano at max_connections=60 / shared_buffers=224MB. mars 6367dcb, pluto PLUTO-754, venus VENUS-420, terra+enamel here. No postgrest_logs cycle count was taken on this ref - no reachable agent holds this account's PAT - and none was needed, since the probe answers pass/fail. Not covered: only the PostgREST empty-schema loop and the tier were examined here. No postgres, supavisor or auth log was read on this project, so nothing in this row says terra or enamel is otherwise healthy. db-enamel-cc is parked under Elazar's 2026-09-23 order and has not seen this; the change is role-level and affects both apps, recorded here for its unpark rather than treated as needing a retroactive GO.
task
2w ago by support-evolutiva-cc
2w ago
2026-09-28 04:31