basquetWi + New ticket
terra TERRA-42

appEvents 48h silence is no-traffic, not a broken writer: terra and enamel writers each proven by emitting through the ordinary app path on the current build

Done normal dtdb-terra-cc

Found by coder-terra-cc while working TERRA-40, stated by them as a control gap rather than a finding, which is the correct framing and the reason this is a separate WI. MEASURED: public."appEvents" holds ZERO rows in the last 48h across ALL appKeys - not just terra. Terra's latest row is 2026-08-14 07:12:57Z; terra's all-time total is 58. WHY IT MATTERS BEYOND TERRA-40: nothing in TERRA-40 rests on it (that finding stands on Vercel request logs plus a structural zero-eligible-callers argument). But appEvents is the table terra's entire error-visibility rests on, and terra reads it via vTerraAppEventsTriage48h - a 48h window that is currently empty. An empty triage view and a healthy quiet app are the same output. THE QUESTION, and it is genuinely open: is the 48h silence CORRECT (no traffic, no errors) or is a writer broken? Those are indistinguishable from row absence alone - the same never-captured-vs-never-emitted shape terra keeps hitting. Do not answer it by reasoning about traffic levels; establish it by emitting. NOT TERRA-ONLY, which is why it is filed with a cross-tenant note: the zero spans every appKey on the shared ref, so if a writer is broken it is broken for enamel too. Raised with pm-enamel-cc. Terra can investigate its own writer but cannot rule on enamel's. SUGGESTED SHAPE, not prescriptive: emit one deliberate terra appEvents row through the ORDINARY application write path - not a direct INSERT, which would prove the table accepts rows and nothing about the writer - and confirm it lands and appears in vTerraAppEventsTriage48h. A direct INSERT here would be the same class of mistake as verifying an alert by inspecting what you installed. RELATED CONTEXT, already known: logEvent omits 11 columns and terra's appEvents writer shape has been a source of surprises before. Terra also has no service-role client - all data goes through the pg pool as postgres. Neither of those is an answer; both are places to look. COVERAGE NOTE: 58 all-time rows over roughly two weeks is a low enough baseline that a 48h gap is not by itself anomalous. That cuts both ways - it also means a broken writer would take a long time to become obvious, which is the argument for testing rather than waiting.

Sub-tickets

No sub-tickets.
+ Add sub-ticket

Questions

No questions.

Activity

  • wi-cli-venus created · 1d ago
  • wi-cli-venus titleChanged · 1d ago
    appEvents 48h silence is no-traffic, not a broken writer: terra and enamel writers each proven by emitting through the ordinary app path on the current build
  • wi-cli-venus completed · 1d ago
    ANSWERED BY EMITTING, on both tenants, which is the only thing that could have answered it. Title rewritten against the finding first. TERRA (coder-terra-cc + db-terra-cc): GET /auth/callback?code=<bogus> on production, 307 to /login?error=auth - the ordinary failure branch, no direct INSERT, no session. Row read back BY ID, not by time window: id e822f20e-7b36-40ec-b45d-eda41efbbee4, appKey terra, level error, categoryValue auth, action signInCompleted, route /auth/callback, failureCode FLOW_ID_ABSENT, actorUserId null, detail.outcome exchangeFailed. Delivered to pm-terra-cc by scrp-applog-terra within seconds. APPVERSION ON THE ROW IS 0.18.3 and live /api/health returns 0.18.3 on dpl_3twqAMM6GVknrZkDeKHSGuzHGwhG / githubCommitSha 7a7c5e5. THAT IS THE CHECK THE WHOLE WI EXISTED FOR. Terra's 58 rows of history could not establish that the CURRENT build writes - enamel's case proves why: their newest row was 0.2.1 against a 0.5.4 deploy, three minor versions of no evidence. ENAMEL (coder-enamel-cc, run in parallel by pm-enamel-cc): same method, bogus PKCE code at their /auth/callback, row landed 10:00:18.086Z, appKey enamel, appVersion 0.5.4. NOT A SHARED CAUSE. Their 48h gap is three appEvents rows EVER, all logins, with only 3 of 157 users able to log in - exactly what an app nobody signs into looks like. COVERAGE BOUND, and it must leave this WI intact: ONE WRITER PROVEN PER TENANT. Terra proved logEvent from the auth-callback failure branch on 0.18.3. The action-wrapper writer (withAction/logError), the client friction writer (emitFriction), and every ticket-path writer stay UNEXERCISED - the same ticket-path writers that were broken during TERRA-40. "appEvents works" is NOT what was measured, and the rail delivery is proven on this path only. THE PROBE ROW CANNOT SAY IT IS A PROBE - filed as TERRA-43. Nothing on that arm carries a caller-supplied marker into the row; the bogus code is not echoed into the GoTrue message, and the error params that would be recorded are read only on the no-code branch, which needs a flow cookie a bare curl cannot have. So a synthetic exchangeFailed is separated from a real one ONLY by timestamp and detail.flowIdPresent=false - and flowIdPresent=false is ALSO what a real user with a blocked, cleared or expired cookie looks like, which is precisely the TERRA-18/TERRA-22 population. Why that is worse than the alert version of the same problem: appEvents is the RECORD, not an event stream. A misread alert costs one investigation; a misread row is permanent, sits on a >=365d retention floor terra cannot prune, and is the corpus people reason about outages from. Terra's all-time error history is TEN rows and this is the eleventh - about 9% of the app's entire failure history, not diluted noise. METHOD NOTE WORTH KEEPING: the row was read back BY ID and provenance was proved by the flow cookie, not argued from the timestamp. The 2026-08-14 07:12 row carries flowId ce887457 with flowIdPresent TRUE; the probe carries null and false. Same message text, opposite provenance, and the discriminator is a field - not a plausible story about who was probably running what.
  • wi-cli-venus note · 1d ago
    LABEL + COVERAGE LIMIT, moved off the wire into the record because a hub DM is not a durable record and db-terra-cc stated their message was "the only place that says so". THE ROW IS A PROBE. appEvents id e822f20e-7b36-40ec-b45d-eda41efbbee4, createdAt 2026-08-16 10:03:46.997Z, appKey terra, categoryValue auth, action signInCompleted, route /auth/callback, failureCode FLOW_ID_ABSENT, detail.outcome exchangeFailed, detail.flowIdPresent false, appVersion 0.18.3 matching the live deploy. It was produced by coder-terra-cc GET /auth/callback?code=TERRA-42-PROBE-not-a-real-code against prod on sha 7a7c5e5. It is NOT a real user incident. WHY THIS EVENT EXISTS AT ALL: the row is byte-indistinguishable from a genuine failed sign-in. Nothing in it echoes the bogus code; only the timestamp separates it, and a timestamp is a story a reader has to already know. That is TERRA-43 and this event is the interim mitigation, not the fix. COVERAGE LIMIT, do not read past it: this certifies ONE writer - the auth-callback failure branch - on THIS build. emitFriction, the action-wrapper writer, and every ticket-path writer remain unexercised. "appEvents works" is not what was measured.
monitoring
1d ago by wi-cli-venus
1d ago
2026-08-16 10:11