webusage
WEBUSAGE-1
Operational baseline for the live EVO-51 pipeline
Done normal
cecoder-enamel-cc
Maintenance home for the already-live web-usage telemetry pipeline: ingest service (/opt/telemetry-ingest/server.mjs + telemetry-ingest.service), nightly rollup+purge (rollup.sh + telemetry-webusage-rollup.timer), webusage schema tables (usageEvents append-only + usageDaily rollup), per-app bearer token registry (.env), repo github.com/ElazarPimentel/telemetry-ingest. Registry row 28. Ongoing ops/maintenance land here so the pipeline stops being untracked (it had no WI because 'EVO-51' collides with EVO-51 nav-link normalization).
Activity
-
8b691b4 telemetry-ingest v0.2.0 - rollup.sh now asserts RETENTION HELD and ROLLUP FAITHFUL in-process, non-zero exit. Live green: 0 overdue rows / table not empty / oldest raw 5.2d BEFORE the cutoff / 0 unfaithful day aggregates. Forced red: cutoff 80d -> 1496 overdue; cutoff 1h after oldest row -> 4 overdue while the first draft's rounded-age gate read 0.0 and passed (that draft was replaced - the comparison is an exact SQL count, nothing branches on the age); rolled totals +1 -> 254 unfaithful pairs. Validated read-only, without running the mutating job. /opt/telemetry-ingest IS the checkout, so the nightly telemetry-webusage-rollup.timer run carries it.
-
Done: telemetry-ingest 8b691b4/v0.2.0. rollup.sh asserts RETENTION HELD + ROLLUP FAITHFUL, non-zero exit on violation, failnotify wired. Forced-red verified (1496 overdue @80d cutoff, 254 unfaithful pairs). First live run 2026-10-05 03:00 UTC.
-
Token registry now mars,pluto,venus,terra (terra added 2026-10-05 for TERRA-270). Issuing it exposed a push hazard, fixed in 7c14d74 / v0.2.1: .gitignore listed .env alone, which does not match a .env.bak-<stamp> sidecar holding the same DSN and the same full token registry, and this repo takes UNSCOPED pushes - one git add -A behind that file publishes the registry. Now .env.* too; forced red (identical sidecar not ignored before, ignored after, invisible to git status), backup trashed.
-
Two corrections/additions after the close. (1) FIRST LIVE RUN IS 2026-10-06 03:00 UTC, not today's: today's timer fired 03:00:26Z and the baseline pushed at 03:51Z, so no run has exercised the two new gates yet - they were validated read-only. (2) Token registry is now mars,pluto,venus,terra (terra added 2026-10-05 for TERRA-270). Issuing it exposed a push hazard, fixed in 7c14d74 / v0.2.1: .gitignore listed .env alone, which does not match a .env.bak-<stamp> sidecar holding the same DSN and the same full token registry, and this repo takes UNSCOPED pushes - one git add -A behind that file publishes the registry. Now .env.* too; forced red (identical sidecar not ignored before, ignored after, invisible to git status), backup trashed.
-
b7dcfb1 + ffc4535, v0.3.0: rejections are now counted and logged. Before this every console.* in the ingest was about an insert or the pool, so 401/403/400 left NO trace anywhere - coder-terra-cc asked mid-bring-up for 'any 400/401 for app=terra' and the honest answer was that the question had no instrument. Added rejected401/403/400 + lastRejectionAt/Reason, byApp.{app}.{accepted,rejected403,rejected400} in /healthz, and one rejectWith() as the sole path to the journal. byApp is keyed ONLY by registry app names (body.app is caller text - a free-keyed map is an unbounded memory write driven by any valid-token holder). NOT folded into /healthz ok: a scanner 401 is not an outage and folding it in trains the hourly reader to ignore the counter that is; health-check.sh keys off insertBatchesFailed/eventsLost/poolErrors, unaffected. Care point: validateEvent's old messages CARRIED THE OFFENDING VALUE ('...raw id: 12345'), harmless as a return value and unacceptable as a log line, since that segment is the raw id the check exists to keep out; codes are now field+rule only. Forced red on all 7 arms with distinct codes, per-app counters landing on terra (1x403, 5x400), journal grep for the rejected raw id and the forged app name returns ZERO, and no probe wrote a row (terra still 0).
task
2026-07-17 by wi-cli-venus
1w ago
2026-10-05 03:51