ENAMEL-37 timer half: schedule fn_runAppEventsPiiScan and alarm on BOTH ok=false and staleness
enamel_051 builds the DETECTOR half of ENAMEL-37: fn_runAppEventsPiiScan writes one opsProbeRuns heartbeat per run carrying ok/findings/coverage, and vOpsProbeLatest exposes age. Nothing invokes it and nothing reads the age - so today the detector is installed and permanently silent, which is the exact failure shape the ruling required it be distinguishable from. TWO ALARM CONDITIONS, and they must not be collapsed into one boolean: (1) ok=false on the latest row - a shape matched, route to the PM of the appKey that owns the flagged rows, not to enamel by default, since appEvents is co-tenanted and terra writes 334 of 365 rows; (2) NO FRESH ROW within the expected interval - covers timer never installed, timer dead, service_role grant dropped, and the scan raising (the runner's INSERT is downstream of the scan, so an exception leaves no row at all). Condition 2 has no invoker of its own inside the DB and must not be checked by anything that shares an invoker with the probe - see the ENAMEL-32 alarm, which is a venus user timer for exactly that reason and whose own liveness gap this is the same class of. Likely shape: a venus user timer reading vOpsProbeLatest through a narrow read-only role, mirroring ops/enamel-outbox-age-alarm. NOT satisfied by enamel_051 and the migration header says so.
Activity
-
parent=#3679
-
DESIGN, with the reach measured first (db-enamel-cc, live on eprpywqwzsvrjeaforei, 2026-09-11). No file written yet, nothing applied. MEASURED CONSTRAINT THAT DECIDES THE SHAPE: of every login role on this ref - applog_writer, audit_terra_cc_ro, authenticator, db_terra_cc_ro, enamel_outbox_alarm_ro - NOT ONE can execute fn_runAppEventsPiiScan and NOT ONE can read vOpsProbeLatest. The only principal that can do either is service_role, which is NOLOGIN and is reachable only through PostgREST's SET ROLE or a service-role API key. So the timer half cannot be built by pointing an existing role at the probe; it needs new grants either way, and that is the part that lives in my lane. THREE POSSIBLE INVOKERS, and why the third wins: (a) pg_cron inside the database. Blocked on ENAMEL-41 (the pg_net/cron ACL ask is Elazar's to send and unresolved), and cron.job_run_details carries PUBLIC SELECT *and DELETE* on this ref, so the audit trail of the runs is erasable by any login role. Not a base to build an alarm on. (b) A Vercel route driven by a host timer. This is the exact shape ENAMEL-32 rejected and the rejection still holds: the EVO-94 host-timer chain does not exist for enamel, so the route would be invoked by nothing and would be silent in precisely the case it was built for. (c) A venus user timer with a NEW narrowly-granted login role. Matches the precedent that is already working in production for the outbox drain and its alarm. This is the shape. TWO ROLES, NOT ONE, AND THAT IS THE WHOLE POINT OF THE ROW. An alarm must not share its subject's invoker (ENAMEL-32). So: - enamel_pii_scan_runner: EXECUTE on fn_runAppEventsPiiScan and nothing else. It runs the scan and writes the heartbeat. It cannot read the results. - enamel_pii_alarm_ro: SELECT on vOpsProbeLatest and nothing else. It reads ok and age. It cannot run the scan. Separate roles, separate units, separate timers. If the runner role is dropped, its grant revoked, or its unit stopped, the alarm still runs and sees no fresh row - which is the dead-probe condition firing correctly. If the alarm is the thing that dies, see the residual below. TWO ALARM CONDITIONS, NEVER ONE BOOLEAN, per the ruling this row descends from: - ok=false: at least one shape matched. Routes to the PM of aro:<appKey> of the flagged appKey, read live from the hub, because a terra finding is terra's PM's to act on. Never carries the finding's content - sampleEventIds at most, which is all the findings column holds anyway by construction. - STALE: no fresh row. One condition covering every cause - timer never installed, timer dead, role dropped, grant revoked, the scan RAISED - because the runner's INSERT is downstream of the scan. Threshold is a multiple of the run interval, stated in the unit, not inferred. State is once-then-suppress per condition, per appKey for the ok=false leg, same as the outbox alarm - a shared flag lets one appKey's alarm suppress another's first fire. RESIDUAL, NAMED RATHER THAN IMPLIED, because this row exists because the last one named its gap honestly: both units are venus user timers, so venus being down silences both, and nothing watches the ALARM's own liveness. Writing an alarm-side heartbeat only moves the regress one level. It terminates at a different system or it does not terminate; what this row will NOT do is pretend it closed. DB HALF IS MINE AND IS THE PREREQUISITE: a migration minting the two roles with those two grants and asserting REACH both directions (each role reaches its one object, neither reaches the other's, neither reaches appEvents or pacientes at all). Role work is Class-A under db/per-tenant-role-design.md, so the shape goes to design review before any DDL is authored. Passwords are minted by hand post-apply and the DSNs travel by DM - never a repo artifact, per the enamel_050 precedent. UNIT HALF is a venus user timer pair in the enamel repo under ops/, symlinked into ~/.config/systemd/user/, DSN and hub bearer from the .llmmsg-env overlay. Same layout as ops/enamel-outbox-age-alarm.*. SEQUENCED BEHIND enamel_052 (ENAMEL-39), which is written, dry-run passed, audit-PASSed and waiting on db-terra-cc's verdict.
-
Class-A design review, shape only, no DDL yet. Marker E43-audit-design-20260911-tworoles. MEASURED-FIRST claim independently re-run against the live catalog: confirmed exactly - none of applog_writer/audit_terra_cc_ro/authenticator/db_terra_cc_ro/enamel_outbox_alarm_ro can EXECUTE fn_runAppEventsPiiScan or SELECT vOpsProbeLatest; only postgres/service_role can, service_role NOLOGIN. (Note: your 'every login role on this ref' is a curated app-tier list, not literally every pg_roles row with rolcanlogin - the platform-internal supabase_* roles and pgbouncer/venusApp exist too, none of them relevant to this decision, just flagging the wording.) Q1 (two roles vs one): TWO ROLES IS RIGHT, agree with your reasoning and go further - the failure mode isn't just 'one revoked grant takes down both halves', it's that a single credential holding both EXECUTE(scan) and SELECT(results) means a compromised or misused DSN can both write a fake-healthy heartbeat AND read to confirm it looks clean, collapsing the alarm's independence from its subject entirely. That's the same 'must not be its own witness' doctrine enamel_051's own header already commits to for content-vs-heartbeat; this is the identity-level version of it. One role + two units does not satisfy ENAMEL-32 because the unit boundary isn't the trust boundary, the credential is. Q2 (venus-timer single point of failure, nothing watches the alarm's liveness): AGREE - name it, do not chase the regress. This is the same shape ENAMEL-32 already accepted (an alarm-side heartbeat only moves the gap one level) and enamel_051's own header names for the scan side ('until a timer exists... an unscheduled probe IS stale, and that is the probe working, not failing'). I have no host-external mechanism in scope to terminate it elsewhere - naming it in the header, same treatment as ENAMEL-32, is the right and sufficient answer here. ONE CORRECTION, independently measured, same gap-class as the enamel_052 comment defect from earlier today: the pg_cron rejection reason 'cron.job_run_details carries PUBLIC SELECT and DELETE here, so the run history is erasable by any login role' is WRONG as stated. relacl on cron.job_run_details does carry a PUBLIC '=rd/supabase_admin' entry, confirmed - but schema-level USAGE on cron (pg_namespace.nspacl) is granted ONLY to postgres and supabase_admin, and has_schema_privilege confirms every other login role including service_role gets 'permission denied for schema cron' before the table grant is ever reached. So the table-level PUBLIC grant is real but unreachable by any role that isn't already postgres/supabase_admin-equivalent - i.e. already able to delete that history by ownership/superuser power regardless of this ACL. This does NOT change the recommendation (the venus-timer approach is independently justified by the ENAMEL-32 invoker-separation precedent and by ENAMEL-41's documented erosion of this ref's extension-ACL hardening, which stands on its own), but the job_run_details claim should not be written into the row as measured fact - it is the identical 'grant list is not reach, check the USAGE/membership hop too' mistake TERRA-234 names, just found in a rejection argument instead of an acceptance one this time.
-
DESIGN REVIEW RULED (coder-enamel-cc, enamel PM seat, 2026-09-11). GO on the shape, with one required addition to the acceptance block and one correction to the measurement. No DDL authorised beyond what is described here. Q1 - TWO ROLES, CONFIRMED, AND THE STATED REASON IS NOT THE DECISIVE ONE. The proposal argues fault isolation: one revoked grant takes down both halves. True, and secondary. THE INVOKER IN THE ENAMEL-32 RULE IS THE ROLE, NOT THE UNIT. One role with both grants plus two units does NOT satisfy it, because both units then authenticate with the SAME credential, so every credential-level failure - password expired, role dropped, connection limit hit, DSN rotated wrong in the overlay, pgbouncer refusing that user - silences the subject AND the alarm in the same instant. The alarm cannot report an outage it is inside of, and a silent alarm is indistinguishable from a clean scan. That is the failure the rule exists for and unit separation does not touch it. Two roles, two credentials, two units, two timers. Q2 - NAME THE RESIDUAL, AND DO NOT BUILD THE HEARTBEAT YET. The intent to name rather than pretend is correct and it is not sufficient on its own: the measured precedent on this project is the applog error rail, which was down 16.8 days with nobody noticing, and its gap was documented. A residual whose only home is a migration header has already failed once here. But the proposal collapses two cases that behave differently. VENUS DOWN silences both units, and an alarm-side heartbeat does not close that - correct as stated. ALARM DEAD ALONE, host up, is a different case: bad DSN, expired password, unit masked or failed, and a heartbeat WOULD close it. That second case is the one that actually happened on applog - a single dead consumer with the host healthy, not a host outage. So the case a heartbeat closes is the likely one, which argues for building it. IT STILL DOES NOT GET BUILT IN THIS MIGRATION, for a reason that is the applog lesson rather than a cost argument: A HEARTBEAT NOBODY READS IS NOT AN INSTRUMENT. An alarm-side row in opsProbeRuns is only liveness if something off-venus reads it, and no such reader exists for enamel today. Specify the READER first; the writer is trivial once the reader is real, and building the writer first produces exactly the rail-with-no-consumer that cost 16.8 days. Filed separately so it is a row and not a sentence. Also note the write is not free here: service_role holds SELECT on opsProbeRuns and NOT INSERT, so the runner heartbeat rides fn_runAppEventsPiiScan being SECURITY DEFINER (verified: prosecdef=true, owner postgres, search_path=public,pg_temp, EXECUTE to postgres+service_role only). An alarm-side heartbeat therefore needs its own DEFINER function or a direct INSERT grant, and the second widens the alarm role past read-only. Neither belongs in a role-minting migration. REQUIRED ADDITION TO THE ACCEPTANCE BLOCK, AND THIS IS THE PART THAT CHANGES THE FILE. The stated assertion is that neither role reaches appEvents or pacientes at all. MEASURED: vOpsProbeLatest has reloptions NULL, so security_invoker is UNSET and defaults FALSE, and the view is owned by postgres - the same owner as opsProbeRuns. The view therefore executes with the OWNER rights, not the caller rights. Two consequences. (i) The design works as written: a bare SELECT grant on the view IS sufficient and enamel_pii_alarm_ro needs no grant on opsProbeRuns - do not add one, it would widen the role from the latest row to the whole run history across every appKey. (ii) THE NO-REACH ASSERTION IS NOT ENFORCEABLE BY GRANTS. Reach through a non-invoker view is governed by the view DEFINITION, which lives outside the acceptance block. The day anyone adds a column sourced from appEvents to vOpsProbeLatest, enamel_pii_alarm_ro silently gains appEvents reach with no grant change and every has_table_privilege assertion in the file still returns false. This is the TERRA-234 rule in its inverse direction: a view ACL says who may ask, never what is reachable. SO ASSERT THE SOURCE SET, NOT ONLY THE PRIVILEGES: resolve the relations vOpsProbeLatest actually depends on through pg_depend joined to pg_rewrite, and assert it is EXACTLY the set containing opsProbeRuns and nothing else. That is a catalog read, it belongs in the file, it costs one query, and it raises if the view is ever widened. Without it the file certifies a grant list and reads as if it certified reach. Keep the privilege assertions too - both directions, each role reaches its one object and not the others - this is additional, not a replacement. CORRECTION TO THE MEASUREMENT, does not change the shape. NOT ONE login role can read vOpsProbeLatest is false as written. Re-measured across all 18 roles: supabase_read_only_user and supabase_etl_admin both hold SELECT on vOpsProbeLatest AND on opsProbeRuns, and both can log in. True statement is that no ENAMEL-OR-TERRA application role can - applog_writer, audit_terra_cc_ro, authenticator, db_terra_cc_ro, enamel_outbox_alarm_ro, venusApp and pgbouncer are all false on every axis. Neither platform role is usable as a timer invoker so the conclusion stands, but an absence claim that names five roles when the ref carries eighteen is the shape that has been wrong here before. Confirmed true as measured: no role but postgres, supabase_admin and service_role can EXECUTE fn_runAppEventsPiiScan, and service_role is NOLOGIN. REJECTIONS RATIFIED. pg_cron: agreed, and the DELETE half is the stronger half of that argument - cron.job_run_details carrying PUBLIC DELETE means the run history of the alarm is erasable by any login role, so the instrument could be silenced without trace. Vercel route: agreed, ENAMEL-32s rejection holds for the same reason it did then. UNCHANGED AND RATIFIED: two alarm conditions never collapsed to one boolean; ok=false routed to the PM of the flagged appKey read live from the hub, never carrying finding content; STALE threshold stated in the unit as a multiple of the interval, not inferred; once-then-suppress state per condition AND per appKey; passwords minted by hand post-apply with DSNs by DM and never a repo artifact, per enamel_050; sequenced behind enamel_052.
-
DB half authored: db/migrations/enamel_053_pii_scan_timer_roles.sql. Both design reviews GO'd two roles; every change either review asked for is folded in. Held for Class-A audit pre-apply review, not applied.
-
DDL AUTHORED AND DRY-RUN, NOT APPLIED: db/migrations/enamel_053_pii_scan_timer_roles.sql. Mints enamel_pii_scan_runner (EXECUTE on fn_runAppEventsPiiScan() only) and enamel_pii_alarm_ro (SELECT on vOpsProbeLatest only), LOGIN NOINHERIT NOBYPASSRLS, REVOKE-then-GRANT convergence, passwords by hand post-apply. THE REQUIRED ADDITION IS IN: ACC-VIEW resolves vOpsProbeLatest's source set through pg_rewrite joined to pg_depend with refclassid pinned to pg_class (an oid is unique only WITHIN a catalog, so a join on refobjid alone can match an unrelated relation) and asserts it equals exactly {public.opsProbeRuns}. It also asserts the two properties the bare-view-grant design depends on and that no privilege assertion can see: reloptions carries no security_invoker, and the view owner equals the base table owner. No grant on opsProbeRuns to either role. THE SAME HOLE ON THE RUNNER SIDE, which neither review named, is named in the header rather than left to be found: EXECUTE on a SECURITY DEFINER function IS reach to everything its body touches. The runner's DIRECT relation reach is asserted EXACTLY EMPTY; its INDIRECT reach is the function body's - appEvents R, pacientes R id-only, opsProbeRuns W - and is governed by enamel_051, not by this file. ACC-FN asserts prosecdef and proowner for that reason: CREATE OR REPLACE drops SECURITY DEFINER silently unless restated (023B), and without it the heartbeat INSERT cannot succeed for any grantee, since service_role holds SELECT and not INSERT on opsProbeRuns. CONTROLS FORCED TO FAIL BEFORE REVIEW, per the rule that a control never seen to fail is a green line: (a) widened the view with a column sourced from appEvents -> ACC-VIEW raised naming {public.appEvents,public.opsProbeRuns}; (b) granted the runner one extra EXECUTE -> ACC-RUNNER-2 raised naming both signatures; (c) granted the runner SELECT on the view -> ACC-RUNNER-1 raised. Unforced run passes clean. Residue after the rollbacks: to_regrole null for both roles, view definition unpolluted. Exact-set arms verified safe first: zero relations in public carry a PUBLIC SELECT grant and zero public functions are PUBLIC-executable today. NO SET ROLE BEHAVIOURAL LEG IN-FILE, deliberately: PG17 gives the creating role implicit ADMIN membership only on the CREATE branch, so the leg would pass on a first apply and raise on the else-alter-role replay path. A leg that works on one of two branches is worse than none because it passes. The real leg is a client connection as each role post-apply, and that is reported in the apply message.
-
TWO MEASUREMENT CORRECTIONS TO MY OWN DESIGN NOTE ON THIS ROW, both independently re-measured by me before accepting, both written into enamel_053's header. 1. 'Of every login role on this ref NOT ONE can execute fn_runAppEventsPiiScan or read vOpsProbeLatest' is FALSE as written. The ref carries fifteen login roles; I named five. supabase_read_only_user and supabase_etl_admin both hold SELECT on vOpsProbeLatest AND on opsProbeRuns and both can log in. TRUE STATEMENT: no enamel-or-terra APPLICATION role can reach either object - applog_writer, audit_terra_cc_ro, authenticator, db_terra_cc_ro, enamel_outbox_alarm_ro, venusApp, pgbouncer all false on every axis - and no role but postgres, supabase_admin and service_role can EXECUTE the scan, service_role NOLOGIN. The conclusion is unchanged; the absence claim was quantified wider than it was measured. Caught by coder-enamel-cc. 2. The pg_cron rejection reason 'cron.job_run_details carries PUBLIC SELECT and DELETE, so the run history is erasable by any login role' is FALSE. The table-level PUBLIC grant is real - I read it right. Reach is what I got wrong: pg_namespace.nspacl on cron is {supabase_admin=UC/supabase_admin,postgres=U*/supabase_admin}, so every other role including service_role is refused AT THE SCHEMA before the table grant is evaluated. has_schema_privilege across all fifteen login roles: true only for postgres, supabase_admin, supabase_read_only_user, supabase_etl_admin. Caught by audit-terra-cc, who also named the class: it is the identical grant-list-is-not-reach mistake TERRA-234 exists to prevent, found in a REJECTION argument instead of an acceptance one. THE PG_CRON REJECTION STILL STANDS on its remaining legs - ENAMEL-41 unresolved and Elazar's to send, and the ENAMEL-32 invoker-separation precedent, which is independent of either. The same wrong claim is in enamel_050's footer at lines 296-299, which is applied and pushed. Corrected in enamel_053's header rather than by editing a pushed migration, and corrected in memory.
-
PASS pre-apply (audit-terra-cc-mtx6dgou4nmb) on enamel_053_pii_scan_timer_roles.sql. Independently reproduced live: vOpsProbeLatest owner/security_invoker/source-set, fn_runAppEventsPiiScan prosecdef/proowner, opsProbeRuns relacl, the corrected 15-login-role reach matrix (supabase_read_only_user + supabase_etl_admin are the only reach besides postgres/supabase_admin), cron nspacl, net schema/table PUBLIC grants - all exact matches to the file's claims. Two-role/two-credential design, exact-set assert style, refclassid-pinned view-source check, identity-signature EXECUTE check all sound. No defects. GO to apply.
-
DB HALF APPLIED AND VERIFIED LIVE. enamel_053 applied from the file (never from recall), audit-terra-cc PASS marker E43-audit-preapply-20260911, no defects, GO to apply. Push is QUEUED behind the venus-host.push-gate, held by coder-terra-cc; not polling, waiting on the hub's free notice. CATALOG READ-BACK: both roles exist with attributes canlogin=1 inherit=0 super=0 bypassrls=0 createrole=0 createdb=0 replication=0; both comments present. enamel_pii_scan_runner direct table reach EMPTY on every privilege including MAINTAIN, EXECUTE reach exactly {fn_runAppEventsPiiScan()}. enamel_pii_alarm_ro SELECT reach exactly {vOpsProbeLatest}, write reach EMPTY, EXECUTE reach EMPTY. Cross arms both false in both directions. BEHAVIOURAL LEG OVER REAL CLIENT CONNECTIONS, which is the part no acceptance block can do - connected as each role through the session pooler with the minted password: runner: select fn_runAppEventsPiiScan() -> RAN. ok=true, findings=[], coverage byAppKey terra 336 / enamel 31. One fresh opsProbeRuns heartbeat written, so vOpsProbeLatest is now 4s old instead of unbounded. runner: select 1 from vOpsProbeLatest limit 1 -> permission denied for view vOpsProbeLatest runner: select 1 from appEvents limit 1 -> permission denied for table appEvents alarm: select probe, age, ok from vOpsProbeLatest -> appEvents:pii | 4s | t alarm: select 1 from opsProbeRuns limit 1 -> permission denied for table opsProbeRuns alarm: select fn_runAppEventsPiiScan() -> permission denied for function fn_runAppEventsPiiScan alarm: delete from opsProbeRuns where false -> permission denied for table opsProbeRuns Every denial probe PARSES before it is refused (no column names, or a where-false delete), so each one measured the ACL and not the parser - the enamel_050 lesson. That last line is the whole point of the two roles, demonstrated rather than argued: the credential that runs the scan cannot see whether the scan found anything, and the credential that reads the verdict cannot produce one. PASSWORDS minted by hand post-apply through the query endpoint, never through apply_migration, so they are in neither the repo nor migration history. Both DSNs written to /home/rob/.local/bin/.llmmsg-env (mode 600, git-ignored - verified with git check-ignore, .gitignore:6) as ENAMEL_PII_SCAN_DSN and ENAMEL_PII_ALARM_DSN. No DM was needed: the unit half of this row is mine, so the consumer is on this host. ROW STAYS OPEN. The DB half is done; ENAMEL-43 is the TIMER half and the venus user timer pair under ops/ does not exist yet. Until it does, vOpsProbeLatest holds one row whose age grows without bound - correctly, per enamel_051's header. ENAMEL-44 (coder-enamel-cc) carries the alarm-liveness reader and is deliberately not folded in here.
-
DB HALF PUSHED: 1041984, app 0.10.7, db/migrations/enamel_053_pii_scan_timer_roles.sql (450 lines). Scoped --only push; app/package.json rode along as the version bump (known gitpush behaviour) and app/pnpm-lock.yaml, which is another lane's edit, did NOT. venus-host.push-gate taken before the run and released by me afterwards - gitpush declined to release it because I held it before the run, which is the OPS-210 behaviour working as documented. audit-terra-cc PASS marker E43-audit-preapply-20260911 also on this row. ROW STAYS OPEN: the venus user timer pair under ops/ is the remaining half.
-
Correction to the cron.job_run_details reasoning cited in my PASS on enamel_053: USAGE on cron schema is NOT limited to {supabase_admin,postgres} per nspacl - pg_read_all_data grants implicit USAGE on ALL schemas (predefined-role definition, invisible to nspacl), reaching postgres/supabase_etl_admin/supabase_read_only_user/pg_write_all_data via membership. Independently confirmed live (has_schema_privilege, pg_auth_members, relacl showing PUBLIC=rd on job_run_details). Six roles reach DELETE on job_run_details, not two. pg_cron rejection decision itself is unaffected (stands on ENAMEL-41/ENAMEL-32). Marker: mtx7f0t5ctcm
-
TIMER HALF BUILT, INSTALLED, ENABLED AND FORCED TO FAIL ON EVERY LEG. Five files under ops/: enamel-pii-scan-runner.{service,timer} and enamel-pii-scan-alarm.{sh,service,timer}, plus the ops/README.md contents table. NOT YET PUSHED - venus-host.push-gate is held by coder02-mars-cc; registered for the release notice, not polling. RUNNER is a unit with NO SCRIPT and no alarming of its own, deliberately: every way it can fail (timer never enabled, timer dead, psql missing, DSN unset, EXECUTE grant dropped, the scan raising) produces the SAME observable - no fresh opsProbeRuns row - and the alarm's staleness leg catches all of them without needing to know which. A notifier inside the runner would only add a second breakable thing that presents as the thing telling you something is broken, and it would share the runner's invoker. OnCalendar=*-*-* 04:23:00 UTC, Persistent=true, no RandomizedDelaySec (a smear would have to be added to the staleness threshold, turning one number into two). 04:23 is off the crowded 03:00/03:17/07:30 UTC rails. CADENCE AND THRESHOLD ARE ONE DECISION IN TWO FILES and both files say so: 24h scan, 26h threshold, two hours of slack. The WI says 'expected interval' without fixing one; the timer file is where it is now fixed. ALARM reads vOpsProbeLatest as enamel_pii_alarm_ro, hourly at :37 (off the contended top of the hour), and raises THREE independent alarms under three separate state keys, which is what 'must not be collapsed into one boolean' means operationally: _stale, finding_<appKey>, and _probe for the alarm's own failures. The ZERO-ROW case is filed under _stale, not _probe - a successful query returning no row means the scan has never run, and blaming the reader for the runner's absence would file the real fault under a key nobody watches. A STALE run EXITS before evaluating ok/findings: reading them would attach a fresh-looking verdict to stale evidence. EVERY LEG FORCED TO FIRE AGAINST THE LIVE HUB BEFORE THE TIMERS WERE ENABLED - a control never seen to fail is a green line: PROBE BROKEN (DSN unset) -> DM delivered, rc=1, _probe=ALARM PROBE RECOVERED (DSN restored) -> DM delivered, _probe=OK STALE (threshold forced to 1 against a 445-min row) -> DM delivered, _stale=ALARM SUPPRESSION (same threshold, second run) -> NOTHING sent, as designed FRESH AGAIN (threshold restored) -> DM delivered, _stale=OK FOUND -> see fixture below; DM delivered, finding_enamel=ALARM SUPPRESSION (second run) -> NOTHING sent CLEARED (fixture deleted) -> DM delivered, state file REMOVED so a recovered appKey does not linger FORCED-FAILURE FIXTURE, RECORDED BECAUSE IT WAS A PRODUCTION WRITE: neither role can produce an ok=false row, so one row was INSERTED into public.opsProbeRuns through the Management API query endpoint - id fc23c80f-0022-4e4a-9d1c-c072679868a3, probe appEvents:pii, ranAt now(), ok=false, findings [{shape:fixture,appKey:enamel,n:1,sampleEventIds:[]}], note naming it a forced-failure fixture. DELETED in the same session, id returned by the delete, and the table re-checked: zero rows matching the fixture note. vOpsProbeLatest confirmed restored to the real row (ok=t, findings []). PER-appKey ROUTING verified without spamming other PMs: the FOUND leg ran with ENAMEL_PII_ALARM_TO overridden to me, and resolve_recipients was exercised separately as a hub READ - aro:enamel -> coder-enamel-cc, aro:terra -> coder-terra-cc, aro:nosuchapp -> no pm, which is the documented fallback path. So the appKey->PM derivation is live, not assumed. RUNNER EXERCISED UNDER SYSTEMD, not just by hand: systemctl --user start enamel-pii-scan-runner.service -> Result=success, ExecMainStatus=0, and vOpsProbeLatest read back a FRESH row at 2026-09-12T00:01:05Z, age 0, ok=true, findings [], 387 rows scanned, coverage byAppKey terra 354 / enamel 33. That is the ExecStart quoting under test, since the SQL carries embedded double quotes. Both timers enabled: alarm next at 00:37 UTC, runner next at 04:23 UTC. shellcheck -S warning clean. Alarm unit also started once in PRODUCTION configuration (real state dir, no literal recipient override): Result=success, no DMs, no state written - correct, there were no transitions. COVERAGE, stated as a gap and not implied away: this pair SHARES AN INVOKER - both are venus user timers in the same systemd instance reading the same ~/.local/bin/.llmmsg-env - so a failure of that instance or that file takes out the watcher and the watched together. They are separated on credential (two roles, verified: the alarm role cannot read opsProbeRuns or appEvents or call the scan fn), on cadence and on state, and NOT on the axis that would need a second host. That residual is ENAMEL-44 and nothing here closes it. Also not covered: this alarm's own liveness, what the scan looks for (enamel_051's), and any finding that appears and clears between two hourly reads (vOpsProbeLatest is DISTINCT ON (probe)). TRANSPORT: both DSNs now carry sslmode=verify-full plus an absolute sslrootcert. Controls A (nonexistent CA path), A' (existent but WRONG CA - self-signed) and B (connect by IP, hostname cannot match) all REFUSE on both DSNs; live connections unchanged at TLSv1.3.
-
Done: PII scan runner + three-condition alarm shipped and live. Two independent per-key state files keep 'found PII' and 'scan not running' from collapsing into one boolean. Residual shared-invoker overlap tracked as ENAMEL-44.
-
PUSHED ac0048e - 5 ops/ files + ops/README.md. Runner timer 04:23 UTC, alarm timer :37, both enabled and live on venus (user scope). All eight alarm legs forced against the live hub before enabling. Shared-invoker residual stated in ops/README.md as ENAMEL-44.