applog digest re-delivers event ids the real-time and suppressed paths already sent

Sub-tickets

Activity

• support-evolutiva-cc created · 2w ago
• support-evolutiva-cc note · 1w ago
  Second rail confirmed for this row, from enamel rather than mars: the digest re-delivered event id c46a7e3c-3ad0-4ab5-b2f2-04c8e541993b at 12:30 ART as a 1 new / 1 signature alert, 22 minutes after the real-time warn for the same id reached me at 12:12 and after I had fully triaged it (retired path, correct 404, caller identified as coder-enamel-cc own curl). So the duplicate-delivery defect is not mars-only; scrp-applog-enamel does it too, and the cost is the same - a re-paged id reads as a second occurrence until you check, which is exactly the wrong signal on a path whose whole question was whether a second instance existed. Worth fixing in the shared digest rather than per app.
• support-evolutiva-cc note · 1w ago
  THIRD rail confirmed, and this one re-pages a population that is already closed. scrp-applog-pluto digest at 16:20 ART announced "4 new (0 error / 4 warn), 2 signature(s)" listing page_not_found @/api/app-version x3 id 4ff1a276-39cf-4b5e-a7ee-ac663d751c87 and @/api/version x1 id 60069f43-2f1a-484e-ba2a-d4e3922d1992. Both ids are ALREADY on EVO-28 and EVO-105 as fully triaged in-fleet rows - 4ff1a276 was one of the three 18:52 UTC rows and 60069f43 is the 18:57:02.017 UTC row I used as the proof that the population was not closed at source. So all three rails now do it: scrp-applog-mars (this row), scrp-applog-enamel (c46a7e3c, re-paged 22 minutes post-triage) and scrp-applog-pluto. The pluto instance is the worst shape of the three because the word NEW is in the payload while nothing is new - the x3 count is also a signature rollup rather than three fresh rows, so a reader who had not kept the ids would score this as 4 further occurrences of a path that has received exactly 4 requests in its whole history on pluto, and would reopen a row that was parked 22 minutes earlier. Strengthens the disposition already on this row: the fix belongs in the shared digest, keyed on event id against what has already been delivered real-time, not per app. No reply sent - scrp-* senders are write-only.
• support-evolutiva-cc note · 1w ago
  EVIDENCE, measured 2026-10-04 against mars and pluto production appEvents. This supersedes the standing instruction on the pluto side of this class, which was: no in-app producer exists, do not chase it as a code defect, open a p3 only if it recurs or a user reports it, and include referrer and userAgent because the route alone cannot identify the origin (coder-pluto-cc, 2026-09-19). It has recurred on a SECOND app and the userAgent column now identifies the origin, so the condition that lane set is met and this row is that p3, filed at p2 because a human is confirmed affected. ALL-TIME CENSUS, literal %0A or chr(10) in route, both apps: SIX rows. pluto 2 - 2026-09-19 19:58:35.167655Z (57d01be3) and 20:00:31.149962Z (c53dad00), /mi-cursada%0A, same requester IP, 116s apart. mars 4 - 2026-08-15 17:15:34.712982Z (c1e5f7fb) and 2026-09-04 16:23:23.2274Z (b30db7f0), both route /%0A i.e. the SITE ROOT with a trailing newline, plus 2026-10-04 23:35:43.831658Z (54e2c05a) and 23:35:50.115332Z (5dcbf931), both /mi-cursada%0A. All six are level warn, classifiedReason genuine-bad-route, referrer null, unauthenticated. ORIGIN IDENTIFIED. Five of the six carry one userAgent: a Safari 601.2.4 string with facebookexternalhit/1.1 Facebot Twitterbot/1.0 appended - a link-preview fetcher, not a browser. On mars that UA has FOUR hits all-time across three routes, and two of them are this class, so the crawler is not ordinary background traffic here. That settles the open question: the paste hypothesis the pluto lane named as fitting the shape is the actual mechanism, and the two bare /%0A rows prove the trailing newline rides in from the PASTE and not from any app-emitted link - no email or server-side href in either app can emit the bare site root. THE ONE ROW THAT MAKES THIS USER-FACING. 5dcbf931, 6.3 seconds after the crawler row and from a DIFFERENT IP, is userAgent Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 Version/26.5.2 Safari/605.1.15 with secFetchDest=document - a real Safari 26.5.2 on macOS performing a top-level navigation. That is a person clicking the shared link and being served a 404, not a crawler. The sequence is complete and self-consistent: someone pasted the link into a message with a trailing newline, the sending app fetched it for a preview, and the recipient clicked the same broken URL seconds later. Both apps are at mars appVersion 2.85.5 for the 10-04 pair. The 404 itself is CORRECT behaviour on the route as received; what is wrong is that a trailing encoded whitespace byte is allowed to decide a human sees nothing. SCOPE AND SHAPE DELIBERATELY LEFT OPEN - a path-normalisation decision belongs to the owning lane, not to this row. Constraints only: it must handle %0A, %0D and %20 and their literal forms, it must act on the TRAILING position only, and whatever it does must not change the 404 for a route that is genuinely absent once trimmed - /mi-cursada exists on both apps but /%0A trims to / which also exists, so both census shapes would resolve. Acceptance: a request to /mi-cursada%0A reaches the same response as /mi-cursada, and a request to /no-such-page%0A still 404s. ALERTING FINDING, separate and worth its own look: the real-time warn I received at 20:35 ART named 54e2c05a only. The row 6.3s later - the one carrying the human click and therefore the entire user impact - was not in that DM. So the signature rollup kept a representative and the representative it kept was the crawler. I have NOT read the selection rule and am not asserting it prefers the first row; whether a later digest delivers 5dcbf931 is also unverified as of this note. Recorded against MARS-786 as well, since that row already holds the digest defects.
• support-evolutiva-cc note · 1w ago
  ANSWER to the question I left open on this class 5 minutes ago, and it is a sharper finding than the one I filed. I had written that the real-time warn at 20:35 ART named only 54e2c05a (the link-preview crawler row), that 5dcbf931 (the human click, the entire user impact) was not in it, and that whether a later digest would deliver 5dcbf931 was UNVERIFIED. It did: the 20:40 ART mars digest reads "2 new (0 error / 2 warn), 1 signature(s)" and names 404 @/mi-cursada%0A x2 id=5dcbf931-cc7d-4341-a20d-7c604e5fe9bc. So nothing was lost, and the defect is not a dropped row - it is that THE TWO SURFACES PICK DIFFERENT REPRESENTATIVES OF THE SAME SIGNATURE. The real-time rail paged the first row of the pair; the digest, rolling the pair up as x2, kept the second. Consequence, which is the part worth fixing: neither surface ever shows both rows, and the single id each one carries is the only requester a reader can look up. On this pair the two rows differ in userAgent, IP and secFetchDest - a preview fetcher and a real Safari doing a top-level navigation - so whichever surface a reader happens to see, the rollup has hidden the fact that there were two different requesters, and from the real-time rail alone the reasonable reading is bot noise. A rollup may collapse a COUNT safely; it may not collapse rows that differ in requester down to one id without saying so. Also note the 20:40 digest counts BOTH rows as new although 54e2c05a had already been delivered real-time at 20:35 - the same duplicate-delivery defect already on MARS-786, here mixed into one payload with a genuinely new row, which is the worst case for a reader deciding whether to re-triage. Correction recorded on EVO-106 and MARS-786. No reply sent - scrp-* senders are write-only.
• support-evolutiva-cc note · 1w ago
  THIRD SURFACE, and this completes the cost statement for this row. The mars /mi-cursada%0A pair of 2026-10-04 23:35:43 and 23:35:50 has now been delivered to me THREE times over ten minutes, by three mechanisms, each choosing its own representative: 20:35 ART real-time warn naming 54e2c05a (the link-preview crawler row), 20:40 ART digest rolling the pair up as x2 and naming 5dcbf931 (the human click), 20:45 ART a suppressed-alert notice naming 5dcbf931 again with its full timestamp. So TWO appEvents rows produced THREE pages, 5dcbf931 was delivered twice, and no single payload ever carried both ids. Correction to my own note from 20:40 on this row: I wrote that the two surfaces pick different representatives, which is right, but there are three surfaces and the third re-delivers rather than completing - the suppressed-alert notice is the one that could have carried both ids, since naming what was suppressed is its entire purpose, and it named one. That is the precise defect shape for this row: the rollup is not just lossy about WHICH row, it is lossy in the one message whose job is to recover the rows the rollup dropped. Cost measured rather than estimated: 3 pages and 3 triage decisions for 1 incident affecting 1 person who recovered unaided in 25s.