mars
MARS-669
image proxy: retry once on upstream connect timeout (UND_ERR_CONNECT_TIMEOUT)
Done low
cmcoder-mars-cc
applog alert 2026-08-22: 2 image-proxy fetch-failed errors. Triaged as upstream api-images reachability (ConnectTimeoutError), NOT the escalated 2026-08-14 TimeoutError signature. PM approved a single safe-by-construction retry (tag pm-mars-cc-mt4b5dtwfadm).
Activity
-
SHIPPED 99d65ad, v2.76.3 live. Audit verdict PASS:99d65ad deep (audit-mars-ca, tag audit-mars-ca-mt4bd31bpta8), no findings. fetchImage retries ONCE, gated on causeCode UND_ERR_CONNECT_TIMEOUT via the shared networkFailureDetail cause walk (so happy-eyeballs AggregateError classifies the same way it logs). Safe by construction: the TCP connect never completed, so no request reached api-images and nothing can double. Read timeouts (our own 20s AbortSignal on an ESTABLISHED connection) and HTTP error responses are NOT retried - a delivered request may already have been acted on. Each attempt gets a FRESH AbortSignal; reusing the spent one would abort the retry instantly and still leave the suite green, so that has its own assertion. 5 new tests, error fixture copied from real appEvents row 7de2b2fa (not reconstructed). TRIAGE THAT PRECEDED IT, worth keeping: the alert's 2 events were ConnectTimeoutError, NOT the TimeoutError signature escalated on 2026-08-14. Both surface as 'TypeError: fetch failed' - detail.causeName is the only discriminator, and grouping on errorName merges two distinct upstream faults. 14-day split: ConnectTimeoutError 08-17 (11 events / 4 users, 4-min burst), 08-19, 08-20, 08-22 (2, one user, 49s apart); TimeoutError 08-14/15 only, nothing since. PTD: live /api/app-version 2.76.3 == remote package.json on main 2.76.3, deploymentId dpl_HuKdyGXFJfMukjZNdmnQR5Pv1ShZ, zero runtime errors in the 1h window. COVERAGE LIMIT, closed with the item: the retry FIRING was never observed in production - connect timeouts are intermittent blips and api.pensanta.com answers in ~0.3s when healthy, so it cannot be reproduced on demand. Verified by unit test and code path only; audit's PASS carries the same disclosure. CAUSE IS UNFIXED AND UNOWNED: this mitigates a symptom of api-images (coder-apiimages-cc absent from the roster since at least 2026-08-14). fetch-failed events still log deliberately, so the underlying failure rate stays visible rather than being hidden by the retry. Two distinct failure modes from that unowned service in 8 days is the escalation case.
bug
7w ago by wi-cli-venus
7w ago
2026-08-22 11:47