Quiet well-known-path 404 noise (robots.txt, apple-touch-icon[-precomposed].png) in page_not_found telemetry

Sub-tickets

Questions

Activity

• wi-cli-venus created · 2026-06-21
• wi-cli-venus commented · 2026-06-21
  Post-PLUTO-180 organic traffic surfaces steady well-known-path 404 noise: /apple-touch-icon.png, /apple-touch-icon-precomposed.png (iOS Safari auto-probe), /robots.txt (crawlers). All genuine 404s, correctly logged by the new catch-all sink, warn/alert=false, 0 email - NOT a regression, the sink working as designed. But they pollute page_not_found telemetry with low-value organic noise every iOS visitor + every crawler generates. Options for coder (pick per intent, NOT yet prioritized): (a) add the real assets - a robots.txt (sensible for the public domain; disallow-crawl for an authed app) + apple-touch-icon; (b) matcher-exclude these well-known paths in src/proxy.ts (favicon.ico already excluded) so they never reach the capture sink; (c) classifier downgrade well-known-bot-paths to a quieter tier. Low priority, 0 user/email impact. Hold for Elazar/PM prioritization before implementing.
• wi-cli-venus commented · 2026-06-21
  Operational note: the well-known-path 404 alerts (apple-touch-icon, robots.txt) recur every few minutes from organic iOS/crawler traffic. A narrow real-time-suppression request DM'd to applog-pluto-cc went unactioned over 30+ min while alert emits continued - suggests applog-pluto-cc is a non-interactive emitter daemon (alive, forwarding) rather than an agent that reads/acts on DMs. So near-term PM-ping suppression likely needs maintainer-side daemon config, not a DM. This reinforces fixing at the SOURCE (option a: add robots.txt + apple-touch-icon assets, or option b: matcher-exclude in proxy.ts) over rail-side suppression. Still low-pri, 0 email/user impact, hold for Elazar prioritization.
• wi-cli-venus commented · 2026-06-21
  Audit security-triage (2026-06-21 17:12Z) surfaced an SEO angle within the well-known-path 404s: /robots.txt AND /sitemap.xml both 404 - these are legit SEO files that are simply not configured, so part of this WI is a real (minor) SEO gap, not pure noise. Recommend splitting the fix intent: (SEO) add a proper robots.txt (disallow-crawl appropriate for an authed app) + sitemap.xml if wanted; (noise) matcher-exclude apple-touch-icon* or add the icon asset. Separately, a credential-recon scanner hit prod same window (25 secret/config paths: .git/*, .env*, *credentials*.json, /api/config) - audit confirmed ALL 404 with zero exposure, NO incident; noted here only as the traffic context that surfaced the SEO gap. Security side needs no action; the scan rows log correctly as security/warn/alert=false/0-email.