Hostinger image-storage monitoring (api-images)

Sub-tickets — 3/4 done

Activity

• wi-cli-venus created · 2026-07-17
• wi-cli-venus note · 2026-07-17
  Scoping (coder-mars-cc 2026-07-17): /stats-readable part = per-app daily Vercel cron /api/cron/storage-stats (daily only - sub-daily fails Hobby deploy) that GETs IMAGE_API_BASE_URL/stats (X-Api-Key auth), writes info-tier appEvents {fileCount,totalBytes,percentUsed}, escalates via existing alert rail. KEY: disk.warning is SERVER-WIDE + already true (71.5% at a 50% threshold) so paging on warning:true fires day-one - alert must trigger on OUR ceiling (proposed >=85%) or tenant-growth spikes, one-shot dedup. Per-tenant: each app key sees only its tenant; pluto/venus copy the route w/ their keys; PM reads the 3 appEvents streams for fleet view. Effort ~half day Mars + near-copy pluto/venus. On-demand admin tile +2h optional. Account-quota still hPanel/Elazar-blocked. Class-S (no schema/auth/email).
• wi-cli-venus note · 2026-07-19
  Quota confirmed via keyed SSH (~/.ssh/sshapiimage) + hPanel 2026-07-18: plan ceiling 25 GB, used 1.32 GB = 5.3%. api.pensanta.com (api-images) is 1.3G of a 1.4G account total; all other sites <54M. Alert line = 85% of 25GB = 21.25 GB used (fires ~3.75GB free). No pressure today; monitoring is low-priority (P3) daily/weekly du. Build option offered to Elazar: venus cron SSHes in, DMs only on crossing 21.25GB. SSH details in LTM.
• wi-cli-venus note · 2026-07-19
  4-PM design poll converged (2026-07-18): (1) CONTINUOUS awareness = per-app admin UI tile always visible 'Esta app: X GB · Pool: Y/25GB libres' — unanimous, this is Elazar's space-left-aware. (2) account-total is THE alerting number; per-app view alone hides cross-app cannibalization. (3) plumbing: ONE venus cron holds SSH key, dus 25GB account-total daily, writes small shared surface; apps pull account-total for tile + own /stats for per-app half; SSH key single-homed, NOT distributed to 3 apps. (4) graduated bands; DM on band-crossing via existing 10min-dedup rail, appEvents at warn+ only, no info noise. Variances: page 85(3of4) vs 90(mars); +optional critical95(venus); venus wanted weekly digest, others crossing-only. PM recommendation to Elazar: info60/warn75/page85/crit95, tile-both-numbers, crossing-only DMs. Owner: bin-venus-cc (needs maintainer row). Class-S. Awaiting Elazar confirm/override before dispatch.
• wi-cli-venus note · 2026-07-19
  Elazar FINAL spec 2026-07-18: (1) Cadence = WEEKLY, not daily ('once a day is too much' — plenty of space; 3-4mo term, ~1.3GB accrued won't approach 5GB of the 25GB ceiling). (2) Surface = storage tile on each app's admin HOME/landing control-panel route (ambient, 'not a big deal'). (3) Alert = SINGLE trigger at 75% (18.75GB) — collapses the graduated 60/75/85/95 bands; reaching 75% means something went wrong. (4) Cron owner = bin-venus-cc: weekly SSH du of the 25GB account-total, compute %, write a small shared surface the 3 apps read for the tile, DM fleet PMs + Elazar on crossing 75%. Needs maintainer-agents.sqlite row. Now ACTIONABLE — dispatching cron to bin-venus-cc + admin-landing tile to 3 app coders.
• wi-cli-venus statusChanged · 2026-07-19
  status=todo
• wi-cli-venus assigned · 5w ago
  coder-apiimages-cc
• wi-cli-venus note · 5w ago
  Scoping vs live box, coder-apiimages-cc 2026-09-11. Read live /stats storage{} (READBACK, not the .env file): quotaBytes=25000000000, measured=true, usedBytes=2138759069, percentUsed=8.6, warning=false, thresholdPercent=80. Three consequences. (1) Gap (2) in the description - 'the marsapi ACCOUNT quota is NOT agent-readable' - is CLOSED. The 25e9 floor is configured on the box and /stats serves it, so the quota half of this WI needs no hPanel check and no Hostinger API creds. It is a FLOOR, not the true ceiling (hPanel's 25600 is unit-ambiguous: 25.6e9 or 26.84e9), which is the safe direction - a low ceiling warns early, a high one warns late. (2) MISMATCH, and it is the actual finding here: the box threshold is 80%, Elazar's FINAL spec of 2026-07-18 on this row is a SINGLE trigger at 75% collapsing the graduated 60/75/85/95 bands. The box was never reconciled to that decision. At 8.6% there is no live consequence today - which is exactly why it survived two months - but the alert would fire ~1.25GB later than decided. Fix is a one-line THRESHOLD_PERCENT change in the box .env shipped with the waiting bundle, not a code change. (3) The 2026-07-17 snapshot in the description is stale: mars is 16223 files / 2.07GB, not 9368 / 1.2GB. Per CLAUDE.md I am not writing a replacement number into the description - the instrument is /stats. Remaining scope is NOT api-images': Elazar assigned the weekly cron to bin-venus-cc and the admin tile to the three app coders (PLUTO-605 done). api-images' share is the /stats surface, which exists, is measured, and is serving. I already hold EVO-92 (daily log-reader timer on venus) and will not fork a second timer for this.
• wi-cli-venus note · 5w ago
  Threshold reconciled on the branch, a640b5a (gitpush.sh v4.53, contracts green: 7 layouts + 16 assertions, .env.example readback verbatim). pmmaster GO, delegated authority. TWO copies changed, not one - STORAGE_WARN_PERCENT was already documented in .env.example, so the box's 80 could have come from the file OR from the stats.php fallback default, and fixing only the file would leave a rebuilt or clobbered .env landing straight back on 80. The code default is the copy nobody diffs. NOT DONE: the live box still reads 80. Closing this needs STORAGE_WARN_PERCENT=75 set in the box .env when nw-whey-cc fires the waiting bundle - verify by /stats readback of thresholdPercent, not by reading the file. Also closed in passing: the quota gap this row was opened for. Live /stats answers quotaBytes 25000000000, measured true, percentUsed 8.6 - QUOTA_BYTES IS set, so no hPanel or Hostinger-API work is needed. 'measured' is the discriminator for whether a quota is configured, not the .env file. Why this survived two months: 8.6% is far from both 75 and 80, so neither value was ever approached and nothing could contradict either - a threshold that is never approached cannot report its own staleness.
• wi-cli-venus verified · 5w ago
  Threshold LIVE at 75 on the box, verified by /stats readback (nw-whey-cc pushed STORAGE_WARN_PERCENT=75 to the box .env by single-file lftp, not a full deploy). Branch carries it too: a640b5a, plus c7a19cf correcting the doc. What is still NOT true, and it is the part a one-line fix hides: the box RUNTIME is pre-a640b5a, so its stats.php fallback default is still 80. The live threshold is correct only for as long as the box .env survives - a clobbered or rebuilt .env lands straight back on 80 until deploytohostinger.sh ships the code default. 'Fixed on the box' and 'fixed in the copy that survives losing the box .env' are two different states and only the first is true today. api-images' share of this row is otherwise complete: quota gap closed (quotaBytes 25e9, measured true), threshold reconciled and live, /stats surface serving. Remaining scope is NOT mine - the weekly cron is bin-venus-cc's and the admin tile is the app coders' (PLUTO-605 done). Not closing the row: closing it would retire that scope with it.
• wi-cli-venus note · 4w ago
  PARKED - fleet-wide STOP (Elazar direct via pmmaster, 2026-09-11 23:57 ART). Nothing in flight, nothing uncommitted, not broken. DONE: api-images' share of this row is complete and live. Threshold reconciled 80 -> 75 in both copies that can hold it (a640b5a: .env.example + the stats.php fallback default), box .env set by nw-whey-cc and verified by /stats readback (thresholdPercent 75), doc corrected in c7a19cf. Quota gap closed earlier - quotaBytes 25e9, measured true, so no hPanel or Hostinger-API work is needed. NOT DONE, and it is a real gap rather than a formality: the box RUNTIME is still pre-a640b5a, so its stats.php fallback is 80 and a clobbered or rebuilt box .env lands straight back on 80. The live threshold is correct only while that file survives. NEXT ACTION (not mine): nw-whey-cc's full deploytohostinger.sh run of the waiting 12-commit bundle, which is gated on a frozen uploads/ snapshot plus Elazar's go. Remaining row scope is also not mine - weekly cron is bin-venus-cc's, admin tile is the app coders' (PLUTO-605 done). Row deliberately left open rather than closed, so that scope does not retire with it. Separately parked in my lane, both already queryable: EVO-2 is blocked on Elazar's trash-vs-reassign call, and the 14-day CHECKSUM warn-window is counting days-observed with an earliest close of ~2026-09-24 (timer enabled, first scheduled fire 2026-09-12 12:02Z). Two decisions still waiting on an Elazar CLI turn, since hub DMs never reach him: per-GET duration logging in serve.php, and a Hostinger ticket on peak connection-accept behaviour - both are MARS-677 residual, whose upstream half I closed (api.pensanta.com IS reachable from Vercel egress; the ECONNREFUSED row was the discriminator).
• coder-apiimages-cc completed · 1w ago
  api-images' share is delivered and live; the row's remaining scope has been split into rows the owning seats can act on, per pmmaster 2026-10-06 - an open item nobody in this lane can act on reads as backlog forever. DELIVERED: the /stats surface exists, is measured and is serving (per-tenant fileCount/totalBytes plus storage{usedBytes,quotaBytes,percentUsed,warning,measured,thresholdPercent}); the quota gap this row was opened for is closed, since live /stats answers quotaBytes 25000000000 with measured true, so no hPanel check and no Hostinger API creds are needed - 'measured' is the discriminator for whether a quota is configured, not the .env file, because the file can be right while the running process holds a stale parse; and the threshold is reconciled to Elazar's final 2026-07-18 spec of a single 75% trigger, live on the box and verified by readback. SPLIT OUT rather than retired with this row: EVO-116 weekly storage read and surfacing (bin-venus-cc - Elazar assigned the cron there in 2026-07 and it never had a row), EVO-117 admin tile for mars/venus/terra or an explicit cancel (pmmaster to route, since whether each app wants it is its owner's call and I hold none of those repos; PLUTO-605 already shipped pluto's), EVO-118 the one piece of api-images' share that is NOT live - the box runtime stats.php fallback is still 80, so the live 75 survives only as long as the box .env does (blocked on nw-whey-cc's bundle). The description's 2026-07-17 snapshot was stale by ~2x and no replacement number is written anywhere: the instrument is GET /stats, not a figure in a row.