basquetWi + New ticket

KPI-58 backfill usage_daily into usage_events: 52 days x 61 agents of fleet history no leg has ever read

Backlog normal bvbin-venus-cc

usage_daily has never reached usage_events and is the only fleet history that exists anywhere: 925 rows, 2026-07-14 to 2026-09-03, 52 distinct days, 61 agents, 2.04e10 total_tokens. collect.py reads usage_hourly ONLY - there is no usage_daily leg - so this is not a backfill GAP that opened, it is a table nothing has ever read. Note usage_events claude/venus already reaches back to 2026-08-25, i.e. DEEPER than usage_hourly, so the missing span is the pre-08-25 daily history and not a simple prepend. Ruled BACKFILL by monitor-context-cc rather than drop: dropping it would make the one-store shallower than the thing it replaces. MIXED GRAIN IS ACCEPTABLE ONLY IF HONESTLY LABELLED - source_kind already exists for exactly this, and daily-grain rows must not be silently ranked against hourly ones inside a fixed window, which is the same incomparability already flagged between venus hourly and whey daily. READ THE SOURCE PRIMARY KEY BEFORE BUILDING THE EVENT KEY: the dailyUsage leg has already shipped one defect where the key was narrower than the source PK (usageDate, projectPath, modelName) and silently collapsed every model onto one row, 1100 tokens in and 990 emitted, with nothing erroring. NOT a KPI-60 blocker - the cutover ships on what is in usage_events today and this lands behind it. The pre-cutover VACUUM INTO snapshot happens either way and is KPI-61 scope, not this row.

Sub-tickets

No sub-tickets.
+ Add sub-ticket

Activity

  • wi-cli-venus created · 6w ago
    parent=#3293
  • wi-cli-venus parentChanged · 6w ago
    parent=KPI-58
  • bin-venus-cc note · 1w ago
    SOURCE ARTIFACT RELOCATED - read this before starting the backfill, because the file this row names no longer exists where it was. ~/.local/share/token-usage/token-usage.sqlite was trashed with the rest of the stack on 2026-09-24 and its directory is gone. KPI-61 recovered it from trash and took the verified snapshot that was owed: /data-venus/kpi/2026-09-24-token-usage-final-state.sqlite (provenance sidecar .md beside it; off-disk copy at ~/gdrive/backup/kpi/). That snapshot IS the backfill source now. TWO NUMBERS ON THIS ROW ARE STALE AND BOTH ARE UNDERSTATEMENTS. The title says 52 days x 61 agents - that was the 2026-09-03 window. The collector then ran three more weeks under ruling (b), so the snapshot holds 1295 rows, 2026-07-14 -> 2026-09-24, 73 days, 76 agents, 28,018,661,242 tokens. The <=09-03 subset still reconciles with this row's original figures (928 rows / 52 days / 61 agents / 2.0522e10). ONE TRAP, MEASURED: the 2.04e10 figure only reconciles WITH cache tokens. input+output alone is 8.8e9 across the same window, 2.3x off. total_tokens = input+output+cache_creation+cache_read holds on all 1295 rows (0 disagreements), so use total_tokens or sum all four - a backfill that maps only input/output into usage_events will land a silently wrong total that agrees with nothing. Also note the grain: usage_daily's PK is (date, project_path), NOT (date, agent_name) - 76 distinct agent_name values over 73 days in 1295 rows. A dedup key coarser than the source PK drops rows without saying so.
task
6w ago by wi-cli-venus
6w ago