Strip Excel text-guard leading apostrophe from ID MDVR across DB (elazar-confirmed, narrow, backfill)
Excel text-guard leading apostrophe in ID MDVR survives into DB, e.g. vehiculo 5630/AF106WZ renders '303030313000. Elazar 2026-09-30: he asked the book's authors to add the leading apostrophe so Excel would not reformat the digit string as a number - transport packaging, not a business value, faithful-import does not protect it. Strip it. Scope measured by coder03 on PROD, all 901 text columns scanned, apostrophe exists ONLY here: vehiculos_datos_colectivos.ip_mdvr 1588/2417, rutap.rutap_rows.id_mdvr 1595, rutap.vehiculo_historial_normalizado.ip_mdvr and .raw_id_mdvr 1595 each, vehiculos_datos_colectivos_archive.ip_mdvr 4. Always leading, never trailing, ltrim(x,chr(39)) unambiguous. Do: (a) strip in rutap-snapshot-import.py narrow to id_mdvr/ip_mdvr only, do not make clean() strip apostrophes generally; (b) backfill existing rows. Leave raw_id_mdvr verbatim (diff contract, book-as-is). Leave _archive alone. Rehearse PROD under ROLLBACK with before/after counts, diff to audit for PASS, then PROD, then DEV. Report row count updated per table. Separately, ip_mdvr vs book's ID MDVR column naming (source of the MOVILBA-617 wrong PDF label) is NOT in scope here - column rename is a bigger move, own item if warranted.
Activity
-
db04-mba-l-ca 2026-09-30 03:35 ART. Independently measured PROD movilba: VDC ip_mdvr 1588/2417 leading chr(39), rutap.rutap_rows.id_mdvr 1595/5545, imported normalized history ip_mdvr/raw_id_mdvr 1595/5545 each, VDC archive 4 historical rows. DEV movilba_dev: VDC 1449/2433 (all ssot_import, zero audit_log app UPDATEs), raw stage 1595/5545, imported normalized history ip_mdvr/raw_id_mdvr 1595 each. All affected PROD business strings are one apostrophe + 12 digits; zero double/trailing apostrophes. Installed projector copies raw_id_mdvr from rutap_rows.id_mdvr and asserts byte-equality, so proposed backfill leaves both evidence columns and archive untouched directly, and updates only normalized history.ip_mdvr plus VDC.ip_mdvr. PROD rollback script at /data-lezama/movilba/migration-evidence/2026-09-30-movilba-618-mdvr-text-guard-DEV-Y-PROD.sql SHA256 edc9ebc75f222e2bcb8ad866976a975359025eaa1296bca391dcbe24b4f19f89: exit 0, history 1595 and VDC 1588 trial updates, raw-twin parity 0 diffs, ROLLBACK; independent post counts unchanged. VDC archive trigger would add 1588 historic rows on real apply. Evidence /data-lezama/movilba/evidencia/MOVILBA-618/2026-09-30-prod-rollback.md. No DB apply yet; importer prevention still to implement and audit.
-
db04-mba-l-ca 2026-09-30 03:40 ART. Added a 39-line narrow post-projection normalization in tools/rutap-snapshot-import.py: for each imported snapshot, strip one leading chr(39) only from normalized rutap.vehiculo_historial_normalizado.ip_mdvr after raw/source equality check and before reconciliation; leave rutap_rows.id_mdvr and raw_id_mdvr unchanged. PROD duplicate-snapshot importer ROLLBACK rehearsal using current 09-28 workbook exited 0; installed reconciler preview reported 1588 VDC ip_mdvr differences from the now-normalized history and all other listed vehicle fields zero; transaction rolled back. SQL backfill draft SHA256 edc9ebc75f222e2bcb8ad866976a975359025eaa1296bca391dcbe24b4f19f89 at /data-lezama/movilba/migration-evidence/2026-09-30-movilba-618-mdvr-text-guard-DEV-Y-PROD.sql, PROD ROLLBACK rehearsal 1595 history + 1588 VDC updates, raw/stage unchanged, archive trigger 1588 additions rolled back. Importer diff and SQL await audit PASS. No DB apply yet.
-
audit pass: importer normalizar_guardia_excel_id_mdvr (39 lines) + backfill SQL SHA edc9ebc7
payload
{"scope":"importer normalizar_guardia_excel_id_mdvr (39 lines) + backfill SQL SHA edc9ebc7","outcome":"pass","evidence":"py_compile clean; traced rutap_reconcile_source (ejecutar_reconciliacion) reads h.* off normalizado post-strip, not raw - no reintroduction landmine; backfill asserts pinned snapshot md5, exact pre-counts, guard+12-digit format, raw/raw_id_mdvr/stage byte-unchanged post, archive read-only via trigger; scope matches Elazar's ask exactly","reportRef":null} -
db04-mba-l-ca 2026-09-30 03:47 ART. Audit PASS recorded for exact importer +39-line diff and SQL SHA256 edc9ebc75f222e2bcb8ad866976a975359025eaa1296bca391dcbe24b4f19f89. Applied SQL with -v apply=true and COMMIT, PROD first then DEV. PROD movilba snapshot7: 1595 imported normalized history ip_mdvr and 1588 current VDC ip_mdvr rows updated; DEV movilba_dev snapshot19: 1595 and 1449 respectively. Independent post-commit SELECT: 0 leading apostrophes in those business columns in both bases; source rutap_rows.id_mdvr and history raw_id_mdvr remain verbatim with 1595 leading apostrophes each; PROD AF106WZ now 303030313000. Existing VDC archive trigger added 1588 PROD and 1449 DEV historical rows, no direct archive edits. Evidence /data-lezama/movilba/evidencia/MOVILBA-618/2026-09-30-applied.md. Audited SQL promoted byte-identical to docs/migrations/2026-09-30-movilba-618-mdvr-text-guard-DEV-Y-PROD.sql; importer and migration are ready for coder03 SHIP/commit. Item remains inProgress until code lands.
-
db04-mba-l-ca 2026-09-30 post-apply collision check. PROD movilba and DEV movilba_dev: 42 current cross-vehicle pairs, 42 distinct normalized ID MDVR values, where guarded and unguarded source strings collapse after stripping; 55 pairs across all periods. No unique index/constraint on either normalized ip_mdvr column; inspected installed view/function definitions and app references reveal no equality join on it. Real shared identifiers, but no key/join breakage established. Evidence: /data-lezama/movilba/evidencia/MOVILBA-618/2026-09-30-applied.md. Coder03 and audit notified. No further data change.
-
db04-mba-l-ca 2026-09-30 03:52 ART. Verified completed in named bases: PROD movilba snapshot 7 and DEV movilba_dev snapshot 19 each have zero leading apostrophes in normalized-history and current VDC ip_mdvr after committed backfill; raw id_mdvr and raw_id_mdvr retain source bytes. Audit PASS is recorded; importer and migration shipped by coder03 as dev c1915ab9/main 6aa71e8e, v5.6.118, dev health green. Post-apply collision check found 42 current cross-vehicle normalized identifier matches per base, with no unique constraint or identified equality join; recorded separately in item and evidence. Hub locks held by db04: none.
-
db04-mba-l-ca 2026-09-30. Additional book-wide measurement reported by coder03-mba-l-cc against current snapshot in both bases, keyed on nro_dominio: 286 ID MDVR values are shared by more than one dominio before removing the Excel guard, 326 after. Thus sharing predates MOVILBA-618; the apostrophe made equal business identifiers appear different. Coder03 also reports 184 shared IDs among current PROD vehicle records and 195 in DEV, with zero guards remaining. These counts have a different denominator from my 42 cross-vehicle guarded/plain pairs in the current-history view; do not present 42 as all shared IDs or as simultaneous open source periods. Source-book temporal review is in MOVILBA-619: 55 guarded/plain cross-vehicle row pairs, seven with overlapping periods and different chassis; Eugenia question pending tracker entry. No rollback or code change.
-
db04-mba-l-ca 2026-10-01 22:00 ART — Trampa del medidor de fidelidad MOVILBA-186, no nueva pérdida de datos: tools/rutap-fidelidad.py:comparar aplica a=plegar(Base['ID MDVR']), b=plegar(v_vehiculos_export.id_mdvr) y marca diferencia con a.upper()!=b.upper(). plegar no retira el apóstrofo inicial de guarda de Excel. Control positivo: AE971FE en el libro es '\''303033333900 y en export 303033333900; ocurre la misma forma en la muestra de 60 y el medidor informa 1589/2419 diferencias id_mdvr en PROD y DEV. El predicado de reproducción es left(Base['ID MDVR'],1)=apóstrofo AND export.id_mdvr=substring(Base['ID MDVR'] FROM 2): la comparación actual lo cuenta como diferencia aunque MOVILBA-618 estableció que el apóstrofo es transporte. No se puede llamar a las 1589 un defecto del dato sin comparar primero el valor de negocio, retirando sólo esa guarda de ID MDVR. Raw rutap_rows.id_mdvr y raw_id_mdvr permanecen verbatim. Sin cambios de código o DB.
-
db04-mba-l-ca 2026-10-01 22:14 ART — Desambiguación de instrumentos: las 1589 diferencias falsas de MOVILBA-186 nacen exclusivamente del predicado a.upper()!=b.upper() entre Base['ID MDVR'] con apóstrofo inicial y export.id_mdvr sin él; 1589/1589 se igualan quitando sólo ese primer apóstrofo. NO son las 1598 falsas diferencias registradas en docs/reference/catalogos.md:66, cuyo origen fue cruzar tipo_motor con el estándar de emisiones. Magnitudes parecidas, campos y causas distintos. Sin DB write.