fix(summary): align CLI success/failure with the ingestor — inserted-based rate + full has_failures

[LukasWodka]

fix(summary): align CLI success/failure with the ingestor (silent-success + wrong staging-reclaim)

Found in the 2026-07-08 prod audit. The CLI's Summary methods diverged from the
ingestor's own success/failure determination, so a partial run could report
success AND reclaim (delete) the staged source:

• HasFailures() was FailedRecords>0 || FileTransferFailures>0 — it IGNORED
  skipped rows, inserted<total, and api_sent<inserted. The ingestor's
  IngestionSummary.has_failures (data-ingestors ingestors/base.py) counts all
  five. So a run that silently SKIPPED rows or inserted fewer than total was
  classified "succeeded" → exit 0 AND the staging-reclaim gate deleted the
  user's source copy. Now mirrors the ingestor exactly. Every counter it reads
  is emitted unconditionally by the ingestor banner (reporting.py) and parsed,
  so the new inserted<total / api_sent<inserted clauses can't false-positive on
  a clean run (all counters equal).

• SuccessRate() was ProcessedRecords/TotalRecords; the ingestor's banner uses
  inserted/total. processed (passed validation) is a superset of inserted (rows
  actually in MySQL), so the CLI OVERSTATED success (e.g. 100 processed / 70
  inserted showed 100%, not 70%) in the panel header + --output-json
  success_rate. Now inserted/total, matching the banner.

• RenderSummary headline: regraded to hard-failures (DB/transfer errors) → red,
  softer partial (skips / short insert-or-sync) → yellow, clean → green — so
  the broader HasFailures drives exit-code + reclaim while the headline still
  distinguishes a hard failure from a partial.

Behavior change (intended): a run with skipped rows, inserted<total, or
api_sent<inserted now exits 9 (was 0) and its staged source is KEPT (not
reclaimed) — matching what the ingestor already reports in its own banner.

Tests: HasFailures + SuccessRate matrices rewritten to the ingestor-aligned
semantics (incl. the processed-overstates case + one case per new clause);
RenderSummary headline cases + one classifyPushOutcome fixture updated to
realistic full-counter summaries. Full cli suite green.

Standalone CLI correctness fix (files untouched by the in-flight CLI branches).
Relates to the data-ingest epic backend#1008 / test-hardening backend#1009.

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

---

Note

Medium Risk
Changes exit codes and staging-reclaim behavior for real customer pushes when ingest is partial; intended fix for silent data loss but alters automation that assumed exit 0 on skipped rows.

Overview
Aligns CLI ingestion outcome with the ingestor so partial runs no longer exit 0 or reclaim staged data when rows were skipped, not fully inserted, or not fully synced to the API.

Summary.HasFailures() now matches the ingestor’s has_failures: it treats skipped rows, inserted < total, and api_sent < inserted as failures—not only DB insert and file-transfer errors. That drives exit code 9 and keeps staged sources when the run isn’t clean.

SuccessRate() switches from processed/total to inserted/total, matching the ingestor banner and JSON success_rate so validated-but-not-inserted rows don’t show 100% success.

RenderSummary still uses the broader HasFailures() for exit/reclaim semantics but splits messaging: hard DB/transfer failures stay red; softer partials (skips / short insert or sync) use yellow; clean runs stay green.

Tests cover the new HasFailures and SuccessRate matrices, headline rendering, and a full-counter “clean” classifyPushOutcome fixture.

^{Reviewed by Cursor Bugbot for commit 0b15162. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 0b15162. Configure here.}

[cursor]

Partial runs mislabeled as skips

Low Severity

RenderSummary routes every soft partial from HasFailures() through a single yellow headline that always says “completed with skips,” even when SkippedRecords is zero and the failure is only InsertedRecords below TotalRecords or APISentRecords below InsertedRecords. Exit code and staging reclaim stay correct, but operators can misread API or insert shortfalls as validator skips.

 

^{Reviewed by Cursor Bugbot for commit 0b15162. Configure here.}