ci: pin the data-ingestors schema ref + add a coverage floor (#1009 P0.3)

[LukasWodka]

Summary

Two of the three CI drift tripwires from #1009. Both make an existing "prints a number, asserts nothing" check into an actual gate — without adding cross-repo flakiness.

(b) Pin the schema-drift check to a SHA, not floating master

sync-schema.sh now builds its fetch URL from scripts/.data-ingestors-ref (a pinned commit SHA), overridable via DATA_INGESTORS_REF. Before, any upstream commit touching the schema reddened every open CLI PR until someone re-synced. Now adopting upstream is a deliberate SHA bump + re-sync in one PR. Pinned to 0de1f148 (current data-ingestors master; the embedded schema matches it, so the check stays green).

(c) Per-package coverage floor

go test -cover printed coverage and asserted nothing, so the two load-bearing, historically thin-tested packages could silently rot. scripts/coverage-floor.sh fails the build if internal/cli (the money path) or internal/submit (the jobs-manager orchestration) drops below its floor — wired into the Test job after go test.

Floors are a ratchet (bump UP only; lowering is a reviewed edit with a reason), set just under current develop:

package	floor	current
internal/cli	68%	70.1%
internal/submit	72%	74.8%

Bump these up once #186/#187 land (they lift internal/cli to ~72%). The script is bash-3.2-portable (no associative arrays — macOS default).

(a) deferred, deliberately

The third tripwire — a scheduled sync-validator-goldens.sh --check against the pinned ingestor — waits for #340 to land on a stable ref: the value-aware --check regenerates against the real ingestor read path and needs the #340 fix, else it fails. Meanwhile the Go-side parity test already enforces the committed goldens on every PR (in the test job), so value-level parity is not unguarded — (a) is the upstream-drift alarm, which correctly follows #340.

Verification

• sync-schema.sh --check matches at the pinned SHA (green).
• coverage-floor.sh passes and bites (a 99% floor fails; a below-current floor passes).
• shellcheck clean; build.yml parses; full go test ./... green.

Type

CI hardening · cli · RFC-0002 foundation. Part of backend#1009 (P0 CI drift tripwires). Independent of #186/#187/#340.

---

Note

Low Risk
Changes are limited to CI scripts and workflow wiring; no runtime CLI behavior changes, with modest operational risk from choosing the pinned schema SHA.

Overview
Pins schema drift checks to a fixed data-ingestors commit via scripts/.data-ingestors-ref instead of tracking floating master, so unrelated upstream schema changes no longer fail every open CLI PR. sync-schema.sh builds its fetch URL from that ref (overridable with DATA_INGESTORS_REF) and validates the ref before interpolating it into the download path.

Adds a coverage ratchet in scripts/coverage-floor.sh, run in the Test job after go test -cover, which fails the build if internal/cli or internal/submit statement coverage drops below configured floors (68% / 72%). Floors are intended to move up only with deliberate edits.

Workflow comments for the schema-drift job are updated to describe the pinned-ref workflow and deliberate SHA bump + re-sync for adopting upstream.

^{Reviewed by Cursor Bugbot for commit 18b4e43. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d44b949. Configure here.}

[saadqbal]

Nice, tight PR 👍 One thing worth closing that I pushed a fix for (97186d3): in coverage-floor.sh, a malformed FLOORS entry with the :floor dropped left min as the whole token, so the awk comparison errored → if awk read that as "not below floor" → printed ok and never set status=1. A typo in the ratchet table silently disabled the check for that package. Added a package:INT shape guard before the awk call.

Two low-sev, non-blocking notes: the floor step re-runs go test (without -race) for both packages, duplicating the main -race -cover ./... run — fine as a simplicity tradeoff; and a compile error surfaces as "could not read coverage" with the go stderr swallowed (fails closed, just misleading when run standalone).

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 97186d3. Configure here.}