Skip to content

Add UK bus-spending imputation plan and DfT calibration targets#153

Open
vahid-ahmadi wants to merge 2 commits into
mainfrom
uk-bus-fare-imputation-calibration
Open

Add UK bus-spending imputation plan and DfT calibration targets#153
vahid-ahmadi wants to merge 2 commits into
mainfrom
uk-bus-fare-imputation-calibration

Conversation

@vahid-ahmadi

Copy link
Copy Markdown

Problem

The published Populace UK population miscalibrates both bus consumption variables (verified against the incumbent enhanced-FRS build and DfT Annual Bus Statistics, at the same simulated year):

variable Populace UK 2023 enhanced-FRS DfT anchor
bus_fare_spending £7.4bn raw, 17.7% of hh, £1,442/spender £3.9bn, 27.4%, £456 ~£4.0bn
bus_subsidy_spending £1.0bn, 18.2% £3.5bn, 33.1% ~£3.5bn

Fare is ~2× too high and over-concentrated; subsidy is ~3.6× too low — exactly the failure mode that occurs when the survey imputation isn't anchored to DfT. The incumbent build applies a DfT calibration (calibrate_bus_fare_spending / calibrate_bus_subsidy_spending); that step was never ported to the Populace UK build.

What this adds (mirrors build/us)

  • build/uk/bus_calibration_targets.py — a TargetRegistry with DfT-anchored household-sum targets for bus_fare_spending (~£4.0bn) and bus_subsidy_spending (~£3.5bn): England DfT BUS05a/BUS05b totals uplifted to UK by the ONS mid-2023 population ratio (1.18). Feeds the calibration solver and aggregate_admin_gate.
  • build/uk/bus_source_stages.json — a source manifest declaring the two imputation stages (LCFS fares, ETB subsidy) as weighted-QRF draws, each followed by a support_clip to the donor's realized range (kills the over-concentrated high-value tail).
  • build/uk/bus_imputation.pyuk_bus_plan(), donor graph, stage names; mirrors us_plan (transforms injected by the caller — no stubs/fallbacks).
  • tests/test_uk_bus.py — plan assembly, donor citations, manifest contract, target value/provenance.

Notes

  • No incumbent data-package reference (passes test_no_incumbent_data_package_references_in_live_tree); survey microdata is supplied as plain tables at call time.
  • Executable QRF/clip handlers are injected by the build runtime, as for the US stages — this PR adds the declarative plan + targets, not the build orchestration, so it does not by itself regenerate populace_uk_2023.
  • Follow-up: rail_subsidy_spending and the road-fuel litre proxies share the same missing-anchor class of bug.

Full populace-build test suite passes locally (uv run pytest), ruff clean.

🤖 Generated with Claude Code

The published Populace UK population miscalibrates the two bus consumption
variables: weighted bus_fare_spending lands ~2x the DfT fare total (and is
concentrated in too few households at implausibly high amounts), while
bus_subsidy_spending lands well below the DfT net-support total. The incumbent
enhanced-FRS build anchors both to DfT Annual Bus Statistics; that anchoring
was never ported to the Populace UK build.

This adds the missing UK build artifacts, mirroring build/us conventions:

- build/uk/bus_calibration_targets.py: a TargetRegistry with DfT-anchored
  household-sum targets for bus_fare_spending (~GBP 4.0bn) and
  bus_subsidy_spending (~GBP 3.5bn) — England DfT BUS05a/BUS05b totals uplifted
  to UK by the ONS mid-2023 population ratio. These feed the calibration solver
  and aggregate_admin_gate.
- build/uk/bus_source_stages.json: a source manifest declaring the two
  imputation stages (LCFS fares, ETB subsidy) as weighted-QRF draws over
  household predictors, each followed by a support_clip to the donor's realized
  range so the imputation does not over-concentrate spending.
- build/uk/bus_imputation.py: uk_bus_plan(), the donor graph and stage names,
  mirroring us_plan (transforms injected by the caller; no stubs/fallbacks).
- tests/test_uk_bus.py: plan assembly + donor citations, manifest contract,
  and target value/provenance checks.

Executable stage transforms are injected by the build caller; survey microdata
is supplied as plain tables at call time. No incumbent data-package reference.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vahid-ahmadi vahid-ahmadi requested a review from MaxGhenis June 20, 2026 17:53
Adds calibrate_bus_spending_levels(): the executable level-calibration the
declarative targets and stages describe. It mirrors the incumbent
enhanced-FRS step exactly (scale = target / actual; column *= scale), scaling
each bus variable's weighted total to its DfT target. Value-scaling only — the
set of spending households and the distribution shape are unchanged, as in the
incumbent build (the spender share is set by the imputation, not calibration).

Verified on the published populace_uk_2023 household table:
  bus_fare_spending     7.360bn -> 4.000bn  (scale 0.543)
  bus_subsidy_spending  0.968bn -> 3.500bn  (scale 3.615)
both landing exactly on their DfT targets.

Tests cover exact-target scaling, spender-set preservation, registry-default
targets, and clear errors on a missing column or a zero aggregate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor

Holding off on merging this as-is. The imputation plan shape is fine, but the calibration targets need to be Ledger-backed before they land in Populace: _DFT_ENGLAND_FARE_RECEIPTS, _DFT_ENGLAND_NET_GOVERNMENT_SUPPORT, and the England-to-UK uplift should be source facts/derived facts with lineage in Ledger, and Populace should reference those facts rather than embed target values in Python. This keeps the no target values in the live repo rule intact and makes the dashboard/provenance surface auditable.

MaxGhenis added a commit that referenced this pull request Jul 6, 2026
) (#322)

populace-fit capped scikit-learn below 1.9 because quantile-forest imported
sklearn.tree._tree.DTYPE, which sklearn 1.9 removed (and sklearn 1.9 also
changed the sample-weight signature of the forest bootstrap helpers). The cap
made the package uninstallable alongside any sklearn>=1.9 consumer: installing
populace-fit into such an environment downgraded scikit-learn 1.9 -> 1.8,
breaking float-pinned reproduction tests built under 1.9 (hit in
populace-dynamics).

quantile-forest 1.4.2 (2026-06-21) tracks both sklearn-1.9 changes
(zillow/quantile-forest#152 restores DTYPE via an np.float32 fallback, #153
fixes the sample-weight bootstrap). Requiring quantile-forest>=1.4.2 lets the
sklearn<1.9 cap be dropped (>=1.5), so populace-fit installs and passes its
suite under both sklearn 1.8 and 1.9 and can coexist in one environment with an
sklearn-1.9 consumer.

populace-fit is the only package that imports quantile_forest (verified across
the repo). Its QRF forests do their own weighted bootstrap and call .fit()
without sample_weight, and the sign gate is a native
HistGradientBoostingClassifier, so nothing in this package touches the
sklearn-1.9-changed forest bootstrap path directly.

Test evidence (populace-fit suite, editable install of populace-frame +
populace-fit, quantile-forest 1.4.2 in both):
- scikit-learn 1.8.0: 76 passed
- scikit-learn 1.9.0: 76 passed

Also documents the coexistence guarantee in populace-fit's README.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

@MaxGhenis MaxGhenis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diagnosis and the plan shape are right: published populace UK bus fares are ~2x the DfT total and subsidy is well under, and anchoring to DfT BUS05 with a QRF-plus-support-clip imputation is the correct fix. The DfT figures check out — England fare receipts £3.4bn and net government support £3.0bn for the year ending March 2025 (note DfT revised these tables on 24 March 2026, and the precise tables are BUS05aii and BUS05i/ii).

But this has to be rebuilt against the architecture that landed after it opened, and it conflicts with main today (agent-assisted review):

  1. Spec-only packages. build/uk is now resource-only — build/uk/__init__.py moved to build/uk_runtime, so editing it is a modify/delete conflict, and test_spec_only_country_packages.py fails on any .py under build/uk. bus_imputation.py and bus_calibration.py are executable and belong in build/uk_runtime; bus_source_stages.json can stay as a spec.

  2. TargetSpec no longer takes aggregation. bus_calibration_targets.py passes aggregation="sum" and test_uk_bus.py asserts it — both break on rebase. The green CI here is stale (it ran when the branch was 107 commits behind main).

  3. Targets must be Ledger-backed, per my earlier comment and the ledger_targets/LedgerTargetReference surface that has since landed. Don't embed the £3.4bn fare receipts, the £3.0bn net support, or the 68.3/57.7 uplift as Python literals — the DfT facts and the England-to-UK derivation belong in Ledger with lineage, referenced from here.

Two substantive points for the rebuild:

  • Vintage: the population is 2023 but the targets are period-2025 DfT. Calibrate 2023 to the year-ending-March-2023 figure, or say why 2025 is the right anchor.
  • The England-to-UK population uplift is crude — England receipts already include London, the dominant bus market, so scaling by population likely overstates the UK total, and Scotland/Wales fund their own subsidy. A per-nation or GB anchor would be better; at minimum caveat it.

Also, the executable calibrator grew this past its "declarative plan + targets" description — I'd split it out or move it to runtime as part of the rework. Direction is good; this is a rework, not a tweak.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants