Lift populace-fit's scikit-learn<1.9 pin via quantile-forest 1.4.2 (#318)

[MaxGhenis]

Fixes #318.

Root cause

populace-fit pinned scikit-learn>=1.5,<1.9 because quantile-forest imported sklearn.tree._tree.DTYPE, a symbol scikit-learn 1.9 removed (sklearn 1.9 also changed the sample-weight signature of the forest bootstrap helpers _generate_sample_indices / _get_n_samples_bootstrap). The <1.9 cap made populace-fit uninstallable alongside any scikit-learn>=1.9 consumer: installing it editable into such an environment (the gate-1 baseline run in populace-dynamics) downgraded scikit-learn 1.9 → 1.8, which shifted C2ST classifier internals enough to break float-pinned noise-floor reproduction tests built under 1.9. The two packages could not coexist in one venv.

Fix

Bump quantile-forest>=1.3 → >=1.4.2 and drop the sklearn cap (>=1.5,<1.9 → >=1.5).

Why 1.4.2 is the right target (upstream, evidence-checked):

• quantile-forest PR zillow/quantile-forest#152 ("Support scikit-learn 1.9 (DTYPE removed from sklearn.tree._tree)") restores DTYPE via a try/except fallback to np.float32 (its value on every sklearn version). Merged 2026-06-10.
• Follow-up PR #153 fixes the sklearn-1.9 sample_weight bootstrap signature change. Merged after Use 2024 SOI filing-season EITC totals #152.
• quantile-forest 1.4.2 was released 2026-06-21; the tag range v1.4.1...v1.4.2 contains both Use 2024 SOI filing-season EITC totals #152 and Add UK bus-spending imputation plan and DfT calibration targets #153. It is therefore the first release compatible with sklearn 1.9. (1.4.1, the version the lockfile previously resolved, predates both.)
• quantile-forest itself never pinned an upper bound on scikit-learn — the <1.9 cap was purely populace-fit's defensive pin, so lifting it is safe once the floor guarantees a compatible quantile-forest.

Why no upper bound on sklearn: populace-fit is the only package in the repo that imports quantile_forest (verified: only packages/populace-fit/pyproject.toml, src/populace/fit/qrf.py, tests/test_qrf.py reference it). Its own sklearn usage is version-stable — the QRF forests run their own weighted bootstrap and call .fit() without sample_weight (so they don't hit the sklearn-1.9-changed bootstrap path directly), and the sign gate is a native HistGradientBoostingClassifier. The evidence below confirms the suite is green on both 1.8 and 1.9.

This is the "upstream / lift the pin" direction from the issue (option 1), which is preferable to making quantile-forest optional (option 2) because the QRF backend is the package's core model.

Both-environment test evidence

Two isolated uv venvs (Python 3.13.9), each an editable install of populace-frame + populace-fit + pytest, differing only in scikit-learn. quantile-forest==1.4.2 in both; all other deps identical (numpy 2.5.1, pandas 3.0.3, scipy 1.18.0, joblib 1.5.3, threadpoolctl 3.6.0).

scikit-learn 1.8.0 (pip install "scikit-learn<1.9"):

collected 76 items
packages/populace-fit/tests/test_compat.py ......
packages/populace-fit/tests/test_dataframe_fit.py .............................
packages/populace-fit/tests/test_model.py ..............
packages/populace-fit/tests/test_qrf.py .........................
packages/populace-fit/tests/test_weighted_fit_contract.py ..
======================== 76 passed in 97.37s ========================

scikit-learn 1.9.0 (pip install "scikit-learn>=1.9"):

sklearn 1.9.0 | quantile_forest 1.4.2
collected 76 items
packages/populace-fit/tests/test_compat.py ......
packages/populace-fit/tests/test_dataframe_fit.py .............................
packages/populace-fit/tests/test_model.py ..............
packages/populace-fit/tests/test_qrf.py .........................
packages/populace-fit/tests/test_weighted_fit_contract.py ..
======================== 76 passed in 14.99s ========================

76/76 pass under both. A scikit-learn>=1.9 repo can now install populace-fit into the same environment without a downgrade.

Changelog

This repo has no changelog infrastructure — no changelog.d/, no changelog.yaml, no CHANGELOG.md, and no towncrier config in any pyproject.toml — so no fragment is added. (Towncrier is a PolicyEngine country-model-repo convention, not used in populace.)

Files changed

• packages/populace-fit/pyproject.toml — bump quantile-forest floor, drop sklearn cap, update the rationale comment.
• uv.lock — quantile-forest 1.4.1 → 1.4.2 (adds cp314 wheels too).
• packages/populace-fit/README.md — document the sklearn coexistence guarantee under Dependencies.

🤖 Generated with Claude Code