From 4234764bdec3438c5d29ed3b65f902a74bb115b9 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Sat, 20 Jun 2026 14:50:48 -0400 Subject: [PATCH] Mark policyengine-us-data as legacy --- .github/workflows/pr.yaml | 16 ++++++++++++++++ AGENTS.md | 8 ++++++++ README.md | 20 ++++++++++++++++++++ REPRODUCTION.md | 8 +++++++- changelog.d/1174.changed.md | 1 + docs/README.md | 9 +++++++++ 6 files changed, 61 insertions(+), 1 deletion(-) create mode 100644 changelog.d/1174.changed.md diff --git a/.github/workflows/pr.yaml b/.github/workflows/pr.yaml index ccf34823d..5b48f0a77 100644 --- a/.github/workflows/pr.yaml +++ b/.github/workflows/pr.yaml @@ -41,11 +41,27 @@ jobs: needs: check-fork steps: - uses: actions/checkout@v6 + with: + fetch-depth: 0 - uses: actions/setup-python@v6 with: python-version: "3.14" + - name: Check whether dependency freshness is relevant + id: freshness-scope + run: | + changed_files="$(git diff --name-only origin/${{ github.base_ref }}...HEAD)" + echo "$changed_files" + if echo "$changed_files" | grep -Eq '^(pyproject\.toml|uv\.lock|policyengine_us_data/|modal_app/|tests/|scripts/|\.github/scripts/check_policyengine_us_dependency\.py)'; then + echo "run=true" >> "$GITHUB_OUTPUT" + else + echo "run=false" >> "$GITHUB_OUTPUT" + fi - name: Require current PolicyEngine US dependency + if: steps.freshness-scope.outputs.run == 'true' run: python .github/scripts/check_policyengine_us_dependency.py --mode fail + - name: Skip dependency freshness for docs-only change + if: steps.freshness-scope.outputs.run != 'true' + run: echo "Skipping PolicyEngine US freshness check; no runtime or dependency files changed." lint: runs-on: ubuntu-latest diff --git a/AGENTS.md b/AGENTS.md index 46b7f9c85..ca9643976 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,6 +2,14 @@ These instructions apply repository-wide. +## Repository status + +`policyengine-us-data` is now a legacy/reference pipeline. Do not add new +production target surfaces, publication paths, or data-generation features here +unless the change is explicitly required for compatibility with an existing +release. Migrate new work to PolicyEngine/populace and Ledger source facts, with +certification owned by policyengine.py. + ## Skills system Canonical AI-facing engineering skills live under `docs/engineering/skills/`. diff --git a/README.md b/README.md index 0bf2e4310..9acfdc75d 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,15 @@ # PolicyEngine US Data +> **Legacy data pipeline.** New US microdata construction, calibration, source +> fact loading, diagnostics, and certified release work should happen in +> [PolicyEngine/populace](https://github.com/PolicyEngine/populace) and the +> Ledger source-fact repository, then be certified through +> [policyengine.py](https://github.com/PolicyEngine/policyengine.py). This +> repository remains available for historical Enhanced CPS reproduction, +> compatibility maintenance, and migration reference while downstream imports are +> retired. Do not start new build or target-surface work here unless it is +> explicitly needed to preserve an existing release. + ## Installation While it is possible to install via PyPi: @@ -41,6 +51,16 @@ The following SSA data sources are used in this project: PolicyEngine constructs its representative household datasets through a multi-step pipeline. Public survey data is merged, stratified, and cloned to geographic variants per household. Each clone is simulated through PolicyEngine US with stochastic take-up, then calibrated via L0-regularized optimization against administrative targets at the national, state, and congressional district levels, producing geographically representative datasets. +This section documents the legacy implementation. The live replacement path is: + +```text +Ledger source facts -> Populace build/calibration -> certified bundle -> policyengine.py release +``` + +New calibration targets should be represented as Ledger facts and consumed by +Populace. Do not add new hard-coded target values or new production publication +paths to this repository. + The Enhanced CPS (`make data-legacy`) produces a national-only calibrated dataset. For the current geography-specific pipeline, see [docs/calibration.md](docs/calibration.md). The repo currently contains two calibration tracks: diff --git a/REPRODUCTION.md b/REPRODUCTION.md index c5e6592d4..35b406d4c 100644 --- a/REPRODUCTION.md +++ b/REPRODUCTION.md @@ -1,5 +1,11 @@ # Reproduction Guide for Enhanced CPS Dataset +> **Historical reproduction guide.** These instructions reproduce older Enhanced +> CPS artifacts from `policyengine-us-data`. They are not the current +> certification path for new PolicyEngine US microdata. New builds should use +> Populace with Ledger source facts and should be promoted only through the +> `policyengine.py` release contract. + This guide provides step-by-step instructions for reproducing the Enhanced CPS dataset and validation results. For more details on the production of geography-specific datasets (including states, congressional districts and some cities, visit our [documentation](https://policyengine.github.io/policyengine-us-data/)). ## Prerequisites @@ -265,4 +271,4 @@ If you use this dataset, please cite: journal={International Journal of Microsimulation}, year={2024} } -``` \ No newline at end of file +``` diff --git a/changelog.d/1174.changed.md b/changelog.d/1174.changed.md new file mode 100644 index 000000000..05cc024ab --- /dev/null +++ b/changelog.d/1174.changed.md @@ -0,0 +1 @@ +Mark `policyengine-us-data` as the legacy US data pipeline and direct new build, calibration, and certification work to Populace, Ledger source facts, and `policyengine.py`. diff --git a/docs/README.md b/docs/README.md index 00f3c7fa4..96aa6f94e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,5 +1,14 @@ # Documentation +> **Legacy documentation.** This site describes the historical +> `policyengine-us-data` pipeline. Current US data construction and calibration +> work should be implemented in +> [PolicyEngine/populace](https://github.com/PolicyEngine/populace), using +> Ledger source facts and certification through +> [policyengine.py](https://github.com/PolicyEngine/policyengine.py). Keep these +> pages available for reproducing and auditing older Enhanced CPS releases, but +> do not treat them as the current production path. + This project uses [MyST Markdown](https://mystmd.org/) for documentation. ## Building Locally