Skip to content
This repository was archived by the owner on Jul 2, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,27 @@ jobs:
needs: check-fork
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: actions/setup-python@v6
with:
python-version: "3.14"
- name: Check whether dependency freshness is relevant
id: freshness-scope
run: |
changed_files="$(git diff --name-only origin/${{ github.base_ref }}...HEAD)"
echo "$changed_files"
if echo "$changed_files" | grep -Eq '^(pyproject\.toml|uv\.lock|policyengine_us_data/|modal_app/|tests/|scripts/|\.github/scripts/check_policyengine_us_dependency\.py)'; then
echo "run=true" >> "$GITHUB_OUTPUT"
else
echo "run=false" >> "$GITHUB_OUTPUT"
fi
- name: Require current PolicyEngine US dependency
if: steps.freshness-scope.outputs.run == 'true'
run: python .github/scripts/check_policyengine_us_dependency.py --mode fail
- name: Skip dependency freshness for docs-only change
if: steps.freshness-scope.outputs.run != 'true'
run: echo "Skipping PolicyEngine US freshness check; no runtime or dependency files changed."

lint:
runs-on: ubuntu-latest
Expand Down
8 changes: 8 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

These instructions apply repository-wide.

## Repository status

`policyengine-us-data` is now a legacy/reference pipeline. Do not add new
production target surfaces, publication paths, or data-generation features here
unless the change is explicitly required for compatibility with an existing
release. Migrate new work to PolicyEngine/populace and Ledger source facts, with
certification owned by policyengine.py.

## Skills system

Canonical AI-facing engineering skills live under `docs/engineering/skills/`.
Expand Down
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# PolicyEngine US Data

> **Legacy data pipeline.** New US microdata construction, calibration, source
> fact loading, diagnostics, and certified release work should happen in
> [PolicyEngine/populace](https://github.com/PolicyEngine/populace) and the
> Ledger source-fact repository, then be certified through
> [policyengine.py](https://github.com/PolicyEngine/policyengine.py). This
> repository remains available for historical Enhanced CPS reproduction,
> compatibility maintenance, and migration reference while downstream imports are
> retired. Do not start new build or target-surface work here unless it is
> explicitly needed to preserve an existing release.

## Installation

While it is possible to install via PyPi:
Expand Down Expand Up @@ -41,6 +51,16 @@ The following SSA data sources are used in this project:

PolicyEngine constructs its representative household datasets through a multi-step pipeline. Public survey data is merged, stratified, and cloned to geographic variants per household. Each clone is simulated through PolicyEngine US with stochastic take-up, then calibrated via L0-regularized optimization against administrative targets at the national, state, and congressional district levels, producing geographically representative datasets.

This section documents the legacy implementation. The live replacement path is:

```text
Ledger source facts -> Populace build/calibration -> certified bundle -> policyengine.py release
```

New calibration targets should be represented as Ledger facts and consumed by
Populace. Do not add new hard-coded target values or new production publication
paths to this repository.

The Enhanced CPS (`make data-legacy`) produces a national-only calibrated dataset. For the current geography-specific pipeline, see [docs/calibration.md](docs/calibration.md).

The repo currently contains two calibration tracks:
Expand Down
8 changes: 7 additions & 1 deletion REPRODUCTION.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Reproduction Guide for Enhanced CPS Dataset

> **Historical reproduction guide.** These instructions reproduce older Enhanced
> CPS artifacts from `policyengine-us-data`. They are not the current
> certification path for new PolicyEngine US microdata. New builds should use
> Populace with Ledger source facts and should be promoted only through the
> `policyengine.py` release contract.

This guide provides step-by-step instructions for reproducing the Enhanced CPS dataset and validation results. For more details on the production of geography-specific datasets (including states, congressional districts and some cities, visit our [documentation](https://policyengine.github.io/policyengine-us-data/)).

## Prerequisites
Expand Down Expand Up @@ -265,4 +271,4 @@ If you use this dataset, please cite:
journal={International Journal of Microsimulation},
year={2024}
}
```
```
1 change: 1 addition & 0 deletions changelog.d/1174.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Mark `policyengine-us-data` as the legacy US data pipeline and direct new build, calibration, and certification work to Populace, Ledger source facts, and `policyengine.py`.
9 changes: 9 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Documentation

> **Legacy documentation.** This site describes the historical
> `policyengine-us-data` pipeline. Current US data construction and calibration
> work should be implemented in
> [PolicyEngine/populace](https://github.com/PolicyEngine/populace), using
> Ledger source facts and certification through
> [policyengine.py](https://github.com/PolicyEngine/policyengine.py). Keep these
> pages available for reproducing and auditing older Enhanced CPS releases, but
> do not treat them as the current production path.

This project uses [MyST Markdown](https://mystmd.org/) for documentation.

## Building Locally
Expand Down