Skip to content

Migrate partner fixtures to employment_income_before_lsr input#8920

Merged
MaxGhenis merged 1 commit into
PolicyEngine:mainfrom
MaxGhenis:codex/fixture-employment-income
Jul 6, 2026
Merged

Migrate partner fixtures to employment_income_before_lsr input#8920
MaxGhenis merged 1 commit into
PolicyEngine:mainfrom
MaxGhenis:codex/fixture-employment-income

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Summary

Fixes #8802. Migrates all 156 partner YAML test fixtures under tests/policy/baseline/partners/ from the derived employment_income input key to the actual input, employment_income_before_lsr. In the YAML runner's build_from_dict path, setting employment_income (a derived variable defined as adds = ["employment_income_before_lsr", "employment_income_behavioral_response"]) holds that value directly but leaves the real input, employment_income_before_lsr, at 0. Programs whose income sources list the input variable -- notably TANF's earned-income list (gov.hhs.tanf.cash.income.sources.earned and several state overrides) -- silently saw $0 earnings, so any test asserting an income-dependent output through that path was passing vacuously.

PR #8795 fixed the 4 SNAP partner fixtures this surfaced through; this PR completes the fixture-level cleanup the issue calls out.

Changes

  • changelog.d/partner-fixtures-employment-income-before-lsr.fixed.md
  • 156 fixture files: renamed 1,577 employment_income: input keys to employment_income_before_lsr: (person-level inputs only -- verified zero occurrences of employment_income in any output assertion across the target set before making this change).
  • 8 of those 156 files had output assertions that shift once real income reaches income-dependent formulas. Every one was individually traced against the program's actual rules (parameter thresholds, formula logic, income-source lists) before repinning -- see the delta table below. None looked like a real bug; all are explained mechanically.

Repin delta table

File Case Variable(s) Old (pinned) New (correct) Why
edge_cases/federal/childcare/head_start/la.yaml head_start_above_income_limit (la) head_start (child) 12360.81 0.0 AGI $50,000 vs. family-of-2 FPG $21,640 (2026) -- clearly income-ineligible (is_head_start_income_eligible requires AGI ≤ FPG). tanf/snap/ssi also correctly $0, so no categorical-eligibility path either.
edge_cases/federal/childcare/head_start/ma.yaml head_start_above_income_limit (ma) head_start (child) 18845.98 0.0 Identical fixture/mechanism to LA.
edge_cases/federal/childcare/head_start/wa.yaml head_start_above_income_limit (wa) head_start (child) 22625.97 0.0 Identical fixture/mechanism to LA.
edge_cases/federal/nutrition/school_meals/la.yaml school_meals_just_below_reduced_threshold_185pct (la) school_meal_tier, school_meal_daily_subsidy 0 (FREE), 7.15 1 (REDUCED), 6.45 FPG ratio computes to 1.8485 -- just below the 1.85 REDUCED cutoff (gov.usda.school_meals.income.limit.REDUCED = 1.85). Old pinned tier (FREE) required the bugged $0-income FPG ratio; real income puts the family solidly in the 1.30-1.85 REDUCED band, not FREE.
edge_cases/federal/nutrition/school_meals/la.yaml school_meals_just_above_reduced_threshold_paid_tier (la) school_meal_tier, school_meal_daily_subsidy 0 (FREE), 7.15 2 (PAID), 0.87 FPG ratio 1.8558 -- just above 1.85 -> PAID tier, matching the test's own name.
edge_cases/federal/nutrition/school_meals/wa.yaml school_meals_just_below_reduced_threshold_185pct (wa) school_meal_tier, school_meal_daily_subsidy 0 (FREE), 7.15 1 (REDUCED), 6.45 Identical fixture/mechanism to LA.
edge_cases/federal/nutrition/school_meals/wa.yaml school_meals_just_above_reduced_threshold_paid_tier (wa) school_meal_tier, school_meal_daily_subsidy 0 (FREE), 7.15 2 (PAID), 0.87 Identical fixture/mechanism to LA.
edge_cases/state/ma/eaedc.yaml ma_eaedc_disabled_arrangement_a (ma) ma_eaedc 5293.2 147.47 Old pinned value exactly equals ma_eaedc_standard_assistance (the unreduced max), confirming it reflects $0 counted income. With real $10,000/yr income flowing through ma_eaedc_countable_earned_income, MA's earned-income disregard (full disregard first 4 months, partial thereafter) leaves 8/12 months at $0 benefit and 4/12 months at $36.87 -- annual sum $147.47.
edge_cases/state/ma/eaedc.yaml ma_eaedc_elderly_arrangement_a (ma) ma_eaedc 5293.2 3299.6 Same mechanism at real $5,000/yr income: 4 months at $316.03 + 8 months at $254.43 = $3299.56, rounds to $3299.6.
edge_cases/state/ma/tanf.yaml tanf_size_2_single_parent_1_child (ma) ma_tafdc, tanf 9536.0 8636.0 MA has no ma_tanf; TANF is ma_tafdc, which reads ma_tcap_gross_earned_income -> gov.states.ma.dta.tcap.gross_income.earned, a list of employment_income_before_lsr (the generic HHS-default income source, not overridden by MA). With the bug, that was $0; with real $6,000/yr income, TAFDC's earned-income disregard reduces the annual grant by exactly $900.
edge_cases/state/ma/tanf.yaml tanf_size_3_single_parent_2_children (ma) ma_tafdc, tanf 11812.0 10912.0 Same mechanism, same $900/yr reduction.
edge_cases/state/ma/tanf.yaml tanf_size_4_couple_2_children (ma) ma_tafdc, tanf 13516.0 12616.0 Same mechanism, same $900/yr reduction.
impactica/2025.yaml impactica (2025) snap 4709.7 5422.5 Cross-program cascade: with the bug, Oregon TANF (or_tanf, which also falls through to the generic employment_income_before_lsr earned-income list) saw $0 earned income and paid a larger grant ($5,184/yr); SNAP counts TANF as unearned income, so SNAP's net income was inflated by that spurious TANF, capping the SNAP benefit at $4,709.70. With real $4,752/yr income, or_tanf correctly drops to $2,808/yr, which lowers SNAP's net income and, given the 30%-of-net-income offset, nets to a higher SNAP benefit of $5,422.50 -- verified by tracing or_tanf, snap_unearned_income, snap_earned_income, snap_net_income, and snap_max_allotment directly under both the buggy and fixed inputs.

Every case above was verified by rebuilding the exact SimulationBuilder().build_from_dict() path the YAML runner uses (not the remapping policyengine_us.Simulation), tracing the relevant parameter values and intermediate variables, and confirming the new number follows deterministically from the program's actual rules. None looked wrong against the program's rules, so nothing in this PR is reported as a new bug -- the repins are exactly the "reveal" the issue describes.

8 of the 8 TANF fixtures named in the task were checked; the other 5 (CA, CO, IL, NC, TX) needed no repin because those states override the generic HHS earned-income list with their own parameter (gov.states.{ca,co,il,nc,tx}.../earned.yaml), which lists the derived employment_income directly rather than employment_income_before_lsr -- so setting employment_income: as an input in build_from_dict happened to produce the correct value for those states' TANF programs even before this fix (the derived variable becomes a held input for that period). MA and WA (federal fallback via wa_tanf_countable_earned_income -> tanf_gross_earned_income) and the aggregate tanf output on the federal/TX fixture (which itself uses TX's employment_income-based list, so needed no repin) were the only ones actually exposed.

Test plan

  • Full tests/policy/baseline/partners/ suite: 732 passed before this change (with buggy $0 TANF-adjacent inputs, passing vacuously) and 732 passed after (with real inputs and 17 output values across 8 files repinned to their newly-correct values). Zero regressions, zero remaining failures.
  • Comprehensive leaf-value audit script (rebuilds every one of the 696 test cases / 5,745 output assertions directly, independent of pytest) confirms zero mismatches between pinned and recomputed values after the repin.
  • Full TANF test tree (gov/hhs/tanf/ + all 28 implemented state TANF program directories, 2617 cases): passes both before and after -- confirms the fixture migration doesn't touch or regress the underlying program logic, only the test inputs.
  • Verified zero employment_income keys exist in any output assertion across the 156-file target set (input-only migration, as required).

156 partner YAML test fixtures set the derived `employment_income`
variable instead of the actual input, `employment_income_before_lsr`.
In the YAML runner's build_from_dict path (which doesn't remap the
way policyengine_us.Simulation does), that left the real input at 0,
so programs whose income sources list it -- notably TANF's earned
list -- silently saw $0 earnings and the tests passed vacuously.

Renamed all 1,577 employment_income: input keys to
employment_income_before_lsr: across the partner fixture tree
(inputs only; verified zero occurrences in output assertions before
making the change). 8 of the 156 files had output assertions that
shift once real income reaches income-dependent formulas -- each was
traced against the program's actual rules (parameter thresholds,
formula logic, income-source lists) before repinning; none looked
like a bug, all are mechanically explained in the PR body.

Full partners suite: 732 passed before and after (0 regressions).
Full TANF test tree (generic + 28 state programs, 2617 cases):
passes unchanged, confirming this only fixes test inputs.

Fixes PolicyEngine#8802

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@MaxGhenis MaxGhenis merged commit dc692ff into PolicyEngine:main Jul 6, 2026
34 checks passed
hua7450 added a commit to hua7450/policyengine-us that referenced this pull request Jul 6, 2026
The second upstream merge brought PolicyEngine#8918 (IL TANF floors each monthly
grant to whole dollars per IDHS PM 10-01-02 and adds the $1 minimum):
10 il_tanf assertions drop by a few dollars (e.g. the partial-year case
becomes exactly 7 x 493 + 3 x 513 = 4,990). Derivation comments updated.
The upstream suite-wide before_lsr fixture migration (PolicyEngine#8920) and its
re-pins of old fixture layouts were superseded by our rewrites; its
migration of the named partner fixtures merged cleanly. The MA TAFDC
inclusive-boundary fix (PolicyEngine#8918) moved no pinned values.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hua7450 added a commit to hua7450/policyengine-us that referenced this pull request Jul 6, 2026
…yEngine#8920

Upstream PolicyEngine#8920 migrated the employment_income keys suite-wide; convert
the two remaining bare self_employment_income keys (my_friend_ben named
fixtures, both zero-valued) so no partner fixture sets any LSR-derived
input directly, and reword the changelog fragment to what this branch
adds beyond PolicyEngine#8920.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

YAML fixtures set employment_income (derived) instead of employment_income_before_lsr (input) — TANF sees $0 earnings

1 participant