Migrate partner fixtures to employment_income_before_lsr input#8920
Merged
MaxGhenis merged 1 commit intoJul 6, 2026
Merged
Conversation
156 partner YAML test fixtures set the derived `employment_income` variable instead of the actual input, `employment_income_before_lsr`. In the YAML runner's build_from_dict path (which doesn't remap the way policyengine_us.Simulation does), that left the real input at 0, so programs whose income sources list it -- notably TANF's earned list -- silently saw $0 earnings and the tests passed vacuously. Renamed all 1,577 employment_income: input keys to employment_income_before_lsr: across the partner fixture tree (inputs only; verified zero occurrences in output assertions before making the change). 8 of the 156 files had output assertions that shift once real income reaches income-dependent formulas -- each was traced against the program's actual rules (parameter thresholds, formula logic, income-source lists) before repinning; none looked like a bug, all are mechanically explained in the PR body. Full partners suite: 732 passed before and after (0 regressions). Full TANF test tree (generic + 28 state programs, 2617 cases): passes unchanged, confirming this only fixes test inputs. Fixes PolicyEngine#8802 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hua7450
added a commit
to hua7450/policyengine-us
that referenced
this pull request
Jul 6, 2026
The second upstream merge brought PolicyEngine#8918 (IL TANF floors each monthly grant to whole dollars per IDHS PM 10-01-02 and adds the $1 minimum): 10 il_tanf assertions drop by a few dollars (e.g. the partial-year case becomes exactly 7 x 493 + 3 x 513 = 4,990). Derivation comments updated. The upstream suite-wide before_lsr fixture migration (PolicyEngine#8920) and its re-pins of old fixture layouts were superseded by our rewrites; its migration of the named partner fixtures merged cleanly. The MA TAFDC inclusive-boundary fix (PolicyEngine#8918) moved no pinned values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hua7450
added a commit
to hua7450/policyengine-us
that referenced
this pull request
Jul 6, 2026
…yEngine#8920 Upstream PolicyEngine#8920 migrated the employment_income keys suite-wide; convert the two remaining bare self_employment_income keys (my_friend_ben named fixtures, both zero-valued) so no partner fixture sets any LSR-derived input directly, and reword the changelog fragment to what this branch adds beyond PolicyEngine#8920. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #8802. Migrates all 156 partner YAML test fixtures under
tests/policy/baseline/partners/from the derivedemployment_incomeinput key to the actual input,employment_income_before_lsr. In the YAML runner'sbuild_from_dictpath, settingemployment_income(a derived variable defined asadds = ["employment_income_before_lsr", "employment_income_behavioral_response"]) holds that value directly but leaves the real input,employment_income_before_lsr, at 0. Programs whose income sources list the input variable -- notably TANF's earned-income list (gov.hhs.tanf.cash.income.sources.earnedand several state overrides) -- silently saw $0 earnings, so any test asserting an income-dependent output through that path was passing vacuously.PR #8795 fixed the 4 SNAP partner fixtures this surfaced through; this PR completes the fixture-level cleanup the issue calls out.
Changes
changelog.d/partner-fixtures-employment-income-before-lsr.fixed.mdemployment_income:input keys toemployment_income_before_lsr:(person-level inputs only -- verified zero occurrences ofemployment_incomein any output assertion across the target set before making this change).Repin delta table
edge_cases/federal/childcare/head_start/la.yamlhead_start_above_income_limit (la)head_start(child)is_head_start_income_eligiblerequires AGI ≤ FPG).tanf/snap/ssialso correctly $0, so no categorical-eligibility path either.edge_cases/federal/childcare/head_start/ma.yamlhead_start_above_income_limit (ma)head_start(child)edge_cases/federal/childcare/head_start/wa.yamlhead_start_above_income_limit (wa)head_start(child)edge_cases/federal/nutrition/school_meals/la.yamlschool_meals_just_below_reduced_threshold_185pct (la)school_meal_tier,school_meal_daily_subsidygov.usda.school_meals.income.limit.REDUCED = 1.85). Old pinned tier (FREE) required the bugged $0-income FPG ratio; real income puts the family solidly in the 1.30-1.85 REDUCED band, not FREE.edge_cases/federal/nutrition/school_meals/la.yamlschool_meals_just_above_reduced_threshold_paid_tier (la)school_meal_tier,school_meal_daily_subsidyedge_cases/federal/nutrition/school_meals/wa.yamlschool_meals_just_below_reduced_threshold_185pct (wa)school_meal_tier,school_meal_daily_subsidyedge_cases/federal/nutrition/school_meals/wa.yamlschool_meals_just_above_reduced_threshold_paid_tier (wa)school_meal_tier,school_meal_daily_subsidyedge_cases/state/ma/eaedc.yamlma_eaedc_disabled_arrangement_a (ma)ma_eaedcma_eaedc_standard_assistance(the unreduced max), confirming it reflects $0 counted income. With real $10,000/yr income flowing throughma_eaedc_countable_earned_income, MA's earned-income disregard (full disregard first 4 months, partial thereafter) leaves 8/12 months at $0 benefit and 4/12 months at $36.87 -- annual sum $147.47.edge_cases/state/ma/eaedc.yamlma_eaedc_elderly_arrangement_a (ma)ma_eaedcedge_cases/state/ma/tanf.yamltanf_size_2_single_parent_1_child (ma)ma_tafdc,tanfma_tanf; TANF isma_tafdc, which readsma_tcap_gross_earned_income->gov.states.ma.dta.tcap.gross_income.earned, a list ofemployment_income_before_lsr(the generic HHS-default income source, not overridden by MA). With the bug, that was $0; with real $6,000/yr income, TAFDC's earned-income disregard reduces the annual grant by exactly $900.edge_cases/state/ma/tanf.yamltanf_size_3_single_parent_2_children (ma)ma_tafdc,tanfedge_cases/state/ma/tanf.yamltanf_size_4_couple_2_children (ma)ma_tafdc,tanfimpactica/2025.yamlimpactica (2025)snapor_tanf, which also falls through to the genericemployment_income_before_lsrearned-income list) saw $0 earned income and paid a larger grant ($5,184/yr); SNAP counts TANF as unearned income, so SNAP's net income was inflated by that spurious TANF, capping the SNAP benefit at $4,709.70. With real $4,752/yr income,or_tanfcorrectly drops to $2,808/yr, which lowers SNAP's net income and, given the 30%-of-net-income offset, nets to a higher SNAP benefit of $5,422.50 -- verified by tracingor_tanf,snap_unearned_income,snap_earned_income,snap_net_income, andsnap_max_allotmentdirectly under both the buggy and fixed inputs.Every case above was verified by rebuilding the exact
SimulationBuilder().build_from_dict()path the YAML runner uses (not the remappingpolicyengine_us.Simulation), tracing the relevant parameter values and intermediate variables, and confirming the new number follows deterministically from the program's actual rules. None looked wrong against the program's rules, so nothing in this PR is reported as a new bug -- the repins are exactly the "reveal" the issue describes.8 of the 8 TANF fixtures named in the task were checked; the other 5 (CA, CO, IL, NC, TX) needed no repin because those states override the generic HHS earned-income list with their own parameter (
gov.states.{ca,co,il,nc,tx}.../earned.yaml), which lists the derivedemployment_incomedirectly rather thanemployment_income_before_lsr-- so settingemployment_income:as an input inbuild_from_dicthappened to produce the correct value for those states' TANF programs even before this fix (the derived variable becomes a held input for that period). MA and WA (federal fallback viawa_tanf_countable_earned_income->tanf_gross_earned_income) and the aggregatetanfoutput on the federal/TX fixture (which itself uses TX'semployment_income-based list, so needed no repin) were the only ones actually exposed.Test plan
tests/policy/baseline/partners/suite: 732 passed before this change (with buggy $0 TANF-adjacent inputs, passing vacuously) and 732 passed after (with real inputs and 17 output values across 8 files repinned to their newly-correct values). Zero regressions, zero remaining failures.gov/hhs/tanf/+ all 28 implemented state TANF program directories,2617cases): passes both before and after -- confirms the fixture migration doesn't touch or regress the underlying program logic, only the test inputs.employment_incomekeys exist in any output assertion across the 156-file target set (input-only migration, as required).