Cast bool-bracket .calc() results to bool before bitwise-NOT#8919
Conversation
single_amount bracket parameters with amount_unit: bool return raw int64 (0/1) from .calc(), not a real boolean array. Applying ~ to that result performs a bitwise negation (~1 = -2, ~0 = -1, both truthy) instead of a logical NOT, which only produces the correct answer when the surrounding expression happens to mask the high bits back down via & with a genuine bool term. That's "correct by accident," and it broke for real in is_snap_ineligible_student (PolicyEngine#8763) once the masking AND was removed. Audited every one of the ~27 amount_unit: bool bracket parameters in the repo for live ~ usage on a raw .calc() result and fixed the two found: - medicaid_community_engagement_pass_through_eligible.py: snap_age_exempt from snap_work.general.age_threshold.exempted.calc(age), negated twice in the snap_pass_through AND-chain. Currently correct only because the leading `snap &` term masks it. - aca_required_contribution_percentage.py: income_eligible from gov.aca.ptc_income_eligibility.calc(magi_frac), negated in capped_out = above_fpl_limit & ~income_eligible. Currently correct only because `above_fpl_limit &` masks it. Every other consumer of the 27 bool-bracket parameters combines the .calc() result via |, &, where(), comparison, or multiplication rather than negating it directly, so no other live bug exists. Two parameters (gov.contrib.reconciliation.medicaid_work_requirement.age_range and gov.local.ca.la.general_relief.housing_subsidy.age_eligibility) have no consumers at all. Added boundary-age regression tests (15/16/59/60) to medicaid_community_engagement_pass_through_eligible.yaml locking the general SNAP work-requirement age exemption thresholds this formula depends on. The existing aca_required_contribution_percentage.yaml suite already locks the 400% FPL boundary the second fix touches. Addresses PolicyEngine#8780 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8919 +/- ##
============================================
- Coverage 100.00% 64.15% -35.85%
============================================
Files 3 2 -1
Lines 55 53 -2
============================================
- Hits 55 34 -21
- Misses 0 19 +19
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Merged over the codecov pair: the reported numbers are self-refuting for this diff — two one-line .astype(bool) casts cannot produce a −35.85% project drop, indicating a partial coverage upload corrupted both contexts. The full test suite is green, and the Medicaid (368) and ACA (270) trees exercising both cast lines pass locally and in CI. Fable review: primary fix plus the second live instance found by the 27-parameter audit (aca_required_contribution_percentage ~income_eligible), boundary tests at the 0/16/60 age brackets, repo otherwise verified clean of the pattern. |
Summary
Addresses #8780.
single_amountbracket parameters withamount_unit: boolreturn a rawint64array (0/1) from.calc(), not a genuine boolean array (a core-library quirk). Applying~(bitwise NOT) to that result performs a bitwise negation —~1 = -2,~0 = -1, both truthy in Python/NumPy — instead of a logical NOT. This produces the correct answer only by accident, when the surrounding expression happens to mask the high bits back down via&with a genuine bool term. That accidental correctness broke for real inis_snap_ineligible_student(#8763) once a masking&term was removed — this PR is the broader defensive audit that issue asked for.This PR addresses the two live instances found; the framework-level root-cause fix (
policyengine-core's bracket.calc()returningint64for bool brackets in the first place) belongs upstream and stays out of scope here, so #8780 stays open per its own framing.Primary fix
policyengine_us/variables/gov/hhs/medicaid/eligibility/medicaid_community_engagement_pass_through_eligible.py~snap_age_exemptbitwise-negates the rawint64bracket output. Today's output is correct only because the leadingsnap &term masks the high bits back down to the genuine logical NOT for real 0/1 inputs — "correct by accident," exactly like the prioris_snap_ineligible_studentincident. Fixed by casting toboolat the.calc()call site:Second live instance found during the audit
policyengine_us/variables/gov/aca/ptc/aca_required_contribution_percentage.pySame pattern:
~income_eligiblebitwise-negates a raw bracket.calc()result, masked into correctness only by the leadingabove_fpl_limit &term. Fixed identically with.astype(bool).Full audit of all 27
amount_unit: boolbracket parametersI enumerated every parameter YAML with
amount_unit: bool(grep -rl "amount_unit: bool" policyengine_us/parameters, 27 files) and, for each, traced every.calc()consumer inpolicyengine_us/variablesandpolicyengine_us/reforms, checking whether the result (or anything derived from it) is ever combined with~.gov.aca.ptc_income_eligibilityaca_required_contribution_percentage.py, see above). Other two consumers (is_aca_ptc_eligible.py,basic_health_program.py) only combine the result via|/where()— benign.gov.contrib.aca.ptc_700_fpl_cliff.income_eligibilityreforms/aca/aca_ptc_700_fpl_cliff.pycombines via&, never negates.gov.contrib.aca.ptc_additional_bracket.income_eligibilityreforms/aca/aca_ptc_additional_bracket.pycombines via&, never negates.gov.contrib.aca.ptc_simplified_bracket.income_eligibilityreforms/aca/aca_ptc_simplified_bracket.pycombines via&, never negates.gov.contrib.reconciliation.medicaid_work_requirement.age_rangegov.hhs.medicaid.eligibility.work_requirements.age_rangemedicaid_work_requirement_eligible.pyuses the.calc()result only insidewhere(work_required_age, ...). A downstream~inis_medicaid_ineligible_due_to_work_requirement.pynegates the fully-resolvedmedicaid_work_requirement_eligibleVariable (viaperson(...), which is always properly typed bool), not the raw bracket output — different, safe pattern.gov.irs.ald.alimony_expense.divorce_year_thresholdalimony_expense_ald.pycombines via multiplication, never negates.gov.local.ca.la.general_relief.housing_subsidy.age_eligibilitygov.states.ar.dhs.medicaid.work_requirements.age_rangear_medicaid_work_requirement_subject.pycombines via&, never negates. A downstream~subjectinar_medicaid_work_requirement_eligible.pynegates the resolved Variable viaperson(...), same safe pattern as #6.gov.states.ca.chhs.eligible_regardless_of_immigration_statusis_ca_medicaid_immigration_status_eligible.pycombines via|, never negates.gov.states.co.hcpf.chp.childco_chp_eligible.pycombines via|(is_age_eligible = is_pregnant | is_child), never negates.gov.states.de.tax.income.subtractions.exclusions.pension.age_thresholdde_pension_exclusion.pyuses the result only insidewhere(...)(twice), never negates.gov.states.il.hfs.hbwd.eligibility.ageil_hbwd_age_eligible.pyreturns the.calc()result directly, no negation.gov.states.il.isbe.pfae.eligibility.age_rangeil_pfae_age_eligible_child.pyreturns the.calc()result directly, no negation.gov.states.il.rta.cta.reduced_fare_program.age_threshold.childil_cta_children_reduced_fare_eligible.pyreturns the.calc()result directly, no negation.gov.states.ma.dot.mbta.income_eligible_reduced_fares.age_thresholdma_mbta_income_eligible_reduced_fare_eligible.pyreturns the.calc()result directly, no negation.gov.states.mi.tax.income.deductions.retirement.expanded.birth_yearmi_expanded_retirement_benefits_deduction_eligible.pyreturns the.calc()result directly (orFalse), no negation.gov.states.mi.tax.income.deductions.retirement.tier_three.age_eligibilitymi_retirement_benefits_deduction_tier_three_eligible.pyreturns the.calc()result directly, no negation.gov.states.mi.tax.income.deductions.standard.tier_three.birth_yearmi_standard_deduction_tier_three_eligible.pyreturns the.calc()result directly, no negation.gov.states.mi.tax.income.deductions.standard.tier_two.birth_yearmi_standard_deduction_tier_two_eligible.pyreturns the.calc()result directly, no negation.gov.states.nc.ncdhhs.scca.age.groupnc_scca_age_group.pyfeeds the result intowhere()/array indexing, never negates. (Unrelated observation: the YAML'samount_unit: booltag doesn't match its integer 1/2/3 bracket amounts — a pre-existing metadata mismatch, out of scope here.)gov.states.tx.dart.reduced_fare.age_threshold.childtx_dart_reduced_fare_age_eligible.pycombines via|, never negates.gov.states.ut.tax.income.credits.ctc.child_age_thresholdut_ctc_potential.pycombines via&, never negates.gov.states.wa.dshs.sfa.student_pathway.age_eligiblewa_sfa_student_pathway_eligible.pycombines via&, never negates.gov.usda.snap.student.age_thresholdfa1f721d46, cited in #8780 as the precedent for this audit.gov.usda.snap.work_requirements.abawd.age_threshold.exemptedmeets_snap_abawd_work_requirements.pycombines via|, never negates.gov.usda.snap.work_requirements.general.age_threshold.exemptedmedicaid_community_engagement_pass_through_eligible.py, see above). Its other consumer,meets_snap_general_work_requirements.py, combines via|and is unaffected.Also confirmed benign per the issue's own framing:
gov/states/co/hcpf/chp/co_chp.py's~is_pregnant * p.copays[element].calc(income_level)— the~applies tois_pregnant(a genuine bool Variable), not to the bracket.calc()result, which is combined via multiplication rather than negation. No change needed.Two broad sanity-check greps across the entire
policyengine_us/variablesandpolicyengine_us/reformstrees (same-line~....calc(and cross-line file-level co-occurrence of.calc(and~) turned up no additional live instances beyond the two fixed here.Tests
policyengine_us/tests/policy/baseline/gov/hhs/medicaid/eligibility/medicaid_community_engagement_pass_through_eligible.yaml— added 4 boundary-age regression cases (ages 15, 16, 59, 60) locking thegov.usda.snap.work_requirements.general.age_threshold.exemptedbracket boundaries (exempt under 16 or 60+, non-exempt 16–59) that the primary fix touches.policyengine_us/tests/policy/baseline/gov/aca/ptc/aca_required_contribution_percentage.yamlsuite already locks the 400% FPL boundary (e.g. "2026 ACA contribution percentage is zero above 400 percent FPL" atmagi_fraction: 4.01) that exercises the exactcapped_outbranch this fix touches.Pass/fail counts (via
uv run policyengine-core test <path> -c policyengine_us):medicaid_community_engagement_pass_through_eligible.yaml: 9 passed (5 pre-existing + 4 new)policyengine_us/tests/policy/baseline/gov/hhs/medicaid/(full tree): 368 passedpolicyengine_us/tests/policy/baseline/gov/usda/snap/eligibility/(full tree): 188 passedpolicyengine_us/tests/policy/baseline/gov/aca/(full tree): 270 passedpolicyengine_us/tests/policy/contrib/aca/(all 3 reform variants): 38 passedpolicyengine_us/tests/policy/baseline/gov/hhs/basic_health_program/: 31 passedAll green, no regressions — both fixes are behavior-preserving (the buggy code was masked into numeric correctness today; these casts remove the fragility without changing any current output).
ruff formatandruff checkpass clean on both modified.pyfiles.Changelog fragment:
changelog.d/bool-bracket-negation-casts.fixed.md.