fix(api): scope NULL branch in NullableDatetimeRangeFilter for start_date lower bounds [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT] - #70629
Open
waterWang wants to merge 2 commits into
Conversation
…date lower bounds For start_date lower bounds (gte/gt), the NULL branch now also requires end_date IS NULL. This prevents terminal dag runs with a NULL start_date (e.g. failed runs that never started) from matching every start_date_gte filter indefinitely. Closes apache#70627
waterWang
requested review from
bugraoz93,
choo121600,
ephraimbuddy,
henry3260,
jason810496,
pierrejeambrun,
rawwar and
shubhamraj-git
as code owners
July 28, 2026 15:20
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
Contributor
|
The suffix of the PR title needs to be changed. There have some garbled text. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
NullableDatetimeRangeFilterwas added in #66696 to replaceCOALESCE(column, now())with index-friendlyORpredicates. For lower bounds (gte/gt), the NULL branch passes unconditionally:This is correct for non-terminal rows (a not-yet-started task will eventually start). But it's wrong for terminal dag runs whose
start_dateisNULL— a failed/success run that never started will never have astart_date, yet it matches everystart_date_gtefilter forever.Root cause
When querying
GET /dags/~/dagRuns?start_date_gte=<recent-timestamp>, the response includes terminal dag runs from years ago (state: failed,start_date: null, populatedend_date) as if they had just occurred. Roughly 99% of a-logical_date-ordered page are these stale null-start_daterows.Fix
Add an
extra_null_conditionparameter toNullableDatetimeRangeFilter. Forstart_datelower bounds, the factory passesmodel.end_date.is_(None), so the NULL branch becomes:A genuinely pending run has
start_date IS NULL AND end_date IS NULL. A terminal run withNULL start_datehasend_date IS NOT NULLand is correctly excluded.Changes
parameters.py: Addextra_null_conditiontoNullableDatetimeRangeFilter; updatedatetime_range_filter_factoryto pass it forstart_datetest_parameters.py: Update existing lower-bound test to verifyend_date IS NULL; add test for terminal-run exclusionRelated issues
Fixes #70627