Make deadline reads and serialization robust to dynamic/malformed intervals by seanghaeli · Pull Request #68919 · apache/airflow

seanghaeli · 2026-06-23T21:30:28Z

DeadlineAlert.interval is a JSON column (serialized timedelta or dynamic VariableInterval), but the /ui/dags/{dag_id}/deadlineAlerts response declared interval: float — any alert with a dict-shaped interval failed validation and the endpoint returned 500, breaking the run-page deadline badge.

Fix: interval becomes float | None with a validator (seconds for fixed intervals, None for dynamic), and interval is dropped from order_by (sorting a JSON column sorts by structure, not duration).

Robustness fixes from earlier revisions split out to #70421 per review.

Closes: #69245

pierrejeambrun

Thanks for the PR.

I would make the code less verbose and keep only relevant comments/pieces. There are too many big comments some of them needs to be removed completely, some of them needs to be trimmed.

Overall looking good, just a few suggestions.

Another pair of eyes would be great on this.

pierrejeambrun

Some changes doesn't seem related / necessary for fixing the related issue.

Can you keep the change as minimal as possible (for the issue we are trying to solve) and separate the rest in another extra PR.

Basically only those seems related:

M airflow-core/src/airflow/api_fastapi/core_api/datamodels/ui/deadline.py
M airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml
M airflow-core/src/airflow/api_fastapi/core_api/routes/ui/deadlines.py
M airflow-core/tests/unit/api_fastapi/core_api/routes/ui/test_deadlines.py

…ervals Hardens the read and (de)serialization paths for deadline alerts so dynamic (``VariableInterval``) and malformed stored data no longer break the UI/API. - UI deadline-alert response: ``DeadlineAlert.interval`` is a JSON column holding the Airflow-serialized interval, not a plain number. Coerce it to seconds for a fixed ``timedelta`` and to ``None`` for a dynamic ``VariableInterval`` (resolved later by the scheduler), instead of letting Pydantic 500 on the dict. The ``interval`` field becomes ``float | None``. - Drop ``interval`` from the sortable columns of the deadline-alerts endpoint: ordering by a JSON column sorts by structure/text, not duration, so the result was arbitrary and misleading. - Deserialization: route by the encoder-stamped ``__class_path`` ahead of the ``reference_type`` name (a custom reference may share a class name with a builtin), and raise a clear error for a reference with no importable ``__class_path`` instead of an opaque ``KeyError``. - ``Deadline.__repr__`` / ``DeadlineAlert.__repr__`` no longer raise: guard the ``dagrun`` relationship (the FK can be set while the relationship is None after a cascade delete) and handle the dict-shaped JSON interval. A ``__repr__`` must never raise. - ``prune_deadlines`` explicitly excludes deadlines already marked ``missed`` so a missed deadline (whose callback is owned by the scheduler/triggerer) and its queued callback are never cascade-deleted. Generated-by: Claude Code (Opus via Claude Code) on behalf of Sean Ghaeli

The sort-key comment restated a constraint already covered by test_order_by_interval_is_rejected; the __class_path comment restated the ValueError message right below it. Generated-by: Claude Code (Opus)

Per review: keep only the interval coercion, order_by removal, and their tests; move repr guards, decoder routing, and prune guard to a follow-up.

seanghaeli · 2026-07-25T00:16:17Z

Hi @pierrejeambrun I've split up the PR so that this one just addresses #69245

pierrejeambrun

Thanks

github-actions · 2026-07-28T13:55:24Z

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

shahar1 · 2026-07-28T18:44:40Z

Please note #70637

seanghaeli · 2026-07-28T19:04:19Z

@shahar1 taking a look

Split out of apache#68919 per review: decoder __class_path routing, clear error for missing __class_path, repr guards for severed dagrun and dict-shaped interval, and prune guard for missed deadlines.

apache#68919 changed _private_ui.yaml without regenerating the TypeScript client and CI stayed green: selective checks decide whether to skip the ts-compile-lint-* hooks from file groups that do not cover the UI codegen input specs, while the hooks themselves trigger on any yaml under the openapi directories. The stale client then masked a type error that broke main for every PR running full static checks. The same gap applied to v2-simple-auth-manager-generated.yaml, so a spec-only change also skipped the simple auth manager UI hook. Track the codegen-input specs in their own file group so spec-only changes run the UI hooks without forcing the full test matrix, and drop the main UI hook trigger for the simple auth manager spec: that pattern was copied in when the hook was created (apache#51725), a month after the spec was renamed v1 to v2 (apache#50705), so it has never matched a file - apache#53636 fixed the sibling hook's copy but missed this one. The main UI codegen does not consume that spec either.

…alformed intervals (#68919) (#70625) * Make deadline reads and serialization robust to dynamic/malformed intervals Hardens the read and (de)serialization paths for deadline alerts so dynamic (``VariableInterval``) and malformed stored data no longer break the UI/API. - UI deadline-alert response: ``DeadlineAlert.interval`` is a JSON column holding the Airflow-serialized interval, not a plain number. Coerce it to seconds for a fixed ``timedelta`` and to ``None`` for a dynamic ``VariableInterval`` (resolved later by the scheduler), instead of letting Pydantic 500 on the dict. The ``interval`` field becomes ``float | None``. - Drop ``interval`` from the sortable columns of the deadline-alerts endpoint: ordering by a JSON column sorts by structure/text, not duration, so the result was arbitrary and misleading. - Deserialization: route by the encoder-stamped ``__class_path`` ahead of the ``reference_type`` name (a custom reference may share a class name with a builtin), and raise a clear error for a reference with no importable ``__class_path`` instead of an opaque ``KeyError``. - ``Deadline.__repr__`` / ``DeadlineAlert.__repr__`` no longer raise: guard the ``dagrun`` relationship (the FK can be set while the relationship is None after a cascade delete) and handle the dict-shaped JSON interval. A ``__repr__`` must never raise. - ``prune_deadlines`` explicitly excludes deadlines already marked ``missed`` so a missed deadline (whose callback is owned by the scheduler/triggerer) and its queued callback are never cascade-deleted. Generated-by: Claude Code (Opus via Claude Code) on behalf of Sean Ghaeli * Remove redundant comments in deadline sort/deserialize The sort-key comment restated a constraint already covered by test_order_by_interval_is_rejected; the __class_path comment restated the ValueError message right below it. Generated-by: Claude Code (Opus) * Trim PR to the UI deadlineAlerts 500 fix only Per review: keep only the interval coercion, order_by removal, and their tests; move repr guards, decoder routing, and prune guard to a follow-up. --------- (cherry picked from commit f840a82) Co-authored-by: Sean Ghaeli <58916776+seanghaeli@users.noreply.github.com> Co-authored-by: Sean Ghaeli <ghaeli@amazon.com>

#68919 changed _private_ui.yaml without regenerating the TypeScript client and CI stayed green: selective checks decide whether to skip the ts-compile-lint-* hooks from file groups that do not cover the UI codegen input specs, while the hooks themselves trigger on any yaml under the openapi directories. The stale client then masked a type error that broke main for every PR running full static checks. The same gap applied to v2-simple-auth-manager-generated.yaml, so a spec-only change also skipped the simple auth manager UI hook. Track the codegen-input specs in their own file group so spec-only changes run the UI hooks without forcing the full test matrix, and drop the main UI hook trigger for the simple auth manager spec: that pattern was copied in when the hook was created (#51725), a month after the spec was renamed v1 to v2 (#50705), so it has never matched a file - #53636 fixed the sibling hook's copy but missed this one. The main UI codegen does not consume that spec either.

…nges (#70643) #68919 changed _private_ui.yaml without regenerating the TypeScript client and CI stayed green: selective checks decide whether to skip the ts-compile-lint-* hooks from file groups that do not cover the UI codegen input specs, while the hooks themselves trigger on any yaml under the openapi directories. The stale client then masked a type error that broke main for every PR running full static checks. The same gap applied to v2-simple-auth-manager-generated.yaml, so a spec-only change also skipped the simple auth manager UI hook. Track the codegen-input specs in their own file group so spec-only changes run the UI hooks without forcing the full test matrix, and drop the main UI hook trigger for the simple auth manager spec: that pattern was copied in when the hook was created (#51725), a month after the spec was renamed v1 to v2 (#50705), so it has never matched a file - #53636 fixed the sibling hook's copy but missed this one. The main UI codegen does not consume that spec either. (cherry picked from commit b43a87e) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

seanghaeli requested review from XD-DENG, ashb, bolkedebruin, bugraoz93, choo121600, ephraimbuddy, jason810496, pierrejeambrun, rawwar and shubhamraj-git as code owners June 23, 2026 21:30

boring-cyborg Bot added area:API Airflow's REST/HTTP API area:DAG-processing area:deadline-alerts AIP-86 (former AIP-57) labels Jun 23, 2026

seanghaeli force-pushed the feature/deadline-response-serialization branch from 12a8e11 to 352702c Compare June 23, 2026 21:30

seanghaeli marked this pull request as draft June 23, 2026 21:44

seanghaeli force-pushed the feature/deadline-response-serialization branch from 352702c to 31bd6d5 Compare June 23, 2026 22:11

seanghaeli marked this pull request as ready for review June 23, 2026 23:00

potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jun 25, 2026

pierrejeambrun reviewed Jun 29, 2026

View reviewed changes

Comment thread airflow-core/src/airflow/api_fastapi/core_api/datamodels/ui/deadline.py Outdated

Comment thread airflow-core/newsfragments/68919.bugfix.rst Outdated

pierrejeambrun removed the ready for maintainer review Set after triaging when all criteria pass. label Jun 29, 2026

pierrejeambrun added this to the Airflow 3.3.1 milestone Jun 29, 2026

pierrejeambrun added the type:bug-fix Changelog: Bug Fixes label Jun 29, 2026

This was referenced Jul 2, 2026

Minor UI bug on the Dag details tab: tabs overlap with Dag scheduling information #69247

Closed

PydanticSerializationError on deadline alerts UI endpoint for Dags overview and Dag run #69245

Closed

potiuk assigned seanghaeli Jul 2, 2026

seanghaeli force-pushed the feature/deadline-response-serialization branch from 31bd6d5 to 1901333 Compare July 7, 2026 23:05

seanghaeli requested a review from henry3260 as a code owner July 7, 2026 23:05

seanghaeli force-pushed the feature/deadline-response-serialization branch from 6daac55 to 4a0ae29 Compare July 8, 2026 06:46

potiuk added the ready for maintainer review Set after triaging when all criteria pass. label Jul 8, 2026

pierrejeambrun reviewed Jul 16, 2026

View reviewed changes

Sean Ghaeli added 3 commits July 24, 2026 23:36

Remove redundant comments in deadline sort/deserialize

1db6432

The sort-key comment restated a constraint already covered by test_order_by_interval_is_rejected; the __class_path comment restated the ValueError message right below it. Generated-by: Claude Code (Opus)

Trim PR to the UI deadlineAlerts 500 fix only

ebd49cd

Per review: keep only the interval coercion, order_by removal, and their tests; move repr guards, decoder routing, and prune guard to a follow-up.

seanghaeli force-pushed the feature/deadline-response-serialization branch from 4a0ae29 to ebd49cd Compare July 24, 2026 23:38

seanghaeli mentioned this pull request Jul 24, 2026

Fix deadline serialization, repr, and prune edge cases #70421

Open

pierrejeambrun added backport-to-v3-3-test Backport to v3-3-test and removed ready for maintainer review Set after triaging when all criteria pass. labels Jul 28, 2026

pierrejeambrun approved these changes Jul 28, 2026

View reviewed changes

pierrejeambrun merged commit f840a82 into apache:main Jul 28, 2026
79 checks passed

shahar1 mentioned this pull request Jul 28, 2026

Regenerate UI client and handle deadline alerts with no fixed interval #70637

Merged

1 task

shahar1 mentioned this pull request Jul 28, 2026

Run UI compile checks in CI when only an OpenAPI spec changes #70643

Merged

1 task

Lee-W mentioned this pull request Jul 29, 2026

Add the Taiwanese Mandarin string for variable-interval deadlines #70655

Merged

1 task

github-actions Bot mentioned this pull request Jul 29, 2026

[v3-3-test] Run UI compile checks in CI when only an OpenAPI spec changes (#70643) #70672

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Make deadline reads and serialization robust to dynamic/malformed intervals - #68919

Make deadline reads and serialization robust to dynamic/malformed intervals#68919
pierrejeambrun merged 3 commits into
apache:mainfrom
aws-mwaa:feature/deadline-response-serialization

seanghaeli commented Jun 23, 2026 •

edited

Loading

Uh oh!

pierrejeambrun left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

pierrejeambrun left a comment

Uh oh!

seanghaeli commented Jul 25, 2026

Uh oh!

pierrejeambrun left a comment

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

shahar1 commented Jul 28, 2026

Uh oh!

seanghaeli commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

seanghaeli commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pierrejeambrun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

seanghaeli commented Jul 25, 2026

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

Backport successfully created: v3-3-test

Uh oh!

shahar1 commented Jul 28, 2026

Uh oh!

seanghaeli commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

seanghaeli commented Jun 23, 2026 •

edited

Loading

pierrejeambrun left a comment •

edited

Loading