Fix openlineage team name handling, add dag_version_data and TI note - #70772
Merged
mobuchowski merged 1 commit intoJul 31, 2026
Conversation
1 task
mobuchowski
approved these changes
Jul 30, 2026
Contributor
There was a problem hiding this comment.
I agree that #70452 can be closed in favour of this. I just have 2 comments for you to address. Also, I would remind you to do a quick manual test before merge to verify this works.
kacpermuda
force-pushed
the
openlineage-team-name-session-fix-and-dag-version-data
branch
from
July 30, 2026 14:25
5ffce56 to
0a4bdc7
Compare
Contributor
|
tests are failing |
kacpermuda
force-pushed
the
openlineage-team-name-session-fix-and-dag-version-data
branch
from
July 30, 2026 16:29
0a4bdc7 to
2883bb1
Compare
kacpermuda
force-pushed
the
openlineage-team-name-session-fix-and-dag-version-data
branch
from
July 31, 2026 09:36
2883bb1 to
8d2379c
Compare
Member
|
Why do we need to bundle three things in one PR? All three look unrelated to me. |
Collaborator
Author
|
Yes, the team_name fix is the main part here, and two other attributes are just small additions to the ol events, just added them as I've noticed while doing the main fix that they are not included in OL events now. I can separate this into two PRs is it's problematic, but given the small size of this change I just put them here. Is that okay ? |
Contributor
|
I agree it seems small enough? |
kacpermuda
deleted the
openlineage-team-name-session-fix-and-dag-version-data
branch
July 31, 2026 13:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The previous implementation called
DagBundleModel.get_team_name()without an explicit session. This triggered@provide_sessionto open a newcreate_session(), which — for ORMDagRunobjects in the scheduler loop — resolves to the same scoped session and commits on exit. That commit fires Airflow'sprohibit_commitHA lock guard, producing errors in multi-scheduler setups.The fix switches to
object_session(dagrun)to obtain the session the ORM object is already attached to, then passes it explicitly toDagModel.get_team_name(dag_id, session=session), bypassing@provide_session'screate_session()path entirely. A similar fix was proposed in #70452; this PR simplifies it further by going directly throughDagModelinstead of the bundle.Once #70760 is in, no DB call will be required for scheduler-side events in Airflow 3.3.1: the stamp is already in memory. However, the
_team_nameattribute is private with no stability guarantee; the check is guarded withhasattrand anisinstancecheck; so we might keep the db call here under try/except with fallback to None.Two new fields added as well:
DagVersion.version_data(added in Airflow 3.3) carries arbitrary DAG-level metadata set at parse time. This PR surfaces it asdag_version_datain the OpenLineageDagRunInfofacet. No logic — just reading the attribute and attaching it to the event, guarded byAIRFLOW_V_3_3_PLUS.TaskInstance.noteis set when an operator or a user manually changes task state in the UI. Attaching it to OpenLineage task events makes manual interventions observable in lineage.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Sonnet 4.6) following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.