feat: null aware RightAnti hash join execution

[saadtajwar]

Which issue does this PR close?

• Part of Support null-aware RightAnti hash join (build on the subquery side) for NOT IN #23931 (not closes yet)

Rationale for this change (copied from issue):

DataFusion plans NOT IN (subquery) as a null-aware anti join, but HashJoinExec only supports null_aware = true for LeftAnti with a single join key (validated in datafusion-physical-plan/src/joins/hash_join/exec.rs). Since HashJoinExec always builds on the left input, the build side of a null-aware anti join is the outer table, not the subquery.

This has two costs:

Memory scales with the wrong side. For SELECT ... FROM big_fact WHERE key NOT IN (SELECT k FROM small_dim), the hash table is built over the entire fact table. Memory is O(outer) when it could be O(subquery).

The operator cannot be distributed. The null-aware logic coordinates three pieces of global state across probe partitions through in-process shared memory: probe_side_has_null: AtomicBool, probe_side_non_empty: AtomicBool, and the visited-build-row bitmap, with the last probe partition to finish emitting the unmatched build rows (hash_join/stream.rs). This is correct and cheap in one process, but engines that split probe partitions across processes get independent copies of all three and produce duplicated or incorrect rows. Ballista hit exactly this (apache/datafusion-ballista#2187) and currently has to force the join into a single task (apache/datafusion-ballista#2188), losing all parallelism.

What changes are included in this PR?

This PR is the first of two - when a path IS built as RightAnti + null aware + CollectLeft partition mode, the join executes as expected. The second PR that will be created after this will support the planner changes to emit null-aware RightAnti & the JoinSelection swap logic!

This PR specifically adds the build_side_has_null field and assigns at build-time, and correctly satisfies the invariant of no rows being output if true + build state with null keys not being output + empty build side outputting all probe rows

Are these changes tested?

Yes

Are there any user-facing changes?

No

[codecov-commenter]

Codecov Report

❌ Patch coverage is 94.80519% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.70%. Comparing base (551c592) to head (eb3ba98).
⚠️ Report is 27 commits behind head on main.

Files with missing lines	Patch %	Lines
...fusion/physical-plan/src/joins/hash_join/stream.rs	95.00%	1 Missing and 1 partial ⚠️
...tafusion/physical-plan/src/joins/hash_join/exec.rs	96.66%	0 Missing and 1 partial ⚠️
datafusion/physical-plan/src/joins/mod.rs	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #23957      +/-   ##
==========================================
+ Coverage   80.65%   80.70%   +0.04%     
==========================================
  Files        1093     1095       +2     
  Lines      371629   372600     +971     
  Branches   371629   372600     +971     
==========================================
+ Hits       299725   300690     +965     
+ Misses      54012    53938      -74     
- Partials    17892    17972      +80     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[saadtajwar]

cc @andygrove - looking forward to your feedback! I'll have the second PR with the planner support up shortly :)

[saadtajwar]

Do we need the partition_mode check for CollectLeft here? Or should we just count on it being enforced/never chosen as anything but CollectLeft beforehand?