perf: Optimize hashing, null-free fast path for percentile_cont, median

[neilconway]

Which issue does this PR close?

• Closes Optimize hashing, null-free fast path in percentile_cont, median #23953

Rationale for this change

This PR makes three improvements to non-distinct PercentileContAccumulator and MedianAccumulator, inspired by recent work on percentile_cont(DISTINCT) (#23946):

1. Switch from SipHash to foldhash for internal hash maps
2. Add a null-free fast path to update_batch and retract_batch
3. Raise an error if we attempt to retract an unknown value in retract_batch, rather than silently ignoring it.

Benchmarks:

  percentile_cont no_nulls   window=256    -61.1%  (249.0 -> 96.6 us)
  percentile_cont with_nulls window=256    -59.4%  (232.7 -> 94.5 us)
  percentile_cont no_nulls   window=4096   -50.9%  (756.3 -> 370.4 us)
  percentile_cont with_nulls window=4096   -48.2%  (552.2 -> 286.2 us)
  percentile_cont no_nulls   window=16384  -47.3%  (2.311 -> 1.218 ms)
  percentile_cont with_nulls window=16384  -42.0%  (1.465 -> 0.851 ms)
  median          no_nulls   window=256    -62.8%  (247.0 -> 92.0 us)
  median          with_nulls window=256    -59.8%  (228.8 -> 92.0 us)
  median          no_nulls   window=4096   -40.0%  (411.4 -> 246.6 us)
  median          with_nulls window=4096   -39.4%  (364.4 -> 221.0 us)
  median          no_nulls   window=16384  -31.5%  (1.107 -> 0.759 ms)
  median          with_nulls window=16384  -32.9%  (0.947 -> 0.637 ms)

What changes are included in this PR?

See above.

Are these changes tested?

Yes: existing tests pass, new tests added for the null-free fast path and the improved error handling.

Are there any user-facing changes?

No.

[neilconway]

cc @viirya

[viirya]

LGTM — nice to see the #23946 ideas carried over to the non-distinct accumulators, and the benchmark wins are substantial.

I went through all three changes:

• foldhash on the to_remove map — matches what we did for the distinct path.
• null-free fast path: update_batch uses extend_from_slice(values.values()) when there are no nulls, which is correct since null_count() == 0 guarantees every slot in the values buffer is a valid logical value (equivalent to iter().flatten()), and it's the wholesale append that drives the speedup. retract_batch's null-free branch mirrors it.
• retract error: checking !to_remove.is_empty() after the removal loop is the right adaptation for the non-distinct multiset — any leftover entries are values that were retracted but never present, which is exactly the divergence worth surfacing rather than silently dropping. Consistent with the internal_err! we added on the distinct side.

Test coverage is good: the error-path test and the dense-vs-sparse update_batch equivalence test cover both new branches. LGTM.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 94.28571% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.69%. Comparing base (b0b9dae) to head (b7c8c5b).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
datafusion/functions-aggregate/src/median.rs	94.54%	3 Missing ⚠️
...afusion/functions-aggregate/src/percentile_cont.rs	94.00%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #23954   +/-   ##
=======================================
  Coverage   80.69%   80.69%           
=======================================
  Files        1095     1095           
  Lines      372529   372623   +94     
  Branches   372529   372623   +94     
=======================================
+ Hits       300626   300706   +80     
- Misses      53942    53950    +8     
- Partials    17961    17967    +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.