Skip to content

pivot, asof/window-join: flatten parted input instead of out-of-bounds read (fixes two segfaults)#323

Merged
singaraiona merged 2 commits into
devfrom
fix/parted-input-guards
Jul 9, 2026
Merged

pivot, asof/window-join: flatten parted input instead of out-of-bounds read (fixes two segfaults)#323
singaraiona merged 2 commits into
devfrom
fix/parted-input-guards

Conversation

@singaraiona

Copy link
Copy Markdown
Collaborator

Closes the last two reachable instances of a confirmed crash class: an operator that fetches table columns with ray_table_get_col and indexes them by row up to ray_table_nrows, which for a parted table reads the raw segment-array wrapper out of bounds → SIGSEGV. GROUP/SORT/WINDOW/equi-JOIN were already guarded; a safety sweep of every table-consuming operator found these two still open (and confirmed the rest safe).

What

  • PIVOT (exec_pivot): a parted table handed to (pivot …) indexed the raw parted columns to the true row count → confirmed SIGSEGV exit 139 on a 40M-row parted table. Guard added at the top: detect parted/MAPCOMMON columns, flatten (concat segments), recompute on the flat table — mirrors the WINDOW guard, including surfacing a mid-flatten merge failure.
  • asof-join / window-join (exec_window_join): a bare parted table handed directly to (asof-join …) (not a (select {from:…}) operand) hit the eager kernel unguarded → confirmed SIGSEGV exit 139. Renamed the kernel to exec_window_join_flat and added a flatten-only wrapper reusing the merged join_flatten_parted. The parted-asof streaming path (feat(join): day-windowed streaming asof-join over parted tables #318) feeds this kernel flat per-day tables, so the guard is inert there and only fires for a bare parted operand.

Not included (verified non-bug)

The audit also flagged datalog dl_add_edb, but I dropped it: the datalog OOB sites live in rule/join kernels that are unreachable from the language — there is no rule-adding builtin exposed, and dl-query/dl-eval don't index EDB rows — so a guard there would fix a non-bug and change no observable behavior.

Verification

  • Both crashes reproduced pre-fix (exit 139) and fixed post-fix, with correct results (pivot 500-row aggregate; asof 5M-row join) verified against flat oracles.
  • Suite 3617/3617 (ASan/UBSan). Release -O3 -Werror clean.
  • New tests: pivot_parted_guard.rfl, asof_parted_guard.rfl — differential oracle (parted result == flat-oracle), no-crash. The asof test deliberately uses a bare parted operand to hit the eager guard, not the streaming path.
  • Perf: TAQ asof/window-join queries (q49–q52) A/B base≈head — the guard is an O(ncols) tag scan, inert on the non-parted path. Only exec_pivot/exec_window_join changed, so other queries are provably unaffected.

exec_pivot fetched columns via ray_table_get_col and indexed them by row up
to ray_table_nrows; a raw RAY_PARTED/MAPCOMMON column (data is a segment-count
sized segs array, not nrows elements) was indexed out of bounds -> SIGSEGV
(confirmed exit 139 on a 40M-row parted table).  Guard mirrors exec_window /
join_with_parted_guard: detect parted columns, flatten (concat segments,
surfacing any mid-flatten merge failure), and recompute on the flat table.
…ds read

exec_window_join fetched time/equality-key columns via ray_table_get_col and
indexed to ray_table_nrows with no parted guard -> OOB read / SIGSEGV (confirmed
exit 139 when a bare parted table is handed directly to (asof-join ...)).
Renamed the kernel to exec_window_join_flat and added a flatten-only wrapper
reusing join_flatten_parted.  The parted-asof STREAMING path feeds this kernel
flat per-day tables, so the guard is inert there and only fires for a bare
parted operand.
@singaraiona singaraiona merged commit 1a4043a into dev Jul 9, 2026
9 checks passed
@hetoku hetoku deleted the fix/parted-input-guards branch July 9, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant