pivot, asof/window-join: flatten parted input instead of out-of-bounds read (fixes two segfaults)#323
Merged
Merged
Conversation
exec_pivot fetched columns via ray_table_get_col and indexed them by row up to ray_table_nrows; a raw RAY_PARTED/MAPCOMMON column (data is a segment-count sized segs array, not nrows elements) was indexed out of bounds -> SIGSEGV (confirmed exit 139 on a 40M-row parted table). Guard mirrors exec_window / join_with_parted_guard: detect parted columns, flatten (concat segments, surfacing any mid-flatten merge failure), and recompute on the flat table.
…ds read exec_window_join fetched time/equality-key columns via ray_table_get_col and indexed to ray_table_nrows with no parted guard -> OOB read / SIGSEGV (confirmed exit 139 when a bare parted table is handed directly to (asof-join ...)). Renamed the kernel to exec_window_join_flat and added a flatten-only wrapper reusing join_flatten_parted. The parted-asof STREAMING path feeds this kernel flat per-day tables, so the guard is inert there and only fires for a bare parted operand.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the last two reachable instances of a confirmed crash class: an operator that fetches table columns with
ray_table_get_coland indexes them by row up toray_table_nrows, which for a parted table reads the raw segment-array wrapper out of bounds → SIGSEGV. GROUP/SORT/WINDOW/equi-JOIN were already guarded; a safety sweep of every table-consuming operator found these two still open (and confirmed the rest safe).What
exec_pivot): a parted table handed to(pivot …)indexed the raw parted columns to the true row count → confirmed SIGSEGV exit 139 on a 40M-row parted table. Guard added at the top: detect parted/MAPCOMMON columns, flatten (concat segments), recompute on the flat table — mirrors the WINDOW guard, including surfacing a mid-flatten merge failure.exec_window_join): a bare parted table handed directly to(asof-join …)(not a(select {from:…})operand) hit the eager kernel unguarded → confirmed SIGSEGV exit 139. Renamed the kernel toexec_window_join_flatand added a flatten-only wrapper reusing the mergedjoin_flatten_parted. The parted-asof streaming path (feat(join): day-windowed streaming asof-join over parted tables #318) feeds this kernel flat per-day tables, so the guard is inert there and only fires for a bare parted operand.Not included (verified non-bug)
The audit also flagged datalog
dl_add_edb, but I dropped it: the datalog OOB sites live in rule/join kernels that are unreachable from the language — there is no rule-adding builtin exposed, anddl-query/dl-evaldon't index EDB rows — so a guard there would fix a non-bug and change no observable behavior.Verification
-O3 -Werrorclean.pivot_parted_guard.rfl,asof_parted_guard.rfl— differential oracle (parted result == flat-oracle), no-crash. The asof test deliberately uses a bare parted operand to hit the eager guard, not the streaming path.exec_pivot/exec_window_joinchanged, so other queries are provably unaffected.