diff --git a/docs/docs/guides/memory.md b/docs/docs/guides/memory.md index a2841094..998c1d2d 100644 --- a/docs/docs/guides/memory.md +++ b/docs/docs/guides/memory.md @@ -68,7 +68,11 @@ In this example, loading 10 million rows consumed roughly 800 MB of heap memory. ## 3. The `.sys.gc` Function -Call `(.sys.gc 0)` to signal that the runtime should reclaim unused memory. Currently this is a lightweight hook that returns `0` — Rayforce uses deterministic ref counting and eager page release via `madvise` during buddy coalescing, so most memory is reclaimed automatically when references are dropped. +Call `(.sys.gc)` to run allocator maintenance after references have been +dropped. It drains cross-thread frees, flushes slab caches, coalesces free +blocks, reclaims empty oversized pools, and incrementally releases aged free +pages. Values themselves use deterministic reference counting; this is not a +tracing collector. ```lisp (.sys.gc) ; 0 @@ -76,7 +80,9 @@ Call `(.sys.gc 0)` to signal that the runtime should reclaim unused memory. Curr !!! note "Note" - Because Rayforce uses deterministic ref counting (not tracing GC), memory is freed immediately when the last reference is released. The buddy allocator coalesces blocks and releases pages back to the OS automatically. `(.sys.gc 0)` exists as a hook for future use. + Because Rayforce uses deterministic reference counting, objects become + reclaimable immediately when their last reference is released. `(.sys.gc)` + performs the allocator-side consolidation and page-return pass. ## 4. The `.sys.info` Function diff --git a/docs/docs/language/functions.md b/docs/docs/language/functions.md index 7602735b..a35f3ef9 100644 --- a/docs/docs/language/functions.md +++ b/docs/docs/language/functions.md @@ -165,7 +165,7 @@ Operations on vectors as collections. | `lead` | unary | Shift values one row forward; last row is null/sentinel | `(lead [10 20 30])` → `[20 30 0Nl]` | | `deltas` | unary | Adjacent differences; first row is null | `(deltas [10 15 13])` → `[0Nl 5 -2]` | | `ratios` | unary | Adjacent ratios as f64; first row is null | `(ratios [2 4 8])` → `[0Nf 2.0 2.0]` | -| `fills` | unary | Forward-fill nullable vectors | `(fills (as 'I64 (list 0N 2 0N)))` → `[0Nl 2 2]` | +| `fills` | unary | Forward-fill nullable vectors, or every column of a table | `(fills (as 'I64 (list 0N 2 0N)))` → `[0Nl 2 2]` | | `sums` | unary | Running sum; nulls are skipped | `(sums [1 2 3])` → `[1 3 6]` | | `avgs` | unary | Running average over non-null values | `(avgs [2 4 6])` → `[2.0 3.0 4.0]` | | `mins` | unary | Running minimum | `(mins [3 1 2])` → `[3 1 1]` | diff --git a/docs/docs/language/math.md b/docs/docs/language/math.md index 0828204b..8136f7ed 100644 --- a/docs/docs/language/math.md +++ b/docs/docs/language/math.md @@ -578,6 +578,8 @@ For common time-series scans, prefer the built-in DAG forms. They avoid per-row (mins [3 1 2]) ; [3 1 1] (maxs [3 1 2]) ; [3 3 3] (fills (as 'I64 (list 0N 2 0N))) ; [0Nl 2 2] +(fills (table [t value] + (list [1 2 3] (as 'I64 (list 10 0N 30))))) ; fills each column (differ [1 1 2 2]) ; [true false true false] (msum 3 [1 2 3 4]) ; [1 3 6 9] (mavg 3 [1 2 3 4]) ; [1.0 1.5 2.0 3.0] diff --git a/docs/docs/namespaces/index.md b/docs/docs/namespaces/index.md index 5a6537e2..4312a61d 100644 --- a/docs/docs/namespaces/index.md +++ b/docs/docs/namespaces/index.md @@ -15,6 +15,7 @@ Rayfall's builtins are organised under dotted namespaces. Names beginning with ` | [`.idx.*`](idx.md) | Accelerator indexes: bloom, hash, sort, zone. | | [`.ipc.*`](ipc.md) | TCP client IPC and the server connection-hook accessor. | | [`.log.*`](log.md) | Write-ahead log: open, write, sync, snapshot, roll, replay, validate. | +| [`.mem.*`](mem.md) | Value sizing and scoped time/allocation measurement. | | [`.os.*`](os.md) | Process environment: `getenv`, `setenv`. | | [`.repl.*`](repl.md) | Interactive REPL control — attach the local REPL to a remote server. | | [`.sys.*`](sys.md) | System info and shell-style commands: build, info, mem, prof, querylog, gc, exec, listen, timeit, env, cmd. | @@ -33,4 +34,4 @@ When the server is started with `-U `, the following dotted builtins a - `.sys.exec`, `.sys.cmd`, `.sys.listen`, `.sys.querylog.enable` - `.time.timer.set`, `.time.timer.del` -`.attr.*`, `.col.*`, `.graph.*`, `.idx.*`, and pure read/inspect builtins (`.db.splayed.get`, `.db.parted.get`, `.db.parted.tables`, `.fs.size`, `.fs.list`, `.log.write`, `.log.sync`, `.log.validate`, `.sys.build`, `.sys.info`, `.sys.mem`, `.sys.prof`, `.sys.querylog`, `.sys.gc`, `.sys.env`, `.sys.args`, `.sys.timeit`, `.ipc.handle`, `.time.now`) are always available. +`.attr.*`, `.col.*`, `.graph.*`, `.idx.*`, `.mem.*`, and pure read/inspect builtins (`.db.splayed.get`, `.db.parted.get`, `.db.parted.tables`, `.fs.size`, `.fs.list`, `.log.write`, `.log.sync`, `.log.validate`, `.sys.build`, `.sys.info`, `.sys.mem`, `.sys.prof`, `.sys.querylog`, `.sys.gc`, `.sys.env`, `.sys.args`, `.sys.timeit`, `.ipc.handle`, `.time.now`) are always available. diff --git a/docs/docs/namespaces/mem.md b/docs/docs/namespaces/mem.md new file mode 100644 index 00000000..f86d49a9 --- /dev/null +++ b/docs/docs/namespaces/mem.md @@ -0,0 +1,71 @@ +# `.mem.*` — memory measurement + +Value-level memory inspection and explicitly scoped query measurement. These +operations are unrestricted and do not change the measured value. + +## Reference + +| Function | Arity | Flags | Description | +|---|---|---|---| +| [`.mem.objsize`](#mem-objsize) | unary | — | Logical bytes retained by a value. | +| [`.mem.ts`](#mem-ts) | 1 | special form | Evaluate once and return result plus time, allocation, peak-memory, result-size, and worker statistics. | + +## `.mem.objsize` { #mem-objsize } + +Signature: `(.mem.objsize value)`. + +Returns the logical byte size of the complete `ray_t` graph reachable from +`value`. Each object header and live payload is counted; a shared child is +counted once. The number deliberately excludes buddy-block slack, global +symbol dictionaries, and opaque native handles, making it stable across +allocator tuning. It is an in-memory size, not a serialized-wire size. + +```lisp +(.mem.objsize 42) ;; 32: one ray_t header +(.mem.objsize [1 2 3]) ;; 56: 32-byte header + three I64 cells +``` + +## `.mem.ts` { #mem-ts } + +Signature: `(.mem.ts expression)`. + +`.mem.ts` is a special form: it evaluates `expression` exactly once inside a +process-wide measurement scope. Secondary-worker allocations and worker busy +time are included. Result sizing and construction of the returned statistics +dictionary happen after the time/allocation scope closes. + +```lisp +(set m (.mem.ts (sum (til 1000000)))) +(at m 'result) +(at m 'peak-live-bytes) +``` + +The returned dictionary has these fields: + +| Key | Meaning | +|---|---| +| `result` | The expression's result. | +| `time-ns` | Monotonic wall-clock duration. | +| `memory-bytes` | Net process-memory delta from the shared per-query profiler/query-log measurement. | +| `allocated-bytes` | Sum of allocator block bytes obtained during the scope, including temporary allocations. | +| `freed-bytes` | Sum of allocator block bytes released before the measured evaluation returned. | +| `net-bytes` | Allocated minus freed bytes at the measurement boundary; it may be negative when the expression releases pre-existing values. | +| `peak-live-bytes` | Maximum positive live-byte delta above the starting boundary. | +| `result-bytes` | Logical `.mem.objsize` of `result`. | +| `alloc-count` | Number of allocator blocks obtained. | +| `free-count` | Number of allocator blocks released. | +| `workers` | Pool workers, including worker 0 (the main thread), that executed at least one parallel task. Zero means no pool dispatch occurred. | +| `worker-busy-ns` | Busy nanoseconds summed across participating workers. | +| `parallelism` | `worker-busy-ns / time-ns`. | + +Only one `.mem.ts` scope may be active in a process, so nested calls return a +`state` error. Allocation instrumentation is normally inactive; ordinary +queries pay only a predicted-not-taken check at allocator entry/exit. While a +scope is active, atomic accounting adds intentional overhead. For performance +benchmarks, time normal executions separately and use `.mem.ts` as an +additional memory-measurement execution. + +The common fields (`time-ns`, `memory-bytes`, `workers`, `worker-busy-ns`, and +`parallelism`) come from the same per-query measurement lifecycle used by the +profiler and query log. The allocation counters and `result-bytes` are the +additional `.mem.ts`-specific detail. diff --git a/docs/docs/namespaces/sys.md b/docs/docs/namespaces/sys.md index 2039d74b..c8b3ce7a 100644 --- a/docs/docs/namespaces/sys.md +++ b/docs/docs/namespaces/sys.md @@ -16,7 +16,7 @@ Process-level introspection (build, memory, host info) and command-style operati | [`.sys.prof`](#sys-prof) | variadic | — | Last profiled query's per-step statistics as a table. | | [`.sys.querylog`](#sys-querylog) | variadic | — | Ambient per-query statistics ring as a table. | | [`.sys.querylog.enable`](#sys-querylog-enable) | variadic | restricted | Toggle query-statistics logging. | -| [`.sys.gc`](#sys-gc) | variadic | — | Garbage-collect hint (no-op; returns 0). | +| [`.sys.gc`](#sys-gc) | variadic | — | Run allocator maintenance and return `0`. | | [`.sys.env`](#sys-env) | variadic | — | Count or list of globally bound names. | | [`.sys.exec`](#sys-exec) | unary | restricted | Run a shell command; return its exit code. | | [`.sys.cmd`](#sys-cmd) | unary | restricted | Dispatch a colon-command string. | @@ -209,7 +209,11 @@ The `-Q` startup flag is the equivalent for enabling logging from launch. ## `.sys.gc` { #sys-gc } -Signature: `(.sys.gc)`. No-op stub — Rayforce uses reference counting, so there's no global collector to kick. Kept as a registered builtin so existing scripts that call it don't error. Returns `0`. +Signature: `(.sys.gc)`. Runs allocator maintenance: drains cross-thread frees, +flushes slab caches, coalesces free buddy blocks, reclaims empty oversized +pools, and incrementally returns aged free pages to the operating system. +Rayforce values remain deterministically reference-counted—this is allocator +GC, not a tracing object collector. Returns `0`. ## `.sys.env` { #sys-env } diff --git a/docs/docs/queries/attributes.md b/docs/docs/queries/attributes.md index 63af3eff..fa6f0290 100644 --- a/docs/docs/queries/attributes.md +++ b/docs/docs/queries/attributes.md @@ -1,15 +1,15 @@ # Column Attributes -Semantic properties stamped onto numeric columns — `sorted`, `unique`, `grouped`, `parted`. Where an [accelerator index](indexes.md) is a *physical* structure (a hash table, a permutation), a column attribute is an *assertion about meaning*: this column is non-descending, these values are distinct. The `.attr.*` family layers over the same storage as `.idx.*`, but its contract is semantic, not structural. +Semantic properties stamped onto columns — `sorted`, `unique`, `grouped`, `parted`. Where an [accelerator index](indexes.md) is a *physical* structure (a hash table, a permutation), a column attribute is an *assertion about meaning*: this column is non-descending, these values are distinct. The `.attr.*` family layers over the same storage as `.idx.*`, but its contract is semantic, not structural. !!! note "v1 status" - The four attributes, their strict verify-on-set kernels, and the `(.attr.*)` Rayfall surface are shipped, along with the as-of-join fast paths that consume them. All attributes are **numeric-only** in v1 — the same numeric type set as `.idx.*`; symbol / string columns are deferred. + The four attributes, their strict verify-on-set kernels, and the `(.attr.*)` Rayfall surface are shipped. `sorted` and `unique` are numeric; the backing-index attributes `grouped` and `parted` also support symbol columns. ## Why Attributes Are Separate from Indexes An [accelerator index](indexes.md) answers *"what structure is attached?"* — a zonemap, a hash table, a sort permutation. A column attribute answers a different question: *"what is semantically true of these values?"* The two markers (`sorted`, `unique`) carry no allocation at all — they are cheap flags. The two backing-index attributes (`grouped`, `parted`) do reuse the index layer underneath (so a `grouped` or `parted` column also shows up via `.idx.has?` and `.idx.info`), but `.attr.*` exists to assert a *property* the engine can trust, not merely to build a structure. -The payoff is that a consumer — most importantly the [as-of join executor](joins.md#pre-sorted-fast-path) — can trust a stamped attribute unconditionally and skip work it would otherwise have to do defensively. +The payoff is that consumers can trust a stamped attribute unconditionally. Besides the [as-of join executor](joins.md#pre-sorted-fast-path), filters and grouped aggregation use the physical layout directly. ## The Four Attributes @@ -22,7 +22,9 @@ The payoff is that a consumer — most importantly the [as-of join executor](joi The two **markers** are independent and may coexist. A column holds at most **one backing index** at a time: setting `grouped` or `parted` replaces any prior backing index. Markers may coexist with a backing index. -`parted` is **verify-only** — it never reorders data. It checks that the column is already block-laid-out (each distinct value occupies one contiguous, ascending run) and errors if it is not. The caller is responsible for ordering the column first. +`parted` is **verify-only** — it never reorders data. It checks that the column is already block-laid-out (each distinct value occupies one contiguous run; numeric keys must also be ascending) and errors if it is not. The caller is responsible for ordering the column first. + +For `where: (in key set)` on a parted key, the filter builds its row selection from the matching `[start, len)` ranges without scanning the key column or expanding full morsels into row ids. If the same query groups by that key and uses supported aggregates, Rayforce skips the filter and group discovery entirely and streams each selected partition directly into the aggregate kernel. The equivalent `grouped` path consumes hash-index row slices. ## Setting, Reading, and Dropping @@ -30,7 +32,7 @@ The two **markers** are independent and may coexist. A column holds at most **o `(.attr.set 'name v) (.attr.get v) (.attr.drop v)` -- `(.attr.set 'name v)` — assert attribute `name` on numeric vector `v`. Strict verify: it scans `v` and **errors on violation**, so a stamped attribute never lies. Returns the column with the attribute recorded. +- `(.attr.set 'name v)` — assert attribute `name` on vector `v`. Strict verification **errors on violation**, so a stamped attribute never lies. Returns the column with the attribute recorded. - `(.attr.get v)` — return the symbol vector of currently-held attributes (the empty symbol vector when none are set). - `(.attr.drop v)` — remove all attributes from `v`. @@ -85,7 +87,7 @@ An unknown attribute name is rejected the same way: (.attr.set 'bogus [1 2 3]) ; ⇒ error (domain): unknown attribute ``` -Setting `sorted` runs an O(n) ascending scan; `unique` and `grouped` build / consult the hash to detect duplicates; `parted` walks the column verifying that each distinct value forms one contiguous ascending block. +Setting `sorted` runs an O(n) ascending scan; `unique` and `grouped` build / consult the hash to detect duplicates; `parted` walks the column verifying that each distinct value forms one contiguous block (and ascending block order for numeric keys). ## Combining Attributes @@ -160,7 +162,7 @@ For a partitioned join `(asof-join [Key Time] L R)`, if the single numeric equal ## Caveats and Limits -- **Numeric types only (v1).** Attributes accept the same numeric type set as `.idx.*` (boolean, integer, float, and temporal types). Symbol / string columns are deferred — including the `parted` equality key in a partitioned as-of join, which must be a numeric ID. +- **Type support.** `sorted` and `unique` accept numeric types. `grouped` and `parted` also accept `SYM`; `STR` remains unsupported. - **Strict verify, no silent stamping.** `.attr.set` errors on violation rather than recording a property it cannot confirm; this is what lets consumers trust the stamp unconditionally. - **Conservative propagation.** Only copy / rebind preserve attributes; every transform drops them. Re-assert with `.attr.set` after a transform. - **One backing index per column.** `grouped` and `parted` share the [accelerator-index](indexes.md) slot, so a column carries at most one of them at a time; the markers are separate and free. diff --git a/docs/docs/queries/indexes.md b/docs/docs/queries/indexes.md index 4a8a35f1..8a0f97bb 100644 --- a/docs/docs/queries/indexes.md +++ b/docs/docs/queries/indexes.md @@ -16,8 +16,11 @@ The executor consults indexes at five sites. Each site has a specific consumpti | `.idx.zone` | `filter` — comparison predicates (EQ/NE/LT/LE/GT/GE) | O(1) all/none short-circuit: if the constant falls outside the column's [min, max] range the filter returns 0 rows without scanning; if every row must pass the whole table is returned. Integer and float families both supported. | | `.idx.bloom` | `filter` — EQ on integer-family columns | Definite-absent proof: if all k probe bits are clear the key is absent and the filter returns 0 rows without scanning. False positives fall through to the scan; there are no false absences. | | `.idx.hash` | `filter` — EQ on integer-family columns | Direct rowsel from hash-chain probe: matching rows are collected in O(matches) without a full column scan. Null-free columns only (HAS_NULLS → scan). | +| `part` backing (`.attr.set 'parted`) | `filter` — EQ predicates | Locate the value block and build the rowsel directly from its contiguous `[start, len)` span, including dense blocks. | | `.idx.sort` | `filter` — range predicates (EQ/LT/LE/GT/GE) | Binary search over the row-id permutation to identify the matching span, then a rowsel is built from that span. NE (two disjoint spans) falls back. A selectivity guard fires when the span exceeds n/128 (~0.78%) — see [Performance Characteristics](#performance-characteristics) for the measured rationale. | | `.idx.hash` | `filter` — IN predicates on integer-family columns | Hash-chain probe for each set element; result is the union rowsel over all matches. | +| `part` backing (`.attr.set 'parted`) | `filter` — IN predicates | Match partition keys once and build the rowsel from their contiguous physical spans. | +| `.idx.hash` / `part` backing | `where key IN …` + `group by key` | Resolve only the requested key slices and aggregate them directly, skipping filter scan and group discovery. Part slices stream as contiguous ranges. | | `.idx.sort` | ORDER BY — single ascending key | The pre-built permutation is reused directly rather than re-sorting. Descending ORDER BY falls back to recompute (reversing the perm would swap tie-group positions relative to stable DESC sort). | | `.idx.sort` | `distinct` | Walk the sort permutation once to collect first-occurrence row ids per distinct value — O(n) with no hash table. SYM/GUID/STR columns fall back. | | `.idx.hash` | `find` | Hash-chain probe returns the minimum row id matching the needle, or a provably-absent signal. Integer-family needles only; float and cross-family needles fall through to the scan. | @@ -71,8 +74,10 @@ Set `RAY_IDX_STATS=1` before running a process. At exit, per-site consult and h idx_route filter_zone consults=3 hits=2 idx_route filter_bloom consults=1 hits=1 idx_route filter_hash consults=5 hits=5 +idx_route filter_part consults=3 hits=3 idx_route filter_range consults=4 hits=3 idx_route in consults=2 hits=2 +idx_route group-slice consults=2 hits=2 idx_route find consults=8 hits=7 idx_route sort consults=6 hits=6 idx_route distinct consults=2 hits=2 diff --git a/docs/docs/queries/pivot.md b/docs/docs/queries/pivot.md index 909e70c6..99ecb155 100644 --- a/docs/docs/queries/pivot.md +++ b/docs/docs/queries/pivot.md @@ -147,6 +147,21 @@ Running computations like cumulative sums can be computed using the DAG-backed t Use `scan` when the accumulator is a custom function. Use `sums`, `avgs`, `mins`, `maxs`, and `prds` for built-in running aggregates, and `msum`, `mavg`, `mmin`, `mmax`, `mcount`, `mvar`, and `mdev` for fixed trailing windows. Constant-window calls are lazy-aware DAG operations and materialize with morsel-based kernels. +When the calculation must restart for each key, use the partitioned `window` +form. A positive `frame:` value is the trailing row count, including the +current row: + +```lisp +(window {from: trades + part: [Sym] + order: [Time] + frame: 3 + funcs: {avg3: (avg Price)}}) +``` + +Use `frame: 'running` for an unbounded cumulative frame and omit `frame:` (or +use `frame: 'whole`) to aggregate over the whole partition. + ### Rank and Order ```lisp diff --git a/docs/docs/reference/all-functions.md b/docs/docs/reference/all-functions.md index 3870e256..20fab102 100644 --- a/docs/docs/reference/all-functions.md +++ b/docs/docs/reference/all-functions.md @@ -68,7 +68,7 @@ Generated from `src/lang/eval.c` in this checkout. The categorized reference bel `update`, `insert`, `upsert`, `left-join`, `inner-join`, `full-join`, `anti-join`, `window-join`, `window-join1`, `asof-join`, `println`, `show`, `format`, `read-csv`, `write-csv`, `.csv.read`, `.csv.splayed`, `.csv.parted`, `.csv.write`, `resolve`, `timeit`, `enlist`, `map-left`, `map-right`, `.db.splayed.set`, `.db.splayed.get`, -`.db.parted.get`, `.db.parted.tables`, `.db.parted.fill`, `alter`, `print`, `.sys.gc`, `.sys.timeit`, +`.db.parted.get`, `.db.parted.tables`, `.db.parted.fill`, `alter`, `print`, `.sys.gc`, `.mem.objsize`, `.mem.ts`, `.sys.timeit`, `.sys.env`, `.sys.args`, `.ipc.open`, `.ipc.handle`, `.repl.disconnect`, `.log.open`, `.log.roll`, `.log.snapshot`, `.log.sync`, `.log.close`, `.log.purge`, `quote`, `return`, `.time.now`, `.time.timer.set`, `fold-left`, `fold-right`, `scan-left`, `scan-right`, `del`, `.sys.build`, `.sys.mem`, `.sys.prof`, @@ -266,7 +266,7 @@ Operations on vectors and lists as collections — set operations, indexing, sea | `lead` | unary | lazy/DAG | Shift values one row forward; last row is null/sentinel | `(lead [10 20 30])` → `[20 30 0Nl]` | | `deltas` | unary | lazy/DAG | Adjacent differences; first row is null | `(deltas [10 15 13])` → `[0Nl 5 -2]` | | `ratios` | unary | lazy/DAG | Adjacent ratios as f64; first row is null | `(ratios [2 4 8])` → `[0Nf 2.0 2.0]` | -| `fills` | unary | lazy/DAG | Forward-fill nullable vectors | `(fills (as 'I64 (list 0N 2 0N)))` → `[0Nl 2 2]` | +| `fills` | unary | lazy/DAG | Forward-fill nullable vectors, or every column of a table | `(fills (as 'I64 (list 0N 2 0N)))` → `[0Nl 2 2]` | | `sums` | unary | lazy/DAG | Running sum; nulls are skipped | `(sums [1 2 3])` → `[1 3 6]` | | `avgs` | unary | lazy/DAG | Running average over non-null values | `(avgs [2 4 6])` → `[2.0 3.0 4.0]` | | `mins` | unary | lazy/DAG | Running minimum | `(mins [3 1 2])` → `[3 1 1]` | @@ -442,6 +442,7 @@ Special forms that bridge to the Rayforce DAG executor for high-performance colu | Function | Type | Flags | Description | Example | |---|---|---|---|---| | `select` | variadic | special | Query table with optional filter, projection, grouping, and aggregation | `(select {from: t a: a})` | +| `window` | variadic | special | Partitioned window query. `frame:` is `'whole`, `'running`, or a positive trailing row count | `(window {from: t part: [sym] order: [time] frame: 5 funcs: {avg5: (avg price)}})` | | `update` | variadic | special, restricted | Add or modify columns in a table (mutates in-place) | `(update {from: t b: (* a 2)})` | | `insert` | variadic | special, restricted | Insert rows into a table | `(insert t {x: 10 y: 20})` | | `upsert` | variadic | special, restricted | Insert or update rows by key match (target, key, row) | `(upsert t 'x {x: 10 y: 20})` | @@ -634,6 +635,8 @@ System interaction, metaprogramming, diagnostics, and runtime inspection. | `eval` | unary | — | Evaluate a parsed Rayfall expression | `(eval (parse "(+ 1 2)"))` → `3` | | `parse` | unary | — | Parse a string into a Rayfall expression tree | `(parse "(+ 1 2)")` | | `.sys.gc` | variadic | — | Trigger GC / heap flush, returns `0` | `(.sys.gc)` | +| `.mem.objsize` | unary | — | Logical bytes retained by an object graph; shared children count once | `(.mem.objsize value)` | +| `.mem.ts` | special form | — | Evaluate once and return result, time, allocation, peak-memory, result-size, and worker statistics as a dict | `(.mem.ts (select {from: trades}))` | | `.sys.exec` | unary | restricted | Execute a shell command, return exit code | `(.sys.exec "ls -la")` | | `.os.getenv` | unary | restricted | Get environment variable value | `(.os.getenv "HOME")` | | `.os.setenv` | binary | restricted | Set environment variable | `(.os.setenv "KEY" "value")` | diff --git a/mkdocs.yml b/mkdocs.yml index 66dadc41..5dc11ad2 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -133,6 +133,7 @@ nav: - .idx — Indexes: docs/namespaces/idx.md - .ipc — IPC client: docs/namespaces/ipc.md - .log — Write-ahead log: docs/namespaces/log.md + - .mem — Measurement: docs/namespaces/mem.md - .os — Process env: docs/namespaces/os.md - .repl — REPL control: docs/namespaces/repl.md - .sys — System info: docs/namespaces/sys.md diff --git a/src/core/ipc.c b/src/core/ipc.c index 7b6aa0b2..a2fce94f 100644 --- a/src/core/ipc.c +++ b/src/core/ipc.c @@ -488,7 +488,7 @@ static ray_t* eval_payload_core(uint8_t* payload, size_t payload_len, * is formatted back to source. Only when logging is on. */ char qsrc[RAY_QLOG_QUERY_MAX]; size_t qsrc_len = 0; - if (qc.t0_ns && msg && !RAY_IS_ERR(msg)) { + if (qc.measure.active && msg && !RAY_IS_ERR(msg)) { if (msg->type == -RAY_STR) { qsrc_len = ray_str_len(msg); if (qsrc_len > sizeof(qsrc)) qsrc_len = sizeof(qsrc); diff --git a/src/core/qlog.c b/src/core/qlog.c index c98b3337..d6a17a5c 100644 --- a/src/core/qlog.c +++ b/src/core/qlog.c @@ -10,24 +10,20 @@ #include #include "core/qlog.h" -#include "core/qstats.h" /* ray_qstats_agg */ /* Single global instance. Zero-initialised: disabled, empty ring. */ ray_qlog_t g_qlog; -void ray_qlog_end(const ray_qlog_ctx_t* c, const char* src, size_t src_len, +void ray_qlog_end(ray_qlog_ctx_t* c, const char* src, size_t src_len, ray_t* result) { - if (c->t0_ns == 0) return; /* logging was off when the query began */ - - int64_t dur_ns = ray_profile_now_ns() - c->t0_ns; - if (dur_ns < 0) dur_ns = 0; - int64_t cur = 0; - ray_sys_get_stat(&cur, NULL); + if (!c->measure.active) return; /* logging was off when query began */ + ray_query_metrics_t metrics; + ray_query_measure_end(&c->measure, &metrics); ray_qlog_row_t* row = &g_qlog.slot[g_qlog.head & (RAY_QLOG_CAP - 1)]; row->time_ns = ray_timestamp_now_ns(); - row->duration_ms = (double)dur_ns / 1e6; - row->memory_kib = (double)(cur - c->mem0) / 1024.0; + row->duration_ms = (double)metrics.time_ns / 1e6; + row->memory_kib = (double)metrics.memory_bytes / 1024.0; /* Result summary + status. ray_len on a scalar atom returns the value, * not a count (the len field aliases i64), so guard atoms as one row. */ @@ -44,12 +40,8 @@ void ray_qlog_end(const ray_qlog_ctx_t* c, const char* src, size_t src_len, : (int64_t)ray_len(result); } - /* Parallelism from the query-scoped qstats aggregate (ray_eval resets the - * slots at each top-level query when capture is armed). */ - uint64_t busy = 0, mx = 0; uint32_t used = 0; - ray_qstats_agg(&used, &busy, &mx); - row->workers = (int32_t)used; - row->parallelism = dur_ns > 0 ? (double)busy / (double)dur_ns : 0.0; + row->workers = (int32_t)metrics.workers; + row->parallelism = metrics.parallelism; /* Bounded copies — no allocation, ring memory stays fixed. */ size_t qn = src ? src_len : 0; diff --git a/src/core/qlog.h b/src/core/qlog.h index d942602c..4b57c1bb 100644 --- a/src/core/qlog.h +++ b/src/core/qlog.h @@ -26,8 +26,7 @@ #include #include -#include "core/profile.h" /* ray_profile_now_ns */ -#include "mem/sys.h" /* ray_sys_get_stat */ +#include "core/qmeasure.h" /* Ring capacity (power of two so the write index masks cleanly) and the inline * caps for the two variable-length fields — both bounded so the ring is a @@ -69,23 +68,18 @@ static inline void ray_qlog_set_enabled(bool on) { atomic_store_explicit(&g_qlog.enabled, on ? 1u : 0u, memory_order_relaxed); } -/* Baseline captured before an eval; fed back to ray_qlog_end for the deltas. - * t0_ns == 0 marks "logging was off at begin", so end is a no-op even if the - * flag flips mid-query. */ -typedef struct { int64_t t0_ns; int64_t mem0; } ray_qlog_ctx_t; +/* Query logging wraps the same measurement scope used by `.mem.ts`. */ +typedef struct { ray_query_measure_t measure; } ray_qlog_ctx_t; static inline void ray_qlog_begin(ray_qlog_ctx_t* c) { - if (!ray_qlog_enabled()) { c->t0_ns = 0; c->mem0 = 0; return; } - int64_t cur = 0; - ray_sys_get_stat(&cur, NULL); - c->mem0 = cur; - c->t0_ns = ray_profile_now_ns(); + c->measure = (ray_query_measure_t){0}; + if (ray_qlog_enabled()) ray_query_measure_begin(&c->measure); } /* Close a query and append its summary row. No-op when logging was off at * begin. `src` is the query source (truncated to RAY_QLOG_QUERY_MAX); `result` * is the final, materialized result (an error object sets a non-ok status). */ -void ray_qlog_end(const ray_qlog_ctx_t* c, const char* src, size_t src_len, +void ray_qlog_end(ray_qlog_ctx_t* c, const char* src, size_t src_len, ray_t* result); /* Materialize the ring into a fresh table (oldest row first). Returns an diff --git a/src/core/qmeasure.c b/src/core/qmeasure.c new file mode 100644 index 00000000..1120ce55 --- /dev/null +++ b/src/core/qmeasure.c @@ -0,0 +1,58 @@ +/* + * Copyright (c) 2025-2026 Anton Kundenko + * All rights reserved. + */ + +#include "core/qmeasure.h" + +#include "core/profile.h" +#include "core/qstats.h" +#include "mem/sys.h" + +void ray_query_workers_begin(bool capture, bool progress) { + uint32_t mode = (capture ? RAY_QS_PROF : 0) | + (progress ? RAY_QS_PROGRESS : 0); + ray_qstats_set_mode(mode); + if (mode) ray_qstats_reset(0); +} + +void ray_query_workers_snapshot(uint32_t* workers, uint64_t* busy_ns) { + uint64_t max_busy = 0; + ray_qstats_agg(workers, busy_ns, &max_busy); +} + +void ray_query_measure_begin(ray_query_measure_t* scope) { + if (!scope) return; + scope->prior_qstats_mode = ray_qstats_mode(); + ray_qstats_set_mode(scope->prior_qstats_mode | RAY_QS_PROF); + ray_qstats_reset(0); + ray_sys_get_stat(&scope->memory0, NULL); + scope->t0_ns = ray_profile_now_ns(); + scope->active = true; +} + +void ray_query_measure_end(ray_query_measure_t* scope, ray_query_metrics_t* out) { + if (!scope || !scope->active) { + if (out) *out = (ray_query_metrics_t){0}; + return; + } + + int64_t elapsed = ray_profile_now_ns() - scope->t0_ns; + int64_t memory = 0; + ray_sys_get_stat(&memory, NULL); + uint32_t workers = 0; + uint64_t busy_ns = 0; + ray_query_workers_snapshot(&workers, &busy_ns); + ray_qstats_set_mode(scope->prior_qstats_mode); + scope->active = false; + + if (!out) return; + if (elapsed < 0) elapsed = 0; + *out = (ray_query_metrics_t){ + .time_ns = elapsed, + .memory_bytes = memory - scope->memory0, + .workers = workers, + .worker_busy_ns = busy_ns, + .parallelism = elapsed > 0 ? (double)busy_ns / (double)elapsed : 0.0, + }; +} diff --git a/src/core/qmeasure.h b/src/core/qmeasure.h new file mode 100644 index 00000000..92afe0a6 --- /dev/null +++ b/src/core/qmeasure.h @@ -0,0 +1,36 @@ +/* + * Copyright (c) 2025-2026 Anton Kundenko + * All rights reserved. + * + * Shared per-query measurement lifecycle. Query logging, profiler spans, + * and `.mem.ts` consume this layer so their wall time, process memory, + * worker count, busy time, and parallelism have one definition. + */ + +#ifndef RAY_QMEASURE_H +#define RAY_QMEASURE_H + +#include +#include + +typedef struct { + int64_t time_ns; + int64_t memory_bytes; + uint32_t workers; + uint64_t worker_busy_ns; + double parallelism; +} ray_query_metrics_t; + +typedef struct { + int64_t t0_ns; + int64_t memory0; + uint32_t prior_qstats_mode; + bool active; +} ray_query_measure_t; + +void ray_query_workers_begin(bool capture, bool progress); +void ray_query_workers_snapshot(uint32_t* workers, uint64_t* busy_ns); +void ray_query_measure_begin(ray_query_measure_t* scope); +void ray_query_measure_end(ray_query_measure_t* scope, ray_query_metrics_t* out); + +#endif /* RAY_QMEASURE_H */ diff --git a/src/lang/eval.c b/src/lang/eval.c index ab9c9c1e..53c5d52b 100644 --- a/src/lang/eval.c +++ b/src/lang/eval.c @@ -38,6 +38,7 @@ #include "vec/vec.h" #include "core/profile.h" #include "core/qstats.h" /* per-worker parallelism counters for eval-path spans */ +#include "core/qmeasure.h" /* shared per-query worker lifecycle */ #include "core/qlog.h" /* ray_qlog_enabled — arm capture for query logging */ #include "store/serde.h" /* ray_serde_size — result footprint for spans */ #include "table/sym.h" @@ -3236,6 +3237,8 @@ static void ray_register_builtins(void) { * the category. */ register_vary (".sys.gc", RAY_FN_NONE, ray_gc_fn); register_unary(".sys.exec", RAY_FN_RESTRICTED, ray_system_fn); + register_unary(".mem.objsize", RAY_FN_NONE, ray_mem_objsize_fn); + register_vary (".mem.ts", RAY_FN_SPECIAL_FORM, ray_mem_ts_fn); /* Registry-dispatched system commands. `.sys.cmd "name args"` is * the colon-command entry point; the per-command direct builtins below * skip the string parse for callers that already have a typed arg @@ -3454,8 +3457,8 @@ static inline void eval_span_payload(ray_prof_span_t* s, ray_t* result) { int64_t cur = 0; ray_sys_get_stat(&cur, NULL); s->sys_cur = cur; s->qs_rows = ray_qstats_sum_rows(); - uint64_t sum = 0, mx = 0; uint32_t used = 0; - ray_qstats_agg(&used, &sum, &mx); + uint64_t sum = 0; uint32_t used = 0; + ray_query_workers_snapshot(&used, &sum); s->qs_busy_ns = sum; s->qs_workers = used; if (result && !RAY_IS_ERR(result) && !RAY_IS_NULL(result)) { @@ -3488,9 +3491,7 @@ ray_t* ray_eval(ray_t* obj) { * capture strictly off (mode 0) when the profiler is disabled. */ bool capture = g_ray_profile.active || ray_qlog_enabled(); bool progress = ray_progress_active(); - uint32_t qsmode = (capture ? RAY_QS_PROF : 0) | (progress ? RAY_QS_PROGRESS : 0); - ray_qstats_set_mode(qsmode); - if (qsmode) ray_qstats_reset(0); + ray_query_workers_begin(capture, progress); } /* Check for external interrupt (e.g. Ctrl-C from REPL) */ diff --git a/src/lang/internal.h b/src/lang/internal.h index c840447f..e45a04cf 100644 --- a/src/lang/internal.h +++ b/src/lang/internal.h @@ -592,6 +592,8 @@ ray_t* ray_eval_builtin_fn(ray_t* x); ray_t* ray_parse_builtin_fn(ray_t* x); ray_t* ray_print_fn(ray_t** args, int64_t n); ray_t* ray_meta_fn(ray_t* x); +ray_t* ray_mem_objsize_fn(ray_t* x); +ray_t* ray_mem_ts_fn(ray_t** args, int64_t n); ray_t* ray_gc_fn(ray_t** args, int64_t n); ray_t* ray_system_fn(ray_t* x); /* `.sys.cmd "name args"` — registry-dispatched system commands with diff --git a/src/mem/heap.c b/src/mem/heap.c index 249eed0d..eb888bb1 100644 --- a/src/mem/heap.c +++ b/src/mem/heap.c @@ -242,6 +242,82 @@ static _Atomic(uint64_t) g_heap_id_cursor = 0; static _Atomic(int64_t) g_direct_bytes = 0; static _Atomic(int64_t) g_direct_count = 0; +/* One process-wide, explicitly-scoped allocation trace for `.mem.ts`. + * The inactive hot-path cost is one relaxed load plus a predicted-not-taken + * branch in ray_alloc/ray_free. While active, atomics make allocations from + * the main and every worker heap contribute to the same query measurement. */ +static _Atomic(uint32_t) g_mem_trace_active = 0; +static _Atomic(uint64_t) g_mem_trace_alloc_count = 0; +static _Atomic(uint64_t) g_mem_trace_free_count = 0; +static _Atomic(uint64_t) g_mem_trace_allocated = 0; +static _Atomic(uint64_t) g_mem_trace_freed = 0; +static _Atomic(int64_t) g_mem_trace_live = 0; +static _Atomic(uint64_t) g_mem_trace_peak = 0; + +static inline void mem_trace_note_alloc(size_t bytes) { + if (RAY_LIKELY(atomic_load_explicit(&g_mem_trace_active, + memory_order_relaxed) != 1)) return; + atomic_fetch_add_explicit(&g_mem_trace_alloc_count, 1, memory_order_relaxed); + atomic_fetch_add_explicit(&g_mem_trace_allocated, (uint64_t)bytes, + memory_order_relaxed); + int64_t live = atomic_fetch_add_explicit(&g_mem_trace_live, (int64_t)bytes, + memory_order_relaxed) + (int64_t)bytes; + uint64_t candidate = live > 0 ? (uint64_t)live : 0; + uint64_t peak = atomic_load_explicit(&g_mem_trace_peak, memory_order_relaxed); + while (candidate > peak && + !atomic_compare_exchange_weak_explicit(&g_mem_trace_peak, &peak, + candidate, + memory_order_relaxed, + memory_order_relaxed)) {} +} + +static inline void mem_trace_note_free(size_t bytes) { + if (RAY_LIKELY(atomic_load_explicit(&g_mem_trace_active, + memory_order_relaxed) != 1)) return; + atomic_fetch_add_explicit(&g_mem_trace_free_count, 1, memory_order_relaxed); + atomic_fetch_add_explicit(&g_mem_trace_freed, (uint64_t)bytes, + memory_order_relaxed); + atomic_fetch_sub_explicit(&g_mem_trace_live, (int64_t)bytes, + memory_order_relaxed); +} + +bool ray_mem_trace_begin(void) { + uint32_t expected = 0; + if (!atomic_compare_exchange_strong_explicit(&g_mem_trace_active, &expected, 2, + memory_order_acq_rel, + memory_order_relaxed)) + return false; + atomic_store_explicit(&g_mem_trace_alloc_count, 0, memory_order_relaxed); + atomic_store_explicit(&g_mem_trace_free_count, 0, memory_order_relaxed); + atomic_store_explicit(&g_mem_trace_allocated, 0, memory_order_relaxed); + atomic_store_explicit(&g_mem_trace_freed, 0, memory_order_relaxed); + atomic_store_explicit(&g_mem_trace_live, 0, memory_order_relaxed); + atomic_store_explicit(&g_mem_trace_peak, 0, memory_order_relaxed); + atomic_store_explicit(&g_mem_trace_active, 1, memory_order_release); + return true; +} + +void ray_mem_trace_end(ray_mem_trace_t* out) { + /* Freeze first: state 2 makes allocation hooks ignore the scope while + * preventing another caller from resetting the counters mid-snapshot. */ + atomic_store_explicit(&g_mem_trace_active, 2, memory_order_release); + if (out) { + out->alloc_count = atomic_load_explicit(&g_mem_trace_alloc_count, + memory_order_relaxed); + out->free_count = atomic_load_explicit(&g_mem_trace_free_count, + memory_order_relaxed); + out->allocated_bytes = atomic_load_explicit(&g_mem_trace_allocated, + memory_order_relaxed); + out->freed_bytes = atomic_load_explicit(&g_mem_trace_freed, + memory_order_relaxed); + out->net_bytes = atomic_load_explicit(&g_mem_trace_live, + memory_order_relaxed); + out->peak_live_bytes = atomic_load_explicit(&g_mem_trace_peak, + memory_order_relaxed); + } + atomic_store_explicit(&g_mem_trace_active, 0, memory_order_release); +} + /* Anonymous (RAM-resident) pool + direct bytes we have committed. File-backed * spill mappings are NOT counted (they can be evicted to disk, never OOM-kill). * When a new anon mapping would push this past the watermark, it is routed to a @@ -1187,7 +1263,11 @@ ray_t* ray_alloc(size_t data_size) { * can still represent the request. */ if (RAY_UNLIKELY(order >= RAY_HEAP_POOL_ORDER)) { ray_t* v = heap_alloc_direct(h, data_size); - if (v) return v; + if (v) { + ray_direct_hdr_t* hdr = (ray_direct_hdr_t*)((char*)v - RAY_DIRECT_HDR); + mem_trace_note_alloc(hdr->map_size); + return v; + } if (order > RAY_HEAP_MAX_ORDER) return NULL; } @@ -1214,6 +1294,7 @@ ray_t* ray_alloc(size_t data_size) { RAY_STAT(h->stats.bytes_allocated += BSIZEOF(order)); RAY_STAT(h->stats.peak_bytes = h->stats.bytes_allocated > h->stats.peak_bytes ? h->stats.bytes_allocated : h->stats.peak_bytes); + mem_trace_note_alloc(BSIZEOF(order)); return v; } } @@ -1275,6 +1356,8 @@ ray_t* ray_alloc(size_t data_size) { RAY_STAT(h->stats.peak_bytes = h->stats.bytes_allocated > h->stats.peak_bytes ? h->stats.bytes_allocated : h->stats.peak_bytes); + mem_trace_note_alloc(BSIZEOF(order)); + return blk; } @@ -1344,6 +1427,7 @@ void ray_free(ray_t* v) { size_t map_size = hdr->map_size; int swap_fd = hdr->swap_fd; char* swap_path = hdr->swap_path; + mem_trace_note_free(map_size); if (h) RAY_STAT(h->stats.free_count++); atomic_fetch_sub_explicit(&g_direct_bytes, (int64_t)map_size, memory_order_relaxed); atomic_fetch_sub_explicit(&g_direct_count, 1, memory_order_relaxed); @@ -1368,6 +1452,8 @@ void ray_free(ray_t* v) { if (order < RAY_ORDER_MIN || order > RAY_HEAP_MAX_ORDER) return; + mem_trace_note_free(BSIZEOF(order)); + #if RAY_MEM_STATS size_t block_size = BSIZEOF(order); #endif diff --git a/src/mem/heap.h b/src/mem/heap.h index 557cc71f..9d1e0f04 100644 --- a/src/mem/heap.h +++ b/src/mem/heap.h @@ -144,6 +144,18 @@ typedef struct { size_t sys_mapped_peak; /* file-backed mapping high-water mark */ } ray_mem_stats_t; +/* Query-scoped allocation trace. Unlike ray_mem_stats_t (a lifetime snapshot + * of one heap), this aggregates allocation/free activity from every thread + * while active. Only one scope may be active process-wide. */ +typedef struct { + uint64_t alloc_count; + uint64_t free_count; + uint64_t allocated_bytes; + uint64_t freed_bytes; + int64_t net_bytes; + uint64_t peak_live_bytes; +} ray_mem_trace_t; + /* ===== Forward Declarations (internal types) ===== */ typedef struct ray_heap ray_heap_t; @@ -162,6 +174,8 @@ void ray_heap_push_pending(ray_heap_t* heap); void ray_heap_drain_pending(void); uint8_t ray_order_for_size(size_t data_size); void ray_mem_stats(ray_mem_stats_t* out); +bool ray_mem_trace_begin(void); +void ray_mem_trace_end(ray_mem_trace_t* out); void ray_heap_gc(void); void ray_heap_release_pages(void); diff --git a/src/ops/collection.c b/src/ops/collection.c index 4122660a..056f7dec 100644 --- a/src/ops/collection.c +++ b/src/ops/collection.c @@ -3851,7 +3851,26 @@ ray_t* ray_ratios_fn(ray_t* x) { ray_t* ray_fills_fn(ray_t* x) { if (ray_is_lazy(x)) return ray_lazy_append(x, OP_FILLS); if (ray_is_vec(x)) return ts_start_lazy_unary(x, ray_fills_op); - return ray_error("type", "fills expects a vector input, got %s", ray_type_name(x ? x->type : 0)); + if (x && x->type == RAY_TABLE) { + int64_t ncols = ray_table_ncols(x); + ray_t* out = ray_table_new(ncols); + if (!out || RAY_IS_ERR(out)) return out ? out : ray_error("oom", NULL); + + for (int64_t c = 0; c < ncols; c++) { + ray_t* col = ray_table_get_col_idx(x, c); + ray_t* filled = fills_vec_eager(col); + if (!filled || RAY_IS_ERR(filled)) { + ray_release(out); + return filled ? filled : ray_error("oom", NULL); + } + + out = ray_table_add_col(out, ray_table_col_name(x, c), filled); + ray_release(filled); + if (!out || RAY_IS_ERR(out)) return out ? out : ray_error("oom", NULL); + } + return out; + } + return ray_error("type", "fills expects a vector or table input, got %s", ray_type_name(x ? x->type : 0)); } ray_t* ray_sums_fn(ray_t* x) { diff --git a/src/ops/exec.c b/src/ops/exec.c index 53a6631d..15274d4e 100644 --- a/src/ops/exec.c +++ b/src/ops/exec.c @@ -30,6 +30,7 @@ #include "mem/heap.h" #include "mem/sys.h" #include "core/qstats.h" /* per-worker parallelism stats for profile spans */ +#include "core/qmeasure.h" /* shared worker aggregate */ #include "core/runtime.h" /* __VM — filter-compaction projection keep-set */ /* Global profiler instance (zero-initialized = inactive) */ @@ -1191,8 +1192,8 @@ static int idx_filter_decode(ray_graph_t* g, ray_op_t* pred_op, * WHERE filter, after ray_optimize. When the compiled filter's predicate * is exactly membership on the single bare group-key column — (in c SET) * or (== c K), no other conjuncts, no chained filter — and that column - * carries a fresh CSR hash index, the surviving groups are exactly the - * key set and each group's rows are its full CSR slice: resolve the + * carries a fresh hash or part index, the surviving groups are exactly the + * key set and each group's rows are its full index slice: resolve the * slices NOW (freshness verified inside the resolver) and arm the hint * on g. exec_group consumes it — aggregating the slices directly, or * folding them into the selection the skipped filter would have @@ -1206,7 +1207,8 @@ bool ray_slice_group_probe(ray_graph_t* g, ray_op_t* root, ray_t* by_col) { if (!in0 || in0->opcode == OP_FILTER) return false; if (by_col->type != RAY_SYM || ray_is_atom(by_col)) return false; if (by_col->attrs & RAY_ATTR_HAS_NULLS) return false; - if (ray_index_kind(by_col) != RAY_IDX_HASH) return false; + ray_idx_kind_t kind = ray_index_kind(by_col); + if (kind != RAY_IDX_HASH && kind != RAY_IDX_PART) return false; ray_op_t* pred = op_child(g, root, 1); if (!pred) return false; @@ -1235,7 +1237,7 @@ bool ray_slice_group_probe(ray_graph_t* g, ray_op_t* root, ray_t* by_col) { ray_idx_consults[IDX_SITE_GROUP_SLICE]++; ray_idx_slice_t* sl = NULL; ray_t* shdr = NULL; - int64_t k = ray_index_hash_sym_slices(by_col, keys, &sl, &shdr); + int64_t k = ray_index_sym_slices(by_col, keys, &sl, &shdr); if (eq_atom) ray_release(eq_atom); if (k < 0) return false; ray_idx_hits[IDX_SITE_GROUP_SLICE]++; @@ -1769,8 +1771,8 @@ ray_t* exec_node(ray_graph_t* g, ray_op_t* op) { int64_t cur = 0; ray_sys_get_stat(&cur, NULL); sp->sys_cur = cur; sp->qs_rows = ray_qstats_sum_rows(); - uint64_t sum = 0, mx = 0; uint32_t used = 0; - ray_qstats_agg(&used, &sum, &mx); + uint64_t sum = 0; uint32_t used = 0; + ray_query_workers_snapshot(&used, &sum); sp->qs_busy_ns = sum; } } @@ -1785,8 +1787,8 @@ ray_t* exec_node(ray_graph_t* g, ray_op_t* op) { int64_t cur = 0; ray_sys_get_stat(&cur, NULL); ep->sys_cur = cur; ep->qs_rows = ray_qstats_sum_rows(); - uint64_t sum = 0, mx = 0; uint32_t used = 0; - ray_qstats_agg(&used, &sum, &mx); + uint64_t sum = 0; uint32_t used = 0; + ray_query_workers_snapshot(&used, &sum); ep->qs_busy_ns = sum; ep->qs_workers = used; /* Result footprint — rows this operator produced. */ @@ -2071,6 +2073,41 @@ static ray_t* exec_node_inner(ray_graph_t* g, ray_op_t* op) { ray_t* input = exec_node(g, op_child(g, op, 0)); if (!input || RAY_IS_ERR(input)) return input; + /* A verified physically-sorted column remains routable even when + * this is the outer half of a split AND predicate. Intersect the + * new contiguous range rowsel with the existing selection instead + * of evaluating the second bound over surviving rows. */ + if (input->type == RAY_TABLE) { + ray_t* scol = NULL; + uint16_t sop = 0; + int64_t ski = 0; + double skf = 0.0; + int sisf = 0; + if (idx_filter_decode(g, op_child(g, op, 1), &scol, &sop, + &ski, &skf, &sisf) && + sop != OP_NE && ray_attr_is_sorted(scol)) { + ray_idx_consults[IDX_SITE_FILTER_RANGE]++; + ray_t* ssel = ray_sorted_range_rowsel(scol, sop, ski, skf, + (bool)sisf); + if (ssel) { + if (g->selection) { + ray_t* merged = ray_rowsel_intersect(g->selection, + ssel); + ray_rowsel_release(ssel); + if (merged) { + ray_rowsel_release(g->selection); + g->selection = merged; + ray_idx_hits[IDX_SITE_FILTER_RANGE]++; + return input; + } + } else { + g->selection = ssel; + ray_idx_hits[IDX_SITE_FILTER_RANGE]++; + return input; + } + } + } + } /* Hash-index point-lookup fast path: when the predicate is * `col == K` on a column with RAY_IDX_HASH attached and * built for the column's current length, install the @@ -2123,7 +2160,9 @@ static ray_t* exec_node_inner(ray_graph_t* g, ray_op_t* op) { } } - /* 3. Hash-eq: integer or SYM EQ; re-check HAS_NULLS. */ + /* 3. Hash/part EQ: integer or SYM; re-check HAS_NULLS. + * A part hit is already one contiguous physical span, so + * it stays eligible at any density. */ if (cmp_op == OP_EQ && !is_float && !(col->attrs & RAY_ATTR_HAS_NULLS)) { /* SYM equality: idx_filter_decode passes the global intern id; @@ -2152,21 +2191,33 @@ static ray_t* exec_node_inner(ray_graph_t* g, ray_op_t* op) { } } if (!sym_probe_skip) { - if (ray_index_kind(col) == RAY_IDX_HASH) - ray_idx_consults[IDX_SITE_FILTER_HASH]++; - ray_t* sel = ray_index_hash_eq_rowsel(col, key_i); - if (sel) { - ray_idx_hits[IDX_SITE_FILTER_HASH]++; - g->selection = sel; - return input; + ray_idx_kind_t eq_kind = ray_index_kind(col); + if (eq_kind == RAY_IDX_HASH || + eq_kind == RAY_IDX_PART) { + idx_site_t site = eq_kind == RAY_IDX_HASH + ? IDX_SITE_FILTER_HASH + : IDX_SITE_FILTER_PART; + ray_idx_consults[site]++; + ray_t* sel = eq_kind == RAY_IDX_HASH + ? ray_index_hash_eq_rowsel(col, key_i) + : ray_index_part_eq_rowsel(col, key_i); + if (sel) { + ray_idx_hits[site]++; + g->selection = sel; + return input; + } } /* sel == NULL: ineligible (wrong/stale index kind, * key out of range) or OOM — fall through to scan. */ } } - /* 4. Sort-index range: EQ/LT/LE/GT/GE (NE returns NULL). */ + /* 4. Sort-index range: EQ/LT/LE/GT/GE (NE returns NULL). + * The physical-sorted route runs before this block so it + * can also intersect an existing chained selection. */ if (cmp_op != OP_NE) { + /* Shuffled physical rows need the + * permutation probe and its sparse-span guard. */ if (ray_index_kind(col) == RAY_IDX_SORT) { ray_idx_consults[IDX_SITE_FILTER_RANGE]++; ray_t* sel = ray_index_range_rowsel(col, cmp_op, @@ -2289,17 +2340,18 @@ static ray_t* exec_node_inner(ray_graph_t* g, ray_op_t* op) { } } - /* 5. Hash-index IN probe: FILTER(IN(SCAN col, CONST set_vec)) - * on an integer-family column with a hash index. Probes the - * hash chain for each unique set element; builds one rowsel - * over the union of matches. */ + /* 5. Hash/part-index IN probe: + * FILTER(IN(SCAN col, CONST set_vec)). Hash indexes probe + * each unique set element; part indexes select their + * contiguous physical spans directly. */ { ray_t* in_col = NULL; ray_t* in_set_lit = NULL; if (idx_filter_in_decode(g, op_child(g, op, 1), &in_col, &in_set_lit)) { bool in_col_is_float = (in_col->type == RAY_F32 || in_col->type == RAY_F64); - if (ray_index_kind(in_col) == RAY_IDX_HASH && + ray_idx_kind_t in_kind = ray_index_kind(in_col); + if ((in_kind == RAY_IDX_HASH || in_kind == RAY_IDX_PART) && !in_col_is_float) { ray_idx_consults[IDX_SITE_IN]++; ray_t* sel = ray_index_in_rowsel(in_col, in_set_lit); diff --git a/src/ops/group.c b/src/ops/group.c index 42a83a59..5c8bb310 100644 --- a/src/ops/group.c +++ b/src/ops/group.c @@ -7370,9 +7370,10 @@ ray_t* exec_group(ray_graph_t* g, ray_op_t* op, ray_t* tbl, int64_t group_limit) * ============================================================================ * * The probe (exec.c ray_slice_group_probe) resolved the WHERE key set - * against the group-key column's CSR hash index: g->sg_slices_hdr holds - * one (domain-id, rows, n) slice per surviving key, ascending by domain - * id — the DA path's emit order. Aggregate each slice directly: the + * against the group-key column's hash or part index: g->sg_slices_hdr holds + * one slice per surviving key, ascending by domain id — the DA path's emit + * order. Hash slices borrow row-id arrays; part slices carry a contiguous + * physical range. Aggregate each slice directly: the * filter scan, the survivor gather and group discovery all vanish; per * group the accumulation applies the same all_sum recurrence da_accum_row * applies row-by-row (same read helpers, same in-order accumulate), so @@ -7525,12 +7526,12 @@ static void sg_accum_fn(void* raw, uint32_t wid, int64_t tstart, int64_t tend) { for (int64_t ti = tstart; ti < tend; ti++) { const sg_task_t* tk = &c->tasks[ti]; const ray_idx_slice_t* sl = &c->slices[tk->gi]; - const int64_t* restrict rows = sl->rows + tk->lo; + const int64_t* restrict rows = sl->rows ? sl->rows + tk->lo : NULL; int64_t n = tk->hi - tk->lo; /* Parted layout: each key's rows form one contiguous run — the * accumulate then streams raw column pointers and vectorizes. */ - bool contig = (n > 0 && rows[n - 1] - rows[0] + 1 == n); - int64_t r0 = (n > 0) ? rows[0] : 0; + bool contig = (n > 0 && (!rows || rows[n - 1] - rows[0] + 1 == n)); + int64_t r0 = n > 0 ? (rows ? rows[0] : sl->first + tk->lo) : 0; for (uint32_t a = 0; a < c->n_aggs; a++) { size_t idx = (size_t)ti * c->n_aggs + a; uint16_t op = c->agg_ops[a]; @@ -7667,7 +7668,13 @@ static ray_t* sg_hint_to_selection(ray_graph_t* g, ray_t* tbl) { int64_t* ids = (int64_t*)ray_data(hdr); int64_t w = 0; for (int64_t i = 0; i < K; i++) { - memcpy(ids + w, sl[i].rows, (size_t)sl[i].n * sizeof(int64_t)); + if (sl[i].rows) { + memcpy(ids + w, sl[i].rows, + (size_t)sl[i].n * sizeof(int64_t)); + } else { + for (int64_t j = 0; j < sl[i].n; j++) + ids[w + j] = sl[i].first + j; + } w += sl[i].n; } /* Slices are disjoint and each ascending; a plain sort restores diff --git a/src/ops/idxop.c b/src/ops/idxop.c index ef60aa77..13076964 100644 --- a/src/ops/idxop.c +++ b/src/ops/idxop.c @@ -42,9 +42,9 @@ uint64_t ray_idx_hits[IDX_SITE__N]; static void idx_stats_dump(void) { static const char* names[IDX_SITE__N] = { - "filter-zone", "filter-bloom", "filter-hash", "filter-range", - "in", "find", "sort", "distinct", "asof", "filter-eq-range", - "group-slice", + "filter-zone", "filter-bloom", "filter-hash", "filter-part", + "filter-range", "in", "find", "sort", "distinct", "asof", + "filter-eq-range", "group-slice", }; for (int i = 0; i < IDX_SITE__N; i++) if (ray_idx_consults[i] || ray_idx_hits[i]) @@ -1339,7 +1339,7 @@ ray_t* ray_index_hash_eq_rowsel(ray_t* col, int64_t key) { } /* -------------------------------------------------------------------------- - * Hash-index IN probe + * Hash/part-index IN probe * * For each unique in-range element of set_vec, walk the hash chain and * collect matching row ids into a single shared buffer, then build a @@ -1366,9 +1366,163 @@ static int cmp_i64_plain(const void* a, const void* b) { return (x > y) - (x < y); } +typedef struct { + int64_t start; + int64_t len; +} idx_span_t; + +static bool sorted_i64_contains(const int64_t* values, int64_t n, + int64_t needle) { + int64_t lo = 0, hi = n; + while (lo < hi) { + int64_t mid = lo + (hi - lo) / 2; + if (values[mid] < needle) lo = mid + 1; + else hi = mid; + } + return lo < n && values[lo] == needle; +} + +/* Build a rowsel from sorted, disjoint physical-row spans. Part indexes + * already describe matches this way, so keep full morsels as ALL and store + * row ids only for partial morsels instead of expanding every matching row. */ +static ray_t* rowsel_from_sorted_spans(int64_t n, const idx_span_t* spans, + int64_t nspans, int64_t total) { + if (n < 0 || nspans < 0 || total < 0 || total > n) return NULL; + if (nspans == 0) + return rowsel_from_sorted_ids(n, NULL, 0); + + uint32_t nsegs = n > 0 + ? (uint32_t)((n + RAY_MORSEL_ELEMS - 1) / RAY_MORSEL_ELEMS) + : 0; + ray_t* pop_hdr = ray_alloc((int64_t)nsegs * (int64_t)sizeof(uint32_t)); + if (!pop_hdr) return NULL; + uint32_t* pop = (uint32_t*)ray_data(pop_hdr); + memset(pop, 0, (size_t)nsegs * sizeof(uint32_t)); + + int64_t counted = 0; + int64_t prev_end = 0; + for (int64_t i = 0; i < nspans; i++) { + int64_t start = spans[i].start; + int64_t len = spans[i].len; + if (start < prev_end || len <= 0 || start < 0 || start > n - len) { + ray_release(pop_hdr); + return NULL; + } + int64_t end = start + len; + counted += len; + prev_end = end; + uint32_t first = (uint32_t)(start / RAY_MORSEL_ELEMS); + uint32_t last = (uint32_t)((end - 1) / RAY_MORSEL_ELEMS); + for (uint32_t s = first; s <= last; s++) { + int64_t ss = (int64_t)s * RAY_MORSEL_ELEMS; + int64_t se = ss + RAY_MORSEL_ELEMS; + if (se > n) se = n; + int64_t a = start > ss ? start : ss; + int64_t b = end < se ? end : se; + pop[s] += (uint32_t)(b - a); + } + } + if (counted != total) { + ray_release(pop_hdr); + return NULL; + } + + int64_t idx_count = 0; + for (uint32_t s = 0; s < nsegs; s++) { + int64_t ss = (int64_t)s * RAY_MORSEL_ELEMS; + int64_t se = ss + RAY_MORSEL_ELEMS; + if (se > n) se = n; + if (pop[s] != 0 && pop[s] != (uint32_t)(se - ss)) + idx_count += pop[s]; + } + + ray_t* block = ray_rowsel_new(n, total, idx_count); + if (!block) { + ray_release(pop_hdr); + return NULL; + } + uint8_t* flags = ray_rowsel_flags(block); + uint32_t* offsets = ray_rowsel_offsets(block); + uint16_t* ids = ray_rowsel_idx(block); + uint32_t cum = 0; + for (uint32_t s = 0; s < nsegs; s++) { + offsets[s] = cum; + int64_t ss = (int64_t)s * RAY_MORSEL_ELEMS; + int64_t se = ss + RAY_MORSEL_ELEMS; + if (se > n) se = n; + if (pop[s] == 0) flags[s] = RAY_SEL_NONE; + else if (pop[s] == (uint32_t)(se - ss)) flags[s] = RAY_SEL_ALL; + else { + flags[s] = RAY_SEL_MIX; + cum += pop[s]; + } + } + offsets[nsegs] = cum; + + memset(pop, 0, (size_t)nsegs * sizeof(uint32_t)); + for (int64_t i = 0; i < nspans; i++) { + int64_t start = spans[i].start; + int64_t end = start + spans[i].len; + uint32_t first = (uint32_t)(start / RAY_MORSEL_ELEMS); + uint32_t last = (uint32_t)((end - 1) / RAY_MORSEL_ELEMS); + for (uint32_t s = first; s <= last; s++) { + if (flags[s] != RAY_SEL_MIX) continue; + int64_t ss = (int64_t)s * RAY_MORSEL_ELEMS; + int64_t se = ss + RAY_MORSEL_ELEMS; + if (se > n) se = n; + int64_t a = start > ss ? start : ss; + int64_t b = end < se ? end : se; + uint32_t out = offsets[s] + pop[s]; + for (int64_t r = a; r < b; r++) + ids[out++] = (uint16_t)(r - ss); + pop[s] += (uint32_t)(b - a); + } + } + ray_release(pop_hdr); + return block; +} + +ray_t* ray_index_part_eq_rowsel(ray_t* col, int64_t key) { + if (!idx_fresh_nonull(col, RAY_IDX_PART)) return NULL; + if (col->type == RAY_F32 || col->type == RAY_F64) return NULL; + if (col->type != RAY_SYM && !hash_key_in_range(col->type, key)) + return rowsel_from_sorted_ids(col->len, NULL, 0); + + ray_index_t* ix = ray_index_payload(col->index); + ray_t* keys = ix->u.part.keys; + ray_t* starts = ix->u.part.starts; + ray_t* lens = ix->u.part.lens; + int64_t nparts = ix->u.part.n_parts; + if (!keys || !starts || !lens || keys->len != nparts || + starts->len != nparts || lens->len != nparts) + return NULL; + + const uint8_t* kb = (const uint8_t*)ray_data(keys); + const int64_t* st = (const int64_t*)ray_data(starts); + const int64_t* ln = (const int64_t*)ray_data(lens); + for (int64_t p = 0; p < nparts; p++) { + int64_t pkey = keys->type == RAY_SYM + ? ray_read_sym(kb, p, RAY_SYM, keys->attrs) + : set_vec_read_i64(kb, keys->type, p); + if (pkey != key) continue; + /* A direct FILTER rowsel still feeds the generic table compactor. + * For a dense partition that O(selected rows × columns) gather is + * slower than the normal predicate scan. Keep dense partitions on + * the scan path; the dedicated GROUP BY slice path remains dense-safe + * because it streams the range directly into its aggregates. */ + if (ln[p] > 64 && ln[p] > (col->len >> 1)) + return NULL; + idx_span_t span = { .start = st[p], .len = ln[p] }; + return rowsel_from_sorted_spans(col->len, &span, 1, ln[p]); + } + return rowsel_from_sorted_ids(col->len, NULL, 0); +} + ray_t* ray_index_in_rowsel(ray_t* col, ray_t* set_vec) { - /* Gate: integer-family or SYM column with fresh hash index, no nulls. */ - if (!idx_fresh_nonull(col, RAY_IDX_HASH)) return NULL; + /* Gate: integer-family or SYM column with a fresh hash/part index. */ + ray_idx_kind_t kind = ray_index_kind(col); + if ((kind != RAY_IDX_HASH && kind != RAY_IDX_PART) || + !idx_fresh_nonull(col, kind)) return NULL; bool col_is_float = (col->type == RAY_F32 || col->type == RAY_F64); if (col_is_float) return NULL; bool col_is_sym = (col->type == RAY_SYM); @@ -1433,6 +1587,45 @@ ray_t* ray_index_in_rowsel(ray_t* col, ray_t* set_vec) { return rowsel_from_sorted_ids(n, NULL, 0); } + if (kind == RAY_IDX_PART) { + ray_index_t* ix = ray_index_payload(col->index); + ray_t* keys = ix->u.part.keys; + ray_t* starts = ix->u.part.starts; + ray_t* lens = ix->u.part.lens; + int64_t nparts = ix->u.part.n_parts; + if (!keys || !starts || !lens || keys->len != nparts || + starts->len != nparts || lens->len != nparts) { + ray_release(set_hdr); + return NULL; + } + + ray_t* spans_hdr = ray_alloc(set_len * (int64_t)sizeof(idx_span_t)); + if (!spans_hdr) { + ray_release(set_hdr); + return NULL; + } + idx_span_t* spans = (idx_span_t*)ray_data(spans_hdr); + const int64_t* stv = (const int64_t*)ray_data(starts); + const int64_t* lnv = (const int64_t*)ray_data(lens); + const uint8_t* kb = (const uint8_t*)ray_data(keys); + int64_t nspans = 0; + int64_t total = 0; + for (int64_t p = 0; p < nparts; p++) { + int64_t key = col_is_sym + ? ray_read_sym(kb, p, RAY_SYM, keys->attrs) + : set_vec_read_i64(kb, keys->type, p); + if (!sorted_i64_contains(set_scratch, set_len, key)) continue; + spans[nspans].start = stv[p]; + spans[nspans].len = lnv[p]; + total += lnv[p]; + nspans++; + } + ray_release(set_hdr); + ray_t* block = rowsel_from_sorted_spans(n, spans, nspans, total); + ray_release(spans_hdr); + return block; + } + /* Resolve every set element to its GROUP up front (i64 compares against * gkeys — no column reads), so the union size is known EXACTLY before * any collection work: a dense union (where the SIMD membership scan @@ -1547,6 +1740,55 @@ static double sort_read_f64(const uint8_t* base, int8_t t, int64_t rid) { double v; memcpy(&v, base + rid * 8, 8); return v; } +/* Build a rowsel for the contiguous physical-row interval [lo, hi). Full + * morsels are represented by an ALL flag and consume no idx[] space; only + * the two possible boundary morsels contribute local row ids. */ +static ray_t* rowsel_from_contiguous_span(int64_t n, int64_t lo, int64_t hi) { + if (n < 0 || lo < 0 || hi < lo || hi > n) return NULL; + + int64_t total = hi - lo; + uint32_t n_segs = n > 0 + ? (uint32_t)((n + RAY_MORSEL_ELEMS - 1) / RAY_MORSEL_ELEMS) + : 0; + int64_t idx_count = 0; + for (uint32_t s = 0; s < n_segs; s++) { + int64_t ss = (int64_t)s * RAY_MORSEL_ELEMS; + int64_t se = ss + RAY_MORSEL_ELEMS; + if (se > n) se = n; + int64_t a = lo > ss ? lo : ss; + int64_t b = hi < se ? hi : se; + int64_t pass = b > a ? b - a : 0; + if (pass > 0 && pass != se - ss) idx_count += pass; + } + + ray_t* block = ray_rowsel_new(n, total, idx_count); + if (!block) return NULL; + uint8_t* flags = ray_rowsel_flags(block); + uint32_t* offsets = ray_rowsel_offsets(block); + uint16_t* ids = ray_rowsel_idx(block); + uint32_t cum = 0; + for (uint32_t s = 0; s < n_segs; s++) { + offsets[s] = cum; + int64_t ss = (int64_t)s * RAY_MORSEL_ELEMS; + int64_t se = ss + RAY_MORSEL_ELEMS; + if (se > n) se = n; + int64_t a = lo > ss ? lo : ss; + int64_t b = hi < se ? hi : se; + int64_t pass = b > a ? b - a : 0; + if (pass == 0) { + flags[s] = RAY_SEL_NONE; + } else if (pass == se - ss) { + flags[s] = RAY_SEL_ALL; + } else { + flags[s] = RAY_SEL_MIX; + for (int64_t r = a; r < b; r++) + ids[cum++] = (uint16_t)(r - ss); + } + } + offsets[n_segs] = cum; + return block; +} + /* lower_bound_i: first sorted position pos where value_at(perm[pos]) >= key. * Positions in [0, n) are searched; perm is the sort permutation. */ static int64_t sort_lower_i(const int64_t* perm, int64_t n, @@ -1666,6 +1908,66 @@ ray_t* ray_index_range_rowsel(ray_t* col, uint16_t cmp_op, return block; } +ray_t* ray_sorted_range_rowsel(ray_t* col, uint16_t cmp_op, + int64_t key_i, double key_f, bool is_float) { + if (!col || RAY_IS_ERR(col) || cmp_op == OP_NE) return NULL; + if (!ray_attr_is_sorted(col) || (col->attrs & RAY_ATTR_HAS_NULLS)) + return NULL; + + bool col_is_float = (col->type == RAY_F32 || col->type == RAY_F64); + bool col_is_int = col->type == RAY_BOOL || col->type == RAY_U8 || + col->type == RAY_I16 || col->type == RAY_I32 || + col->type == RAY_I64 || col->type == RAY_DATE || + col->type == RAY_TIME || col->type == RAY_TIMESTAMP; + if (!col_is_float && !col_is_int) return NULL; + if ((bool)is_float != col_is_float) return NULL; + + int64_t n = col->len; + const uint8_t* base = (const uint8_t*)ray_data(col); + int8_t t = col->type; + int64_t lower = 0, upper = 0; + if (!is_float) { + int64_t hi = n; + while (lower < hi) { + int64_t mid = lower + (hi - lower) / 2; + if (sort_read_i64(base, t, mid) < key_i) lower = mid + 1; + else hi = mid; + } + upper = lower; + hi = n; + while (upper < hi) { + int64_t mid = upper + (hi - upper) / 2; + if (sort_read_i64(base, t, mid) <= key_i) upper = mid + 1; + else hi = mid; + } + } else { + int64_t hi = n; + while (lower < hi) { + int64_t mid = lower + (hi - lower) / 2; + if (sort_read_f64(base, t, mid) < key_f) lower = mid + 1; + else hi = mid; + } + upper = lower; + hi = n; + while (upper < hi) { + int64_t mid = upper + (hi - upper) / 2; + if (sort_read_f64(base, t, mid) <= key_f) upper = mid + 1; + else hi = mid; + } + } + + int64_t lo, hi; + switch (cmp_op) { + case OP_LT: lo = 0; hi = lower; break; + case OP_LE: lo = 0; hi = upper; break; + case OP_GT: lo = upper; hi = n; break; + case OP_GE: lo = lower; hi = n; break; + case OP_EQ: lo = lower; hi = upper; break; + default: return NULL; + } + return rowsel_from_contiguous_span(n, lo, hi); +} + /* -------------------------------------------------------------------------- * Bloom-index definite-absent probe * @@ -2409,12 +2711,14 @@ int64_t ray_index_find_row(ray_t* col, int64_t key) { * sites outside this file (the eq+range conjunction in exec.c) construct * selections from index slices. */ -int64_t ray_index_hash_sym_slices(ray_t* col, ray_t* keys, - ray_idx_slice_t** out, ray_t** hdr_out) { +int64_t ray_index_sym_slices(ray_t* col, ray_t* keys, + ray_idx_slice_t** out, ray_t** hdr_out) { *out = NULL; *hdr_out = NULL; if (!col || !keys) return -1; if (col->type != RAY_SYM) return -1; - if (!idx_fresh_nonull(col, RAY_IDX_HASH)) return -1; + ray_idx_kind_t kind = ray_index_kind(col); + if ((kind != RAY_IDX_HASH && kind != RAY_IDX_PART) || + !idx_fresh_nonull(col, kind)) return -1; bool atom = (keys->type == -RAY_SYM); int64_t nkeys; @@ -2446,13 +2750,40 @@ int64_t ray_index_hash_sym_slices(ray_t* col, ray_t* keys, if (!shdr) { ray_free(dhdr); return -1; } ray_idx_slice_t* sl = (ray_idx_slice_t*)ray_data(shdr); int64_t k = 0; - for (int64_t i = 0; i < nd; i++) { - const int64_t* rows = NULL; - int64_t n = 0; - int hit = ray_index_hash_group(col, doms[i], &rows, &n); - if (hit < 0) { ray_free(dhdr); ray_free(shdr); return -1; } - if (hit == 0 || n == 0) continue; /* in domain, no rows */ - sl[k].dom = doms[i]; sl[k].rows = rows; sl[k].n = n; k++; + if (kind == RAY_IDX_HASH) { + for (int64_t i = 0; i < nd; i++) { + const int64_t* rows = NULL; + int64_t n = 0; + int hit = ray_index_hash_group(col, doms[i], &rows, &n); + if (hit < 0) { ray_free(dhdr); ray_free(shdr); return -1; } + if (hit == 0 || n == 0) continue; + sl[k].dom = doms[i]; sl[k].rows = rows; + sl[k].first = 0; sl[k].n = n; k++; + } + } else { + ray_index_t* ix = ray_index_payload(col->index); + ray_t* pkeys = ix->u.part.keys; + ray_t* starts = ix->u.part.starts; + ray_t* lens = ix->u.part.lens; + int64_t np = ix->u.part.n_parts; + if (!pkeys || !starts || !lens || pkeys->len != np || + starts->len != np || lens->len != np) { + ray_free(dhdr); ray_free(shdr); return -1; + } + const uint8_t* pk = (const uint8_t*)ray_data(pkeys); + const int64_t* ps = (const int64_t*)ray_data(starts); + const int64_t* pl = (const int64_t*)ray_data(lens); + for (int64_t i = 0; i < nd; i++) { + for (int64_t p = 0; p < np; p++) { + int64_t pdom = ray_read_sym(pk, p, RAY_SYM, pkeys->attrs); + if (pdom != doms[i]) continue; + if (pl[p] > 0) { + sl[k].dom = doms[i]; sl[k].rows = NULL; + sl[k].first = ps[p]; sl[k].n = pl[p]; k++; + } + break; + } + } } ray_free(dhdr); if (k == 0) { ray_free(shdr); return 0; } diff --git a/src/ops/idxop.h b/src/ops/idxop.h index 27bcee4b..973b3197 100644 --- a/src/ops/idxop.h +++ b/src/ops/idxop.h @@ -191,9 +191,9 @@ static inline ray_index_t* ray_index_payload(ray_t* idx) { * Diagnostic, unsynchronized (same caveat as ray_expr_bail_counts). */ typedef enum { IDX_SITE_FILTER_ZONE = 0, IDX_SITE_FILTER_BLOOM, IDX_SITE_FILTER_HASH, - IDX_SITE_FILTER_RANGE, IDX_SITE_IN, IDX_SITE_FIND, IDX_SITE_SORT, - IDX_SITE_DISTINCT, IDX_SITE_ASOF, IDX_SITE_FILTER_EQRANGE, - IDX_SITE_GROUP_SLICE, IDX_SITE__N + IDX_SITE_FILTER_PART, IDX_SITE_FILTER_RANGE, IDX_SITE_IN, IDX_SITE_FIND, + IDX_SITE_SORT, IDX_SITE_DISTINCT, IDX_SITE_ASOF, + IDX_SITE_FILTER_EQRANGE, IDX_SITE_GROUP_SLICE, IDX_SITE__N } idx_site_t; extern uint64_t ray_idx_consults[IDX_SITE__N]; extern uint64_t ray_idx_hits[IDX_SITE__N]; @@ -326,11 +326,17 @@ bool ray_index_bloom_absent(ray_t* col, int64_t key); * has NaN / -0 semantics the unfused compare kernel handles. */ ray_t* ray_index_hash_eq_rowsel(ray_t* col, int64_t key); -/* ===== Hash-index IN probe ===== +/* Direct equality selection for a fresh part index. `key` is a numeric + * value or a column-domain SYM id. Dense matches remain eligible because + * the part payload already stores one contiguous physical span. */ +ray_t* ray_index_part_eq_rowsel(ray_t* col, int64_t key); + +/* ===== Hash/part-index IN probe ===== * - * Build a rowsel of all rows whose value is in set_vec via per-element - * hash-chain probes. Integer-family and SYM columns (same conservatism as - * the hash-eq path: F32/F64 NaN/-0 semantics belong to the scan kernel). + * Build a rowsel of all rows whose value is in set_vec via per-element hash + * probes or direct physical spans from a part index. Integer-family and SYM + * columns (same conservatism as the hash-eq path: F32/F64 NaN/-0 semantics + * belong to the scan kernel). * * set_vec must match the column family: an integer-family typed vec for an * integer column, a SYM vec for a SYM column (each set symbol is resolved @@ -340,10 +346,10 @@ ray_t* ray_index_hash_eq_rowsel(ray_t* col, int64_t key); * Returns: * - A fresh rowsel block (rc=1) on success — install on g->selection. * An empty set yields a valid all-NONE rowsel (NOT NULL). - * - NULL when the column is not eligible (no hash index, stale, float- - * family column, unsupported set type, OOM) or the union is too dense - * (total chain-step budget exceeded — the SIMD membership scan wins at - * that density). Caller falls back to the scan path. */ + * - NULL when the column is not eligible (no hash/part index, stale, + * float-family column, unsupported set type, OOM) or a hash-index union + * is too dense. Part-index unions retain dense full morsels efficiently. + * Caller falls back to the scan path. */ ray_t* ray_index_in_rowsel(ray_t* col, ray_t* set_vec); /* ===== Hash-index find (point lookup) ===== @@ -387,20 +393,23 @@ ray_t* ray_index_rowsel_from_ids(int64_t nrows, const int64_t* ids, int64_t n); * valid while the index is). */ typedef struct { int64_t dom; /* column-domain key id */ - const int64_t* rows; /* ascending row ids (borrowed) */ + const int64_t* rows; /* ascending row ids (borrowed); NULL if contiguous */ + int64_t first; /* first physical row when rows == NULL */ int64_t n; /* slice length (> 0) */ } ray_idx_slice_t; /* Resolve a SYM eq key (sym ATOM, global intern id) or IN set (SYM vec, - * any domain) against `col`'s fresh CSR hash index into per-key slices, + * any domain) against `col`'s fresh hash or part index into per-key slices, * canonicalized ascending by column-domain id (duplicates collapsed; - * absent / rowless keys dropped). On success returns K >= 0 and, for + * absent / rowless keys dropped). Hash slices borrow CSR row ids; part + * slices use rows==NULL plus their contiguous physical first row. On success + * returns K >= 0 and, for * K > 0, *out points into a fresh alloc block returned via *hdr_out * (release with ray_free). K == 0 → provably no surviving rows (*out * and *hdr_out NULL). Returns -1 when ineligible: stale or missing - * hash index, null-bearing or non-SYM column, wrong key shape. */ -int64_t ray_index_hash_sym_slices(ray_t* col, ray_t* keys, - ray_idx_slice_t** out, ray_t** hdr_out); + * hash/part index, null-bearing or non-SYM column, wrong key shape. */ +int64_t ray_index_sym_slices(ray_t* col, ray_t* keys, + ray_idx_slice_t** out, ray_t** hdr_out); /* ===== Sort-index range probe ===== * @@ -421,6 +430,14 @@ int64_t ray_index_hash_sym_slices(ray_t* col, ray_t* keys, ray_t* ray_index_range_rowsel(ray_t* col, uint16_t cmp_op, int64_t key_i, double key_f, bool is_float); +/* Build a rowsel by binary-searching a physically non-descending column. + * Unlike the sort-index route above, matching row ids are already one + * contiguous interval, so dense spans are eligible and the rowsel stores + * only the at-most-two partial boundary morsels. Requires the verified + * RAY_ATTR_SORTED marker and a null-free numeric column. */ +ray_t* ray_sorted_range_rowsel(ray_t* col, uint16_t cmp_op, + int64_t key_i, double key_f, bool is_float); + /* ===== Sort-index ORDER BY permutation probe ===== * * Returns a BORROWED pointer to the ascending permutation vector stored in diff --git a/src/ops/ops.h b/src/ops/ops.h index 20e7c4e8..abbf2d59 100644 --- a/src/ops/ops.h +++ b/src/ops/ops.h @@ -519,9 +519,9 @@ typedef struct ray_graph { /* Slice-group hint (FILTER(in/eq on c) + GROUP(by c) fusion). * Armed by ray_slice_group_probe (exec.c) INSTEAD of executing the * WHERE filter when the predicate is exactly membership on the - * single bare group-key column and that column carries a fresh CSR - * hash index: surviving groups are then exactly the key set and - * each group's rows are its full CSR slice. sg_col is the retained + * single bare group-key column and that column carries a fresh hash + * or part index: surviving groups are then exactly the key set and + * each group's rows are its full index slice. sg_col is the retained * key column; sg_slices_hdr is an alloc block of ray_idx_slice_t * (idxop.h) resolved at probe time, ascending by column-domain id. * exec_group consumes the hint — aggregating the slices directly or diff --git a/src/ops/query.c b/src/ops/query.c index 02c048de..7a43fecc 100644 --- a/src/ops/query.c +++ b/src/ops/query.c @@ -4351,7 +4351,7 @@ static uint8_t win_kind_from_name(const char* s, size_t len) { } /* - * (window {from: T part: [col …] order: [col …] frame: 'whole|'running + * (window {from: T part: [col …] order: [col …] frame: 'whole|'running|N * funcs: {outname: (fn inputcol) …}}) * * Builds a DAG OP_WINDOW node over the table T, executes it, then renames the @@ -4360,7 +4360,9 @@ static uint8_t win_kind_from_name(const char* s, size_t len) { * Special form: args[0] is the unevaluated dict literal AST. * `from:` is the only clause that requires full eval (it's a variable ref). * `part:`/`order:` arrive as RAY_SYM vectors or -RAY_SYM atoms (column refs). - * `frame:` arrives as a quoted -RAY_SYM atom ('whole or 'running). + * `frame:` is either a quoted -RAY_SYM atom ('whole or 'running), or a + * positive integer N for a fixed trailing N-row frame including the current + * row. * `funcs:` is a nested RAY_DICT whose values are list expressions (fn col). */ ray_t* ray_window_fn(ray_t** args, int64_t n) { @@ -4403,22 +4405,41 @@ ray_t* ray_window_fn(ray_t** args, int64_t n) { } /* ── frame: ── */ ray_t* frame_expr = dict_get(dict, "frame"); + uint8_t frame_start = RAY_BOUND_UNBOUNDED_PRECEDING; uint8_t frame_end = RAY_BOUND_UNBOUNDED_FOLLOWING; /* default: whole partition */ + int64_t frame_start_n = 0; if (frame_expr) { - if (frame_expr->type != -RAY_SYM) { - DICT_VIEW_CLOSE(fv); ray_release(tbl); - return ray_error("domain", "window: frame must be 'whole or 'running"); - } - ray_t* fs = ray_sym_str(frame_expr->i64); - if (fs) { - const char* fsp = ray_str_ptr(fs); - size_t fsl = ray_str_len(fs); - if (fsl == 7 && memcmp(fsp, "running", 7) == 0) - frame_end = RAY_BOUND_CURRENT_ROW; - else if (!(fsl == 5 && memcmp(fsp, "whole", 5) == 0)) { + if (frame_expr->type == -RAY_SYM) { + ray_t* fs = ray_sym_str(frame_expr->i64); + if (fs) { + const char* fsp = ray_str_ptr(fs); + size_t fsl = ray_str_len(fs); + if (fsl == 7 && memcmp(fsp, "running", 7) == 0) + frame_end = RAY_BOUND_CURRENT_ROW; + else if (!(fsl == 5 && memcmp(fsp, "whole", 5) == 0)) { + DICT_VIEW_CLOSE(fv); ray_release(tbl); + return ray_error("domain", "window: frame must be 'whole, 'running, or a positive row count"); + } + } + } else { + int64_t width = 0; + switch (frame_expr->type) { + case -RAY_I64: width = frame_expr->i64; break; + case -RAY_I32: width = frame_expr->i32; break; + case -RAY_I16: width = frame_expr->i16; break; + case -RAY_U8: + case -RAY_BOOL: width = frame_expr->u8; break; + default: + DICT_VIEW_CLOSE(fv); ray_release(tbl); + return ray_error("domain", "window: frame must be 'whole, 'running, or a positive row count"); + } + if (width < 1) { DICT_VIEW_CLOSE(fv); ray_release(tbl); - return ray_error("domain", "window: frame must be 'whole or 'running"); + return ray_error("range", "window: trailing frame row count must be >= 1"); } + frame_start = RAY_BOUND_N_PRECEDING; + frame_end = RAY_BOUND_CURRENT_ROW; + frame_start_n = width - 1; } } @@ -4572,8 +4593,8 @@ ray_t* ray_window_fn(ray_t** args, int64_t n) { func_kinds, func_inputs, func_params, (uint32_t)n_funcs, RAY_FRAME_ROWS, - RAY_BOUND_UNBOUNDED_PRECEDING, frame_end, - 0, 0); + frame_start, frame_end, + frame_start_n, 0); scratch_free(funcs_hdr); scratch_free(order_hdr); scratch_free(part_hdr); if (!win_op) { scratch_free(outname_hdr); ray_graph_free(g); ray_release(tbl); @@ -8320,7 +8341,7 @@ ray_t* ray_select(ray_t** args, int64_t n) { root = ray_optimize(g, root); /* Slice-group fusion: when the WHERE predicate is exactly * membership (in/==) on the single bare group-key column - * and that column carries a fresh CSR hash index, skip + * and that column carries a fresh hash or part index, skip * the filter scan entirely — exec_group aggregates the * key slices directly (or folds them into the equivalent * selection). count-distinct outputs stay on the filter diff --git a/src/ops/rowsel.c b/src/ops/rowsel.c index 30d1043a..321e8d70 100644 --- a/src/ops/rowsel.c +++ b/src/ops/rowsel.c @@ -578,3 +578,114 @@ ray_t* ray_rowsel_refine(ray_t* existing, ray_t* pred) { ray_release(pop_block); return block; } + +ray_t* ray_rowsel_intersect(ray_t* a, ray_t* b) { + if (!a || !b) return NULL; + ray_rowsel_t* am = ray_rowsel_meta(a); + ray_rowsel_t* bm = ray_rowsel_meta(b); + if (am->nrows != bm->nrows || am->n_segs != bm->n_segs) return NULL; + + uint32_t ns = am->n_segs; + const uint8_t* af = ray_rowsel_flags(a); + const uint8_t* bf = ray_rowsel_flags(b); + const uint32_t* ao = ray_rowsel_offsets(a); + const uint32_t* bo = ray_rowsel_offsets(b); + const uint16_t* ai = ray_rowsel_idx(a); + const uint16_t* bi = ray_rowsel_idx(b); + int64_t total = 0; + int64_t idx_count = 0; + + /* Count the intersection per segment. MIX arrays are sorted by every + * producer, so MIX∩MIX is a linear two-cursor walk. */ + for (uint32_t s = 0; s < ns; s++) { + int64_t ss = (int64_t)s * RAY_MORSEL_ELEMS; + int64_t se = ss + RAY_MORSEL_ELEMS; + if (se > am->nrows) se = am->nrows; + int64_t seg_len = se - ss; + int64_t pc = 0; + if (af[s] == RAY_SEL_NONE || bf[s] == RAY_SEL_NONE) { + pc = 0; + } else if (af[s] == RAY_SEL_ALL && bf[s] == RAY_SEL_ALL) { + pc = seg_len; + } else if (af[s] == RAY_SEL_ALL) { + pc = bo[s + 1] - bo[s]; + } else if (bf[s] == RAY_SEL_ALL) { + pc = ao[s + 1] - ao[s]; + } else { + uint32_t ap = ao[s], ae = ao[s + 1]; + uint32_t bp = bo[s], be = bo[s + 1]; + while (ap < ae && bp < be) { + uint16_t av = ai[ap], bv = bi[bp]; + if (av < bv) ap++; + else if (bv < av) bp++; + else { pc++; ap++; bp++; } + } + } + total += pc; + if (pc > 0 && pc != seg_len) idx_count += pc; + } + + ray_t* out = ray_rowsel_new(am->nrows, total, idx_count); + if (!out) return NULL; + uint8_t* of = ray_rowsel_flags(out); + uint32_t* oo = ray_rowsel_offsets(out); + uint16_t* oi = ray_rowsel_idx(out); + uint32_t cum = 0; + + for (uint32_t s = 0; s < ns; s++) { + oo[s] = cum; + int64_t ss = (int64_t)s * RAY_MORSEL_ELEMS; + int64_t se = ss + RAY_MORSEL_ELEMS; + if (se > am->nrows) se = am->nrows; + int64_t seg_len = se - ss; + + if (af[s] == RAY_SEL_NONE || bf[s] == RAY_SEL_NONE) { + of[s] = RAY_SEL_NONE; + } else if (af[s] == RAY_SEL_ALL && bf[s] == RAY_SEL_ALL) { + of[s] = RAY_SEL_ALL; + } else { + const uint16_t* src = NULL; + uint32_t begin = 0, end = 0; + if (af[s] == RAY_SEL_ALL) { + src = bi; begin = bo[s]; end = bo[s + 1]; + } else if (bf[s] == RAY_SEL_ALL) { + src = ai; begin = ao[s]; end = ao[s + 1]; + } + + if (src) { + uint32_t pc = end - begin; + if ((int64_t)pc == seg_len) { + of[s] = RAY_SEL_ALL; + } else { + for (uint32_t p = begin; p < end; p++) oi[cum++] = src[p]; + of[s] = pc == 0 ? RAY_SEL_NONE : RAY_SEL_MIX; + } + } else { + uint32_t ap = ao[s], ae = ao[s + 1]; + uint32_t bp = bo[s], be = bo[s + 1]; + uint32_t count = 0; + uint32_t cp = ap, dp = bp; + while (cp < ae && dp < be) { + uint16_t av = ai[cp], bv = bi[dp]; + if (av < bv) cp++; + else if (bv < av) dp++; + else { count++; cp++; dp++; } + } + if ((int64_t)count == seg_len) { + of[s] = RAY_SEL_ALL; + continue; + } + while (ap < ae && bp < be) { + uint16_t av = ai[ap], bv = bi[bp]; + if (av < bv) ap++; + else if (bv < av) bp++; + else { oi[cum++] = av; ap++; bp++; } + } + } + if (of[s] != RAY_SEL_ALL) + of[s] = cum == oo[s] ? RAY_SEL_NONE : RAY_SEL_MIX; + } + } + oo[ns] = cum; + return out; +} diff --git a/src/ops/rowsel.h b/src/ops/rowsel.h index e9fa3c05..a4139e4f 100644 --- a/src/ops/rowsel.h +++ b/src/ops/rowsel.h @@ -184,6 +184,11 @@ ray_t* ray_rowsel_to_indices(ray_t* sel); * the old selection after replacing it. */ ray_t* ray_rowsel_refine(ray_t* existing, ray_t* pred); +/* Intersect two rowsels over the same source row domain. Both inputs are + * borrowed; the returned rowsel is fresh. Runs in O(n_segs + mixed ids), + * preserving NONE/ALL compression without materializing a bool vector. */ +ray_t* ray_rowsel_intersect(ray_t* a, ray_t* b); + /* ────────────────────────────────────────────────────────────────── * Streaming builder — per-morsel selection emitter * diff --git a/src/ops/system.c b/src/ops/system.c index 38905fef..4d0751d3 100644 --- a/src/ops/system.c +++ b/src/ops/system.c @@ -29,7 +29,10 @@ #include "ops/internal.h" /* ray_group_perpart_runs — (.sys.mem) counter */ #include "mem/heap.h" #include "mem/sys.h" +#include "core/block.h" +#include "ops/idxop.h" #include "core/profile.h" /* g_ray_profile — (.sys.prof) */ +#include "core/qmeasure.h" /* shared per-query measurement lifecycle */ #include "core/qlog.h" /* g_qlog — (.sys.querylog) */ #include "store/serde.h" #include "store/splay.h" @@ -479,9 +482,310 @@ ray_t* ray_meta_fn(ray_t* x) { return ray_dict_new(keys, vals); } -/* (.sys.gc) -- no-op garbage collection trigger, return 0. Variadic - * so the call site can be (.sys.gc) without the dummy-arg ceremony. */ -ray_t* ray_gc_fn(ray_t** args, int64_t n) { (void)args; (void)n; return ray_i64(0); } +/* Object-graph size ------------------------------------------------------- + * + * `.mem.objsize` reports the logical bytes retained by a Rayfall value: + * every reachable ray_t header and its live payload, with shared children + * counted once. It deliberately does not report buddy-block slack, global + * symbol dictionaries, or opaque native handles (graphs/HNSW/lazy plans). + * That makes the result stable across allocator tuning and useful for + * inspecting materialised query results. + * + * The traversal is iterative and pointer-deduplicated. Besides preventing + * double-counting shared table columns, the visited set makes it safe for any + * future compound object that introduces a cycle. */ +typedef struct { + ray_t** stack; + size_t stack_len; + size_t stack_cap; + ray_t** seen; + size_t seen_count; + size_t seen_cap; +} ray_objsize_walk_t; + +static uint64_t objsize_ptr_hash(const ray_t* p) { + uint64_t x = (uint64_t)(uintptr_t)p; + x ^= x >> 33; + x *= UINT64_C(0xff51afd7ed558ccd); + x ^= x >> 33; + x *= UINT64_C(0xc4ceb9fe1a85ec53); + return x ^ (x >> 33); +} + +static bool objsize_stack_push(ray_objsize_walk_t* w, ray_t* child) { + if (!child || RAY_IS_ERR(child)) return true; + if (w->stack_len == w->stack_cap) { + size_t cap = w->stack_cap ? w->stack_cap * 2 : 64; + if (cap < w->stack_cap || cap > SIZE_MAX / sizeof(ray_t*)) return false; + ray_t** grown = (ray_t**)ray_sys_realloc(w->stack, cap * sizeof(ray_t*)); + if (!grown) return false; + w->stack = grown; + w->stack_cap = cap; + } + w->stack[w->stack_len++] = child; + return true; +} + +static bool objsize_seen_grow(ray_objsize_walk_t* w) { + size_t cap = w->seen_cap ? w->seen_cap * 2 : 128; + if (cap < w->seen_cap || cap > SIZE_MAX / sizeof(ray_t*)) return false; + ray_t** grown = (ray_t**)ray_sys_alloc(cap * sizeof(ray_t*)); + if (!grown) return false; + memset(grown, 0, cap * sizeof(ray_t*)); + + if (w->seen) { + size_t mask = cap - 1; + for (size_t i = 0; i < w->seen_cap; i++) { + ray_t* p = w->seen[i]; + if (!p) continue; + size_t slot = (size_t)objsize_ptr_hash(p) & mask; + while (grown[slot]) slot = (slot + 1) & mask; + grown[slot] = p; + } + ray_sys_free(w->seen); + } + w->seen = grown; + w->seen_cap = cap; + return true; +} + +/* Returns true when p was newly inserted, false when already seen. OOM is + * reported separately through *ok so a duplicate is not confused with an + * allocation failure. */ +static bool objsize_seen_insert(ray_objsize_walk_t* w, ray_t* p, bool* ok) { + if (!w->seen_cap || (w->seen_count + 1) * 10 >= w->seen_cap * 7) { + if (!objsize_seen_grow(w)) { *ok = false; return false; } + } + size_t mask = w->seen_cap - 1; + size_t slot = (size_t)objsize_ptr_hash(p) & mask; + while (w->seen[slot]) { + if (w->seen[slot] == p) return false; + slot = (slot + 1) & mask; + } + w->seen[slot] = p; + w->seen_count++; + return true; +} + +static size_t objsize_shallow(ray_t* v) { + if (v->attrs & RAY_ATTR_SLICE) return sizeof(ray_t); + if (v->type == RAY_LAMBDA) + return sizeof(ray_t) + 7 * sizeof(ray_t*); + if (v->type == RAY_INDEX) + return sizeof(ray_t) + sizeof(ray_index_t); + if (RAY_IS_PARTED(v->type)) + return sizeof(ray_t) + (v->len > 0 ? (size_t)v->len * sizeof(ray_t*) : 0); + if (v->type == RAY_MAPCOMMON) + return sizeof(ray_t) + 2 * sizeof(ray_t*); + return ray_block_size(v); +} + +static bool objsize_push_index_children(ray_objsize_walk_t* w, ray_index_t* ix) { +#define OBJSIZE_PUSH(child) do { if (!objsize_stack_push(w, (child))) return false; } while (0) + switch ((ray_idx_kind_t)ix->kind) { + case RAY_IDX_HASH: + OBJSIZE_PUSH(ix->u.hash.table); OBJSIZE_PUSH(ix->u.hash.gkeys); + OBJSIZE_PUSH(ix->u.hash.offs); OBJSIZE_PUSH(ix->u.hash.rows); + break; + case RAY_IDX_SORT: OBJSIZE_PUSH(ix->u.sort.perm); break; + case RAY_IDX_BLOOM: OBJSIZE_PUSH(ix->u.bloom.bits); break; + case RAY_IDX_CHUNK_ZONE: + OBJSIZE_PUSH(ix->u.chunk_zone.mins); + OBJSIZE_PUSH(ix->u.chunk_zone.maxs); + OBJSIZE_PUSH(ix->u.chunk_zone.null_bits); + break; + case RAY_IDX_PART: + OBJSIZE_PUSH(ix->u.part.keys); OBJSIZE_PUSH(ix->u.part.starts); + OBJSIZE_PUSH(ix->u.part.lens); + break; + case RAY_IDX_DICT: + OBJSIZE_PUSH(ix->u.dict.codes); OBJSIZE_PUSH(ix->u.dict.first_occ); + break; + case RAY_IDX_ZONE: + case RAY_IDX_NONE: + break; + } +#undef OBJSIZE_PUSH + return true; +} + +static bool objsize_push_children(ray_objsize_walk_t* w, ray_t* v) { +#define OBJSIZE_PUSH(child) do { if (!objsize_stack_push(w, (child))) return false; } while (0) + if (v->type == RAY_LAMBDA) { + ray_t** slots = (ray_t**)ray_data(v); + for (int i = 0; i < 4; i++) OBJSIZE_PUSH(slots[i]); + OBJSIZE_PUSH(slots[5]); + OBJSIZE_PUSH(slots[6]); + return true; + } + if (ray_is_atom(v)) { + if (v->type == -RAY_GUID && v->obj) OBJSIZE_PUSH(v->obj); + if (v->type == -RAY_STR && + !((v->slen >= 1 && v->slen <= 7) || (v->slen == 0 && !v->obj))) + OBJSIZE_PUSH(v->obj); + return true; + } + if (v->attrs & RAY_ATTR_SLICE) { + OBJSIZE_PUSH(v->slice_parent); + return true; + } + if (v->type == RAY_INDEX) + return objsize_push_index_children(w, ray_index_payload(v)); + if (v->attrs & RAY_ATTR_HAS_INDEX) { + OBJSIZE_PUSH(v->index); + /* An indexed STR vector still owns its external character pool in + * aux bytes 8..15; the index pointer only occupies bytes 0..7. */ + if (v->type == RAY_STR && v->str_pool) + OBJSIZE_PUSH(v->str_pool); + return true; + } + if (v->type == RAY_STR && v->str_pool) + OBJSIZE_PUSH(v->str_pool); + if (RAY_IS_PARTED(v->type)) { + ray_t** segs = (ray_t**)ray_data(v); + for (int64_t i = 0; i < v->len; i++) OBJSIZE_PUSH(segs[i]); + return true; + } + if (v->type == RAY_MAPCOMMON || v->type == RAY_TABLE || v->type == RAY_DICT) { + ray_t** slots = (ray_t**)ray_data(v); + OBJSIZE_PUSH(slots[0]); + OBJSIZE_PUSH(slots[1]); + return true; + } + if (v->type == RAY_LIST) { + ray_t** elems = (ray_t**)ray_data(v); + for (int64_t i = 0; i < v->len; i++) OBJSIZE_PUSH(elems[i]); + } +#undef OBJSIZE_PUSH + return true; +} + +ray_t* ray_mem_objsize_fn(ray_t* root) { + if (!root) return ray_error("type", "objsize expects a value"); + ray_objsize_walk_t w = {0}; + if (!objsize_stack_push(&w, root)) return ray_error("oom", NULL); + + uint64_t total = 0; + bool ok = true; + while (w.stack_len && ok) { + ray_t* v = w.stack[--w.stack_len]; + if (!objsize_seen_insert(&w, v, &ok)) continue; + size_t shallow = objsize_shallow(v); + if (UINT64_MAX - total < shallow) { ok = false; break; } + total += shallow; + if (!objsize_push_children(&w, v)) ok = false; + } + ray_sys_free(w.stack); + ray_sys_free(w.seen); + if (!ok) return ray_error("oom", "objsize traversal overflow or allocation failure"); + if (total > INT64_MAX) return ray_error("limit", "objsize exceeds I64 range"); + return ray_i64((int64_t)total); +} + +/* (.mem.ts expr) — evaluate one expression under a process-wide allocation + * trace and return a named measurement dictionary. SPECIAL_FORM: expr is + * deliberately not pre-evaluated. Result sizing and dictionary construction + * happen after both clocks stop, so they do not contaminate the measured + * expression. */ +ray_t* ray_mem_ts_fn(ray_t** args, int64_t n) { + if (n != 1) + return ray_error("arity", ".mem.ts expects 1 argument, got %lld", + (long long)n); + if (!ray_mem_trace_begin()) + return ray_error("state", ".mem.ts measurement already active"); + + ray_query_measure_t measure; + ray_query_measure_begin(&measure); + ray_t* result = ray_eval(args[0]); + ray_query_metrics_t metrics; + ray_query_measure_end(&measure, &metrics); + + ray_mem_trace_t mt; + ray_mem_trace_end(&mt); + + if (!result) return ray_error("state", ".mem.ts expression returned no value"); + if (RAY_IS_ERR(result)) return result; + + ray_t* result_size = ray_mem_objsize_fn(result); + if (!result_size || RAY_IS_ERR(result_size)) { + ray_release(result); + return result_size ? result_size : ray_error("oom", NULL); + } + + enum { MEM_TS_FIELDS = 13 }; + static const char* names[MEM_TS_FIELDS] = { + "result", "time-ns", "memory-bytes", "allocated-bytes", "freed-bytes", + "net-bytes", "peak-live-bytes", "result-bytes", "alloc-count", + "free-count", "workers", "worker-busy-ns", "parallelism" + }; + static const uint8_t name_lens[MEM_TS_FIELDS] = { + 6, 7, 12, 15, 11, 9, 15, 12, 11, 10, 7, 14, 11 + }; + + ray_t* items[MEM_TS_FIELDS] = { + result, + ray_i64(metrics.time_ns), + ray_i64(metrics.memory_bytes), + ray_i64((int64_t)mt.allocated_bytes), + ray_i64((int64_t)mt.freed_bytes), + ray_i64(mt.net_bytes), + ray_i64((int64_t)mt.peak_live_bytes), + result_size, + ray_i64((int64_t)mt.alloc_count), + ray_i64((int64_t)mt.free_count), + ray_i64((int64_t)metrics.workers), + ray_i64((int64_t)metrics.worker_busy_ns), + ray_f64(metrics.parallelism) + }; + for (int i = 0; i < MEM_TS_FIELDS; i++) { + if (!items[i] || RAY_IS_ERR(items[i])) { + ray_t* err = items[i] ? items[i] : ray_error("oom", NULL); + for (int j = 0; j < MEM_TS_FIELDS; j++) + if (j != i && items[j] && !RAY_IS_ERR(items[j])) ray_release(items[j]); + return err; + } + } + + ray_t* keys = ray_sym_vec_new(RAY_SYM_W64, MEM_TS_FIELDS); + ray_t* vals = ray_list_new(MEM_TS_FIELDS); + if (!keys || RAY_IS_ERR(keys) || !vals || RAY_IS_ERR(vals)) { + ray_t* err = (!keys || RAY_IS_ERR(keys)) + ? (keys ? keys : ray_error("oom", NULL)) + : vals; + if (keys && !RAY_IS_ERR(keys)) ray_release(keys); + if (vals && !RAY_IS_ERR(vals)) ray_release(vals); + for (int i = 0; i < MEM_TS_FIELDS; i++) ray_release(items[i]); + return err; + } + + for (int i = 0; i < MEM_TS_FIELDS; i++) { + int64_t sid = ray_sym_intern(names[i], name_lens[i]); + keys = ray_vec_append(keys, &sid); + if (RAY_IS_ERR(keys)) { + ray_release(vals); + for (int j = i; j < MEM_TS_FIELDS; j++) ray_release(items[j]); + return keys; + } + vals = ray_list_append(vals, items[i]); + ray_release(items[i]); + if (RAY_IS_ERR(vals)) { + ray_release(keys); + for (int j = i + 1; j < MEM_TS_FIELDS; j++) ray_release(items[j]); + return vals; + } + } + return ray_dict_new(keys, vals); +} + +/* (.sys.gc) -- perform the allocator maintenance pass used by v1: drain + * foreign frees, flush slab caches, coalesce blocks and release reclaimable + * pages/pools. Rayforce values are reference-counted, so this is allocator + * GC rather than a tracing collector. Variadic to allow `(.sys.gc)`. */ +ray_t* ray_gc_fn(ray_t** args, int64_t n) { + (void)args; (void)n; + ray_heap_gc(); + return ray_i64(0); +} /* (system cmd) -- run shell command, return exit code */ ray_t* ray_system_fn(ray_t* x) { diff --git a/src/ops/window.c b/src/ops/window.c index 081b3a75..df4612af 100644 --- a/src/ops/window.c +++ b/src/ops/window.c @@ -141,7 +141,7 @@ static void win_compute_partition( ray_t* const* order_vecs, uint32_t n_order, ray_t* const* func_vecs, const uint8_t* func_kinds, const int64_t* func_params, uint32_t n_funcs, - uint8_t frame_start, uint8_t frame_end, + uint8_t frame_start, uint8_t frame_end, int64_t frame_start_n, const int64_t* sorted_idx, int64_t ps, int64_t pe, ray_t* const* result_vecs, const bool* is_f64) { @@ -154,6 +154,8 @@ static void win_compute_partition( ray_t* rvec = result_vecs[f]; bool whole = (frame_start == RAY_BOUND_UNBOUNDED_PRECEDING && frame_end == RAY_BOUND_UNBOUNDED_FOLLOWING); + bool trailing = (frame_start == RAY_BOUND_N_PRECEDING && + frame_end == RAY_BOUND_CURRENT_ROW); switch (kind) { case RAY_WIN_ROW_NUMBER: { @@ -199,6 +201,12 @@ static void win_compute_partition( if (whole) { for (int64_t i = ps; i < pe; i++) out[sorted_idx[i]] = part_len; + } else if (trailing) { + int64_t width = frame_start_n + 1; + for (int64_t i = ps; i < pe; i++) { + int64_t seen = i - ps + 1; + out[sorted_idx[i]] = seen < width ? seen : width; + } } else { for (int64_t i = ps; i < pe; i++) out[sorted_idx[i]] = i - ps + 1; @@ -210,38 +218,44 @@ static void win_compute_partition( if (is_f64[f]) { double* out = (double*)ray_data(rvec); if (whole) { - double t = 0.0; + double total = 0.0; for (int64_t i = ps; i < pe; i++) if (!ray_vec_is_null(fvec, sorted_idx[i])) - t += win_read_f64(fvec, sorted_idx[i]); - /* Single-null float model: window SUM can overflow → ±Inf; - * canonicalize to NULL_F64 (HAS_NULLS via win_finalize_nulls). */ - t = ray_f64_fin(t); - for (int64_t i = ps; i < pe; i++) - out[sorted_idx[i]] = t; + total += win_read_f64(fvec, sorted_idx[i]); + total = ray_f64_fin(total); + for (int64_t i = ps; i < pe; i++) out[sorted_idx[i]] = total; } else { double acc = 0.0; for (int64_t i = ps; i < pe; i++) { - if (!ray_vec_is_null(fvec, sorted_idx[i])) - acc += win_read_f64(fvec, sorted_idx[i]); - out[sorted_idx[i]] = ray_f64_fin(acc); + int64_t row = sorted_idx[i]; + if (!ray_vec_is_null(fvec, row)) acc += win_read_f64(fvec, row); + if (trailing) { + int64_t drop = i - frame_start_n - 1; + if (drop >= ps && !ray_vec_is_null(fvec, sorted_idx[drop])) + acc -= win_read_f64(fvec, sorted_idx[drop]); + } + out[row] = ray_f64_fin(acc); } } } else { int64_t* out = (int64_t*)ray_data(rvec); if (whole) { - int64_t t = 0; + int64_t total = 0; for (int64_t i = ps; i < pe; i++) if (!ray_vec_is_null(fvec, sorted_idx[i])) - t += win_read_i64(fvec, sorted_idx[i]); - for (int64_t i = ps; i < pe; i++) - out[sorted_idx[i]] = t; + total += win_read_i64(fvec, sorted_idx[i]); + for (int64_t i = ps; i < pe; i++) out[sorted_idx[i]] = total; } else { int64_t acc = 0; for (int64_t i = ps; i < pe; i++) { - if (!ray_vec_is_null(fvec, sorted_idx[i])) - acc += win_read_i64(fvec, sorted_idx[i]); - out[sorted_idx[i]] = acc; + int64_t row = sorted_idx[i]; + if (!ray_vec_is_null(fvec, row)) acc += win_read_i64(fvec, row); + if (trailing) { + int64_t drop = i - frame_start_n - 1; + if (drop >= ps && !ray_vec_is_null(fvec, sorted_idx[drop])) + acc -= win_read_i64(fvec, sorted_idx[drop]); + } + out[row] = acc; } } } @@ -251,156 +265,135 @@ static void win_compute_partition( if (!fvec) break; double* out = (double*)ray_data(rvec); if (whole) { - double t = 0.0; - int64_t cnt = 0; - for (int64_t i = ps; i < pe; i++) - if (!ray_vec_is_null(fvec, sorted_idx[i])) { - t += win_read_f64(fvec, sorted_idx[i]); cnt++; - } - if (cnt > 0) { - double avg = ray_f64_fin(t / (double)cnt); - for (int64_t i = ps; i < pe; i++) - out[sorted_idx[i]] = avg; - } else { - for (int64_t i = ps; i < pe; i++) - win_set_null(rvec, sorted_idx[i]); - } - } else { - double acc = 0.0; - int64_t cnt = 0; + double total = 0.0; + int64_t count = 0; for (int64_t i = ps; i < pe; i++) { - if (!ray_vec_is_null(fvec, sorted_idx[i])) { - acc += win_read_f64(fvec, sorted_idx[i]); cnt++; + int64_t row = sorted_idx[i]; + if (!ray_vec_is_null(fvec, row)) { + total += win_read_f64(fvec, row); + count++; } - if (cnt > 0) - out[sorted_idx[i]] = ray_f64_fin(acc / (double)cnt); - else - win_set_null(rvec, sorted_idx[i]); } - } - break; - } - case RAY_WIN_MIN: { - if (!fvec) break; - if (is_f64[f]) { - double* out = (double*)ray_data(rvec); - if (whole) { - double mn = DBL_MAX; int found = 0; - for (int64_t i = ps; i < pe; i++) { - if (ray_vec_is_null(fvec, sorted_idx[i])) continue; - double v = win_read_f64(fvec, sorted_idx[i]); - if (!found || v < mn) { mn = v; found = 1; } - } - if (found) { - for (int64_t i = ps; i < pe; i++) - out[sorted_idx[i]] = mn; - } else { - for (int64_t i = ps; i < pe; i++) - win_set_null(rvec, sorted_idx[i]); - } + if (count > 0) { + double avg = ray_f64_fin(total / (double)count); + for (int64_t i = ps; i < pe; i++) out[sorted_idx[i]] = avg; } else { - double mn = DBL_MAX; int found = 0; - for (int64_t i = ps; i < pe; i++) { - if (!ray_vec_is_null(fvec, sorted_idx[i])) { - double v = win_read_f64(fvec, sorted_idx[i]); - if (!found || v < mn) { mn = v; found = 1; } - } - if (found) - out[sorted_idx[i]] = mn; - else - win_set_null(rvec, sorted_idx[i]); - } + for (int64_t i = ps; i < pe; i++) win_set_null(rvec, sorted_idx[i]); } } else { - int64_t* out = (int64_t*)ray_data(rvec); - if (whole) { - int64_t mn = INT64_MAX; int found = 0; - for (int64_t i = ps; i < pe; i++) { - if (ray_vec_is_null(fvec, sorted_idx[i])) continue; - int64_t v = win_read_i64(fvec, sorted_idx[i]); - if (!found || v < mn) { mn = v; found = 1; } - } - if (found) { - for (int64_t i = ps; i < pe; i++) - out[sorted_idx[i]] = mn; - } else { - for (int64_t i = ps; i < pe; i++) - win_set_null(rvec, sorted_idx[i]); + double acc = 0.0; + int64_t count = 0; + for (int64_t i = ps; i < pe; i++) { + int64_t row = sorted_idx[i]; + if (!ray_vec_is_null(fvec, row)) { + acc += win_read_f64(fvec, row); + count++; } - } else { - int64_t mn = INT64_MAX; int found = 0; - for (int64_t i = ps; i < pe; i++) { - if (!ray_vec_is_null(fvec, sorted_idx[i])) { - int64_t v = win_read_i64(fvec, sorted_idx[i]); - if (!found || v < mn) { mn = v; found = 1; } + if (trailing) { + int64_t drop = i - frame_start_n - 1; + if (drop >= ps && !ray_vec_is_null(fvec, sorted_idx[drop])) { + acc -= win_read_f64(fvec, sorted_idx[drop]); + count--; } - if (found) - out[sorted_idx[i]] = mn; - else - win_set_null(rvec, sorted_idx[i]); } + if (count > 0) out[row] = ray_f64_fin(acc / (double)count); + else win_set_null(rvec, row); } } break; } + case RAY_WIN_MIN: case RAY_WIN_MAX: { if (!fvec) break; + bool want_max = kind == RAY_WIN_MAX; if (is_f64[f]) { double* out = (double*)ray_data(rvec); if (whole) { - double mx = -DBL_MAX; int found = 0; + double best = want_max ? -DBL_MAX : DBL_MAX; + int found = 0; for (int64_t i = ps; i < pe; i++) { - if (ray_vec_is_null(fvec, sorted_idx[i])) continue; - double v = win_read_f64(fvec, sorted_idx[i]); - if (!found || v > mx) { mx = v; found = 1; } + int64_t row = sorted_idx[i]; + if (ray_vec_is_null(fvec, row)) continue; + double v = win_read_f64(fvec, row); + if (!found || (want_max ? v > best : v < best)) { best = v; found = 1; } } - if (found) { - for (int64_t i = ps; i < pe; i++) - out[sorted_idx[i]] = mx; - } else { - for (int64_t i = ps; i < pe; i++) - win_set_null(rvec, sorted_idx[i]); + for (int64_t i = ps; i < pe; i++) { + if (found) out[sorted_idx[i]] = best; + else win_set_null(rvec, sorted_idx[i]); } } else { - double mx = -DBL_MAX; int found = 0; - for (int64_t i = ps; i < pe; i++) { - if (!ray_vec_is_null(fvec, sorted_idx[i])) { - double v = win_read_f64(fvec, sorted_idx[i]); - if (!found || v > mx) { mx = v; found = 1; } + if (!trailing) { + double best = want_max ? -DBL_MAX : DBL_MAX; + int found = 0; + for (int64_t i = ps; i < pe; i++) { + int64_t row = sorted_idx[i]; + if (!ray_vec_is_null(fvec, row)) { + double v = win_read_f64(fvec, row); + if (!found || (want_max ? v > best : v < best)) { best = v; found = 1; } + } + if (found) out[row] = best; + else win_set_null(rvec, row); + } + } else { + for (int64_t i = ps; i < pe; i++) { + int64_t lo = i - frame_start_n; + if (lo < ps) lo = ps; + double best = want_max ? -DBL_MAX : DBL_MAX; + int found = 0; + for (int64_t j = lo; j <= i; j++) { + int64_t row = sorted_idx[j]; + if (ray_vec_is_null(fvec, row)) continue; + double v = win_read_f64(fvec, row); + if (!found || (want_max ? v > best : v < best)) { best = v; found = 1; } + } + if (found) out[sorted_idx[i]] = best; + else win_set_null(rvec, sorted_idx[i]); } - if (found) - out[sorted_idx[i]] = mx; - else - win_set_null(rvec, sorted_idx[i]); } } } else { int64_t* out = (int64_t*)ray_data(rvec); if (whole) { - int64_t mx = INT64_MIN; int found = 0; + int64_t best = want_max ? INT64_MIN : INT64_MAX; + int found = 0; for (int64_t i = ps; i < pe; i++) { - if (ray_vec_is_null(fvec, sorted_idx[i])) continue; - int64_t v = win_read_i64(fvec, sorted_idx[i]); - if (!found || v > mx) { mx = v; found = 1; } + int64_t row = sorted_idx[i]; + if (ray_vec_is_null(fvec, row)) continue; + int64_t v = win_read_i64(fvec, row); + if (!found || (want_max ? v > best : v < best)) { best = v; found = 1; } } - if (found) { - for (int64_t i = ps; i < pe; i++) - out[sorted_idx[i]] = mx; - } else { - for (int64_t i = ps; i < pe; i++) - win_set_null(rvec, sorted_idx[i]); + for (int64_t i = ps; i < pe; i++) { + if (found) out[sorted_idx[i]] = best; + else win_set_null(rvec, sorted_idx[i]); } } else { - int64_t mx = INT64_MIN; int found = 0; - for (int64_t i = ps; i < pe; i++) { - if (!ray_vec_is_null(fvec, sorted_idx[i])) { - int64_t v = win_read_i64(fvec, sorted_idx[i]); - if (!found || v > mx) { mx = v; found = 1; } + if (!trailing) { + int64_t best = want_max ? INT64_MIN : INT64_MAX; + int found = 0; + for (int64_t i = ps; i < pe; i++) { + int64_t row = sorted_idx[i]; + if (!ray_vec_is_null(fvec, row)) { + int64_t v = win_read_i64(fvec, row); + if (!found || (want_max ? v > best : v < best)) { best = v; found = 1; } + } + if (found) out[row] = best; + else win_set_null(rvec, row); + } + } else { + for (int64_t i = ps; i < pe; i++) { + int64_t lo = i - frame_start_n; + if (lo < ps) lo = ps; + int64_t best = want_max ? INT64_MIN : INT64_MAX; + int found = 0; + for (int64_t j = lo; j <= i; j++) { + int64_t row = sorted_idx[j]; + if (ray_vec_is_null(fvec, row)) continue; + int64_t v = win_read_i64(fvec, row); + if (!found || (want_max ? v > best : v < best)) { best = v; found = 1; } + } + if (found) out[sorted_idx[i]] = best; + else win_set_null(rvec, sorted_idx[i]); } - if (found) - out[sorted_idx[i]] = mx; - else - win_set_null(rvec, sorted_idx[i]); } } } @@ -474,20 +467,23 @@ static void win_compute_partition( } case RAY_WIN_FIRST_VALUE: { if (!fvec) break; - bool first_null = ray_vec_is_null(fvec, sorted_idx[ps]); if (is_f64[f]) { double* out = (double*)ray_data(rvec); - double first = first_null ? 0.0 : win_read_f64(fvec, sorted_idx[ps]); for (int64_t i = ps; i < pe; i++) { - out[sorted_idx[i]] = first; - if (first_null) win_set_null(rvec, sorted_idx[i]); + int64_t src = trailing ? i - frame_start_n : ps; + if (src < ps) src = ps; + bool is_null = ray_vec_is_null(fvec, sorted_idx[src]); + out[sorted_idx[i]] = is_null ? 0.0 : win_read_f64(fvec, sorted_idx[src]); + if (is_null) win_set_null(rvec, sorted_idx[i]); } } else { int64_t* out = (int64_t*)ray_data(rvec); - int64_t first = first_null ? 0 : win_read_i64(fvec, sorted_idx[ps]); for (int64_t i = ps; i < pe; i++) { - out[sorted_idx[i]] = first; - if (first_null) win_set_null(rvec, sorted_idx[i]); + int64_t src = trailing ? i - frame_start_n : ps; + if (src < ps) src = ps; + bool is_null = ray_vec_is_null(fvec, sorted_idx[src]); + out[sorted_idx[i]] = is_null ? 0 : win_read_i64(fvec, sorted_idx[src]); + if (is_null) win_set_null(rvec, sorted_idx[i]); } } break; @@ -533,21 +529,27 @@ static void win_compute_partition( if (!fvec) break; int64_t nth = func_params[f]; if (nth < 1) nth = 1; - bool nth_null = (nth > part_len) || - ray_vec_is_null(fvec, sorted_idx[ps + nth - 1]); if (is_f64[f]) { double* out = (double*)ray_data(rvec); - double val = nth_null ? 0.0 : win_read_f64(fvec, sorted_idx[ps + nth - 1]); for (int64_t i = ps; i < pe; i++) { - out[sorted_idx[i]] = val; - if (nth_null) win_set_null(rvec, sorted_idx[i]); + int64_t lo = trailing ? i - frame_start_n : ps; + if (lo < ps) lo = ps; + int64_t src = lo + nth - 1; + bool is_null = src > (whole ? pe - 1 : i) || + ray_vec_is_null(fvec, sorted_idx[src]); + out[sorted_idx[i]] = is_null ? 0.0 : win_read_f64(fvec, sorted_idx[src]); + if (is_null) win_set_null(rvec, sorted_idx[i]); } } else { int64_t* out = (int64_t*)ray_data(rvec); - int64_t val = nth_null ? 0 : win_read_i64(fvec, sorted_idx[ps + nth - 1]); for (int64_t i = ps; i < pe; i++) { - out[sorted_idx[i]] = val; - if (nth_null) win_set_null(rvec, sorted_idx[i]); + int64_t lo = trailing ? i - frame_start_n : ps; + if (lo < ps) lo = ps; + int64_t src = lo + nth - 1; + bool is_null = src > (whole ? pe - 1 : i) || + ray_vec_is_null(fvec, sorted_idx[src]); + out[sorted_idx[i]] = is_null ? 0 : win_read_i64(fvec, sorted_idx[src]); + if (is_null) win_set_null(rvec, sorted_idx[i]); } } break; @@ -566,6 +568,7 @@ typedef struct { uint32_t n_funcs; uint8_t frame_start; uint8_t frame_end; + int64_t frame_start_n; int64_t* sorted_idx; int64_t* part_offsets; ray_t** result_vecs; @@ -580,7 +583,7 @@ static void win_par_fn(void* arg, uint32_t worker_id, win_compute_partition( ctx->order_vecs, ctx->n_order, ctx->func_vecs, ctx->func_kinds, ctx->func_params, - ctx->n_funcs, ctx->frame_start, ctx->frame_end, + ctx->n_funcs, ctx->frame_start, ctx->frame_end, ctx->frame_start_n, ctx->sorted_idx, ctx->part_offsets[p], ctx->part_offsets[p + 1], ctx->result_vecs, ctx->is_f64); } @@ -670,15 +673,13 @@ ray_t* exec_window(ray_graph_t* g, ray_op_t* op, ray_t* tbl) { ray_op_ext_t* ext = find_ext(g, op->id); if (!ext) return ray_error("nyi", NULL); - /* Frame-spec validation (review 2.10): the compute path honors EXACTLY two - * frames — the whole partition (UNBOUNDED PRECEDING .. UNBOUNDED FOLLOWING) - * and the running frame (UNBOUNDED PRECEDING .. CURRENT ROW, ROWS mode). - * Numeric N_PRECEDING/N_FOLLOWING bounds and RANGE-mode CURRENT ROW were - * accepted but SILENTLY degraded to the running frame, yielding the wrong - * window. Reject any unsupported frame LOUDLY instead. - * - whole: end == UNBOUNDED_FOLLOWING (ROWS == RANGE here, mode moot) - * - running: end == CURRENT_ROW and ROWS mode - * Both require start == UNBOUNDED_PRECEDING. */ + /* Frame-spec validation: the compute path honors the whole partition, + * the running frame, and fixed trailing ROWS frames. Other numeric + * bounds and RANGE-mode CURRENT ROW must be rejected loudly rather than + * silently degrading to a different frame. + * - whole: UNBOUNDED PRECEDING .. UNBOUNDED FOLLOWING + * - running: UNBOUNDED PRECEDING .. CURRENT ROW in ROWS mode + * - trailing: N PRECEDING .. CURRENT ROW in ROWS mode */ { uint8_t fs = ext->window.frame_start; uint8_t fe = ext->window.frame_end; @@ -687,10 +688,13 @@ ray_t* exec_window(ray_graph_t* g, ray_op_t* op, ray_t* tbl) { bool running_frame = (fs == RAY_BOUND_UNBOUNDED_PRECEDING && fe == RAY_BOUND_CURRENT_ROW && ext->window.frame_type == RAY_FRAME_ROWS); - if (!whole_frame && !running_frame) + bool trailing_frame = (fs == RAY_BOUND_N_PRECEDING && + fe == RAY_BOUND_CURRENT_ROW && + ext->window.frame_type == RAY_FRAME_ROWS && + ext->window.frame_start_n >= 0); + if (!whole_frame && !running_frame && !trailing_frame) return ray_error("nyi", "unsupported window frame: only " - "ROWS BETWEEN UNBOUNDED PRECEDING AND {CURRENT ROW | " - "UNBOUNDED FOLLOWING} are honored"); + "whole, running, and fixed trailing ROWS frames are honored"); } /* Case-a streaming fast-path: if PARTITION BY includes the physical/date @@ -1383,6 +1387,7 @@ ray_t* exec_window(ray_graph_t* g, ray_op_t* op, ray_t* tbl) { .func_params = ext->window.func_params, .n_funcs = n_funcs, .frame_start = ext->window.frame_start, .frame_end = ext->window.frame_end, + .frame_start_n = ext->window.frame_start_n, .sorted_idx = sorted_idx, .part_offsets = part_offsets, .result_vecs = result_vecs, .is_f64 = is_f64, }; @@ -1393,6 +1398,7 @@ ray_t* exec_window(ray_graph_t* g, ray_op_t* op, ray_t* tbl) { order_vecs, n_order, func_vecs, ext->window.func_kinds, ext->window.func_params, n_funcs, ext->window.frame_start, ext->window.frame_end, + ext->window.frame_start_n, sorted_idx, part_offsets[p], part_offsets[p + 1], result_vecs, is_f64); } diff --git a/src/vec/vec.c b/src/vec/vec.c index 12376785..158fae6e 100644 --- a/src/vec/vec.c +++ b/src/vec/vec.c @@ -103,6 +103,9 @@ static inline bool vec_any_nulls(const ray_t* v) { * reads them. Detect rc>1 and copy the saved pointers via * ray_index_retain_saved instead of moving them out. */ static inline void vec_drop_index_inplace(ray_t* v) { + /* Any payload/length mutation invalidates the physical-order promise, + * even when no accelerator index is attached. */ + v->attrs &= (uint8_t)~RAY_ATTR_SORTED; if (!(v->attrs & RAY_ATTR_HAS_INDEX)) return; ray_t* idx = v->index; ray_index_t* ix = ray_index_payload(idx); diff --git a/test/rfl/collection/time_series_prefix.rfl b/test/rfl/collection/time_series_prefix.rfl index 9b87448c..079af346 100644 --- a/test/rfl/collection/time_series_prefix.rfl +++ b/test/rfl/collection/time_series_prefix.rfl @@ -7,6 +7,17 @@ (fills (as 'F64 (list 0Nf 1.5 0Nf 3.0))) -- (as 'F64 (list 0Nf 1.5 1.5 3.0)) (fills [1 2 3]) -- [1 2 3] +;; Tables are filled column-wise. This is the shape produced by pivot: the +;; index column is already dense while sparse value columns carry forward +;; independently. +(set FT (table [t a b] (list [1 2 3 4] (as 'F64 (list 10.0 0Nf 30.0 0Nf)) (as 'I64 (list 0N 20 0N 40))))) +(set FF (fills FT)) +(key FF) -- [t a b] +(at FF 't) -- [1 2 3 4] +(at FF 'a) -- (as 'F64 [10.0 10.0 30.0 30.0]) +(at FF 'b) -- (as 'I64 (list 0N 20 20 40)) +(type FF) -- 'TABLE + ;; Running aggregates skip null payloads. sum/prod use aggregate identities; ;; avg/min/max stay null until the first non-null value. (sums [1 2 3 4]) -- [1 3 6 10] diff --git a/test/rfl/integration/slice_group.rfl b/test/rfl/integration/slice_group.rfl index 99a4252d..30397c63 100644 --- a/test/rfl/integration/slice_group.rfl +++ b/test/rfl/integration/slice_group.rfl @@ -22,6 +22,9 @@ ;; Sorted twin (contiguous slices — the streaming fast loops). (set TSRT (xasc TP ['s 'v])) (set TSI (table [s v f w] (list (.attr.set 'grouped (at TSRT 's)) (at TSRT 'v) (at TSRT 'f) (at TSRT 'w)))) +;; Same contiguous layout backed by the part index. This must feed the same +;; slice aggregation kernel without expanding each partition into row ids. +(set TPI (table [s v f w] (list (.attr.set 'parted (at TSRT 's)) (at TSRT 'v) (at TSRT 'f) (at TSRT 'w)))) ;; group aa sees v ∈ {0,8,…,3992}: count=500, sum=998000; bb: {1,9,…}: 998500 ;; ── IN set: sums/counts/avg, indexed vs plain vs hand values ── @@ -72,8 +75,11 @@ ;; incl. arith-of-aggs decomposition wrappers ── (set Q3I (select {wp: (/ (sum (* w f)) (as 'F64 (sum w))) from: TI by: s where: (in s ['aa 'bb 'cc 'dd])})) (set Q3P (select {wp: (/ (sum (* w f)) (as 'F64 (sum w))) from: TP by: s where: (in s ['aa 'bb 'cc 'dd])})) +(set Q3PI (select {wp: (/ (sum (* w f)) (as 'F64 (sum w))) from: TPI by: s where: (in s ['aa 'bb 'cc 'dd])})) (min (== (at Q3I 'wp) (at Q3P 'wp))) -- true (min (== (at Q3I 's) (at Q3P 's))) -- true +(min (== (at Q3PI 'wp) (at Q3P 'wp))) -- true +(min (== (at Q3PI 's) (at Q3P 's))) -- true ;; ── eq single key ── (set Q4I (select {sv: (sum v) from: TI by: s where: (== s 'ee)})) diff --git a/test/rfl/query/window.rfl b/test/rfl/query/window.rfl index 0b025a39..1cf618ef 100644 --- a/test/rfl/query/window.rfl +++ b/test/rfl/query/window.rfl @@ -29,6 +29,16 @@ ;; full result in original row order: [7 2 16 4 8 15] (at (window {from: T part: [k] order: [v] frame: 'running funcs: {cs: (sum v)}}) 'cs) -- [7 2 16 4 8 15] +;; ── fixed trailing frame → resets at each partition boundary ──────────── +;; A positive integer frame is the number of rows including the current row. +(set M (table [k o v] (list ['a 'a 'b 'a 'b 'a] [1 2 1 3 2 4] [10 20 100 30 200 40]))) +(set MW (window {from: M part: [k] order: [o] frame: 3 funcs: {ma: (avg v) ms: (sum v) mn: (min v) mx: (max v) n: (count v)}})) +(at MW 'ma) -- [10.0 15.0 100.0 20.0 150.0 30.0] +(at MW 'ms) -- [10 30 100 60 300 90] +(at MW 'mn) -- [10 10 100 10 100 20] +(at MW 'mx) -- [10 20 100 30 200 40] +(at MW 'n) -- [1 2 1 3 2 3] + ;; ── fby shape: rows equal to their group's min (keeps ties) ───────────────── ;; min over k: a→2 (only row v=2), b→4 (rows v=4,4 both) → 3 rows total (count (select {from: (window {from: T part: [k] funcs: {mn: (min v)}}) where: (== v mn)})) -- 3 @@ -43,6 +53,8 @@ (window {from: T funcs: {x: (median v)}}) !- domain ;; (b) bad frame symbol (window {from: T frame: 'bogus funcs: {mn: (min v)}}) !- domain +;; (b2) trailing row count must be positive +(window {from: T frame: 0 funcs: {mn: (min v)}}) !- range ;; (c) first without order: (window {from: T part: [k] funcs: {f: (first v)}}) !- domain ;; (d) missing funcs: diff --git a/test/rfl/system/mem_ts.rfl b/test/rfl/system/mem_ts.rfl new file mode 100644 index 00000000..ae120851 --- /dev/null +++ b/test/rfl/system/mem_ts.rfl @@ -0,0 +1,29 @@ +;; .mem.ts evaluates exactly once and returns named, query-scoped statistics. +(set _mts (.mem.ts (til 10000))) +(count (at _mts 'result)) -- 10000 +(> (at _mts 'time-ns) 0) -- true +(>= (at _mts 'memory-bytes) 0) -- true +(> (at _mts 'allocated-bytes) 0) -- true +(>= (at _mts 'allocated-bytes) (at _mts 'peak-live-bytes)) -- true +(> (at _mts 'peak-live-bytes) 0) -- true +(> (at _mts 'result-bytes) 0) -- true +(> (at _mts 'alloc-count) 0) -- true +(>= (at _mts 'free-count) 0) -- true +(>= (at _mts 'workers) 0) -- true +(>= (at _mts 'worker-busy-ns) 0) -- true +(>= (at _mts 'parallelism) 0.0) -- true + +;; The result sizing is performed after the trace and agrees with the public +;; value-level operation. +(== (at _mts 'result-bytes) (.mem.objsize (at _mts 'result))) -- true + +;; A parallel aggregation must report allocation churn and return its value. +(set _mtsp (.mem.ts (sum (til 1000000)))) +(at _mtsp 'result) -- 499999500000 +(> (at _mtsp 'allocated-bytes) 0) -- true +(> (at _mtsp 'workers) 0) -- true +(> (at _mtsp 'worker-busy-ns) 0) -- true + +;; Measurement scopes are process-global and cannot nest. +(try (.mem.ts (.mem.ts (til 10))) (fn [e] 'nested-rejected)) -- 'nested-rejected +(try (.mem.ts) (fn [e] 'arity-rejected)) -- 'arity-rejected diff --git a/test/rfl/system/objsize.rfl b/test/rfl/system/objsize.rfl new file mode 100644 index 00000000..f5f3c93f --- /dev/null +++ b/test/rfl/system/objsize.rfl @@ -0,0 +1,15 @@ +;; .mem.objsize reports logical retained bytes, not serialized size or buddy +;; allocator capacity. Every ray_t contributes its 32-byte header. + +(.mem.objsize 42) -- 32 +(.mem.objsize [1 2 3]) -- 56 + +;; Shared children are counted once: LIST header + two pointer slots + vector. +(set _objsize_shared [1 2 3]) +(.mem.objsize (list _objsize_shared _objsize_shared)) -- 104 + +;; A backing index and its child vectors are part of the retained graph. +(> (.mem.objsize (.attr.set 'parted [1 1 2 2])) (.mem.objsize [1 1 2 2])) -- true + +;; The allocator-maintenance builtin remains callable from Rayfall. +(.sys.gc) -- 0 diff --git a/test/test_idx_route.c b/test/test_idx_route.c index 69dbe1ba..ed205f80 100644 --- a/test/test_idx_route.c +++ b/test/test_idx_route.c @@ -105,6 +105,27 @@ static int attach_hash_to_col(ray_t* tbl, const char* name) { return 0; } +/* Attach a part index using the same retain/write-back cycle. */ +static int attach_part_to_col(ray_t* tbl, const char* name) { + int64_t sym_id = ray_sym_intern(name, (int64_t)strlen(name)); + int64_t ncols = ray_table_ncols(tbl); + int64_t slot = -1; + for (int64_t i = 0; i < ncols; i++) { + if (ray_table_col_name(tbl, i) == sym_id) { slot = i; break; } + } + if (slot < 0) return -1; + + ray_t* col = ray_table_get_col_idx(tbl, slot); + if (!col || RAY_IS_ERR(col)) return -1; + ray_t* w = col; + ray_retain(w); + ray_t* r = ray_index_attach_part(&w); + if (!r || RAY_IS_ERR(r)) { ray_release(w); return -1; } + ray_table_set_col_idx(tbl, slot, w); + ray_release(w); + return 0; +} + /* Attach a zone index to the named column in `tbl`. * Same retain/write-back/release cycle as attach_hash_to_col: * ray_table_set_col_idx retains `w` before releasing the old pointer, @@ -1083,6 +1104,120 @@ static test_result_t test_range_sorted_all_segments(void) { PASS(); } +/* sorted_marker_dense_range: physically sorted k=i with only the verified + * sorted marker (no sort index). (>= k 100000) selects 62% of the table — + * deliberately far beyond the shuffled sort-index selectivity guard. The + * direct path must still hit because the result is one contiguous row span. */ +static ray_op_t* pred_ge_100000(ray_graph_t* g) { + return ray_ge(g, ray_scan(g, "k"), ray_const_i64(g, 100000)); +} + +static test_result_t test_sorted_marker_dense_range(void) { + ray_heap_init(); + (void)ray_sym_init(); + + const int64_t N = 262144; + ray_t* kv = ray_vec_new(RAY_I64, N); + ray_t* vv = ray_vec_new(RAY_I64, N); + TEST_ASSERT_FALSE(RAY_IS_ERR(kv)); + TEST_ASSERT_FALSE(RAY_IS_ERR(vv)); + kv->len = vv->len = N; + int64_t* kd = (int64_t*)ray_data(kv); + int64_t* vd = (int64_t*)ray_data(vv); + for (int64_t i = 0; i < N; i++) kd[i] = vd[i] = i; + kv->attrs |= RAY_ATTR_SORTED; /* fixture is verified by construction */ + + ray_t* tbl = ray_table_new(2); + tbl = ray_table_add_col(tbl, ray_sym_intern("k", 1), kv); + tbl = ray_table_add_col(tbl, ray_sym_intern("v", 1), vv); + ray_release(kv); + ray_release(vv); + TEST_ASSERT_FALSE(RAY_IS_ERR(tbl)); + TEST_ASSERT_EQ_I((int)ray_index_kind(ray_table_get_col(tbl, + ray_sym_intern("k", 1))), RAY_IDX_NONE); + + uint64_t cons_before = ray_idx_consults[IDX_SITE_FILTER_RANGE]; + uint64_t hits_before = ray_idx_hits[IDX_SITE_FILTER_RANGE]; + ray_t* r = run_filter(tbl, pred_ge_100000); + TEST_ASSERT_FALSE(RAY_IS_ERR(r)); + TEST_ASSERT_EQ_I(ray_table_nrows(r), N - 100000); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_consults[IDX_SITE_FILTER_RANGE] - cons_before), 1); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_hits[IDX_SITE_FILTER_RANGE] - hits_before), 1); + + ray_t* rv = ray_table_get_col(r, ray_sym_intern("v", 1)); + TEST_ASSERT_FALSE(!rv || RAY_IS_ERR(rv)); + TEST_ASSERT_EQ_I(ray_vec_get_i64(rv, 0), 100000); + TEST_ASSERT_EQ_I(ray_vec_get_i64(rv, ray_len(rv) - 1), N - 1); + + ray_release(r); + ray_release(tbl); + ray_sym_destroy(); + ray_heap_destroy(); + PASS(); +} + +/* Two-sided ranges are split into chained FILTER nodes by the optimizer. + * Both bounds must route and intersect as rowsels; the outer bound must not + * fall back to scanning the inner selection. */ +static test_result_t test_sorted_marker_chained_range(void) { + ray_heap_init(); + (void)ray_sym_init(); + + const int64_t N = 262144; + ray_t* kv = ray_vec_new(RAY_I64, N); + ray_t* vv = ray_vec_new(RAY_I64, N); + TEST_ASSERT_FALSE(RAY_IS_ERR(kv)); + TEST_ASSERT_FALSE(RAY_IS_ERR(vv)); + kv->len = vv->len = N; + int64_t* kd = (int64_t*)ray_data(kv); + int64_t* vd = (int64_t*)ray_data(vv); + for (int64_t i = 0; i < N; i++) kd[i] = vd[i] = i; + kv->attrs |= RAY_ATTR_SORTED; + ray_t* tbl = ray_table_new(2); + tbl = ray_table_add_col(tbl, ray_sym_intern("k", 1), kv); + tbl = ray_table_add_col(tbl, ray_sym_intern("v", 1), vv); + ray_release(kv); ray_release(vv); + + ray_graph_t* g = ray_graph_new(tbl); + ray_op_t* base = ray_const_table(g, tbl); + ray_op_t* ge = ray_ge(g, ray_scan(g, "k"), ray_const_i64(g, 100000)); + ray_op_t* inner = ray_filter(g, base, ge); + ray_op_t* lt = ray_lt(g, ray_scan(g, "k"), ray_const_i64(g, 100100)); + ray_op_t* outer = ray_filter(g, inner, lt); + uint64_t hits_before = ray_idx_hits[IDX_SITE_FILTER_RANGE]; + ray_t* r = ray_execute(g, outer); + TEST_ASSERT_FALSE(RAY_IS_ERR(r)); + TEST_ASSERT_EQ_I(ray_table_nrows(r), 100); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_hits[IDX_SITE_FILTER_RANGE] - hits_before), 2); + ray_t* rv = ray_table_get_col(r, ray_sym_intern("v", 1)); + TEST_ASSERT_EQ_I(ray_vec_get_i64(rv, 0), 100000); + TEST_ASSERT_EQ_I(ray_vec_get_i64(rv, 99), 100099); + + ray_release(r); + ray_graph_free(g); + ray_release(tbl); + ray_sym_destroy(); + ray_heap_destroy(); + PASS(); +} + +/* The direct route trusts RAY_ATTR_SORTED, so every payload mutation must + * clear it even when the vector has no attached accelerator index. */ +static test_result_t test_sorted_marker_mutation_invalidates(void) { + ray_heap_init(); + int64_t data[] = { 1, 2, 3 }; + ray_t* v = ray_vec_from_raw(RAY_I64, data, 3); + TEST_ASSERT_FALSE(RAY_IS_ERR(v)); + v->attrs |= RAY_ATTR_SORTED; + int64_t replacement = 0; + v = ray_vec_set(v, 1, &replacement); + TEST_ASSERT_FALSE(RAY_IS_ERR(v)); + TEST_ASSERT_FALSE(ray_attr_is_sorted(v)); + ray_release(v); + ray_heap_destroy(); + PASS(); +} + /* (== k 7) on the dense-dup fixture below. */ static ray_op_t* pred_eq_7_dups(ray_graph_t* g) { return ray_eq(g, ray_scan(g, "k"), ray_const_i64(g, 7)); @@ -1175,6 +1310,69 @@ static ray_op_t* pred_in_100_5(ray_graph_t* g) { return ray_in(g, ray_scan(g, "k"), set_op); } +static ray_t* make_part_sym_table(void) { + (void)ray_sym_init(); + const int64_t n = 4096; + int64_t block_ids[] = { + ray_sym_intern("part_z", 6), + ray_sym_intern("part_a", 6), + ray_sym_intern("part_m", 6), + ray_sym_intern("part_b", 6) + }; + int64_t ends[] = {700, 2000, 2900, 4096}; + ray_t* syms = ray_sym_vec_new(RAY_SYM_W64, n); + ray_t* vals = ray_vec_new(RAY_I64, n); + if (!syms || !vals || RAY_IS_ERR(syms) || RAY_IS_ERR(vals)) { + if (syms && !RAY_IS_ERR(syms)) ray_release(syms); + if (vals && !RAY_IS_ERR(vals)) ray_release(vals); + return ray_error("oom", NULL); + } + syms->len = vals->len = n; + int64_t p = 0; + for (int64_t i = 0; i < n; i++) { + while (i >= ends[p]) p++; + ray_write_sym(ray_data(syms), i, (uint64_t)block_ids[p], + RAY_SYM, syms->attrs); + ((int64_t*)ray_data(vals))[i] = i; + } + ray_t* tbl = ray_table_new(2); + tbl = ray_table_add_col(tbl, ray_sym_intern("sym", 3), syms); + tbl = ray_table_add_col(tbl, ray_sym_intern("v", 1), vals); + ray_release(syms); + ray_release(vals); + return tbl; +} + +/* Select two non-adjacent, dense partitions plus an absent symbol. */ +static ray_op_t* pred_in_part_syms(ray_graph_t* g) { + int64_t ids[] = { + ray_sym_intern("part_a", 6), + ray_sym_intern("part_b", 6), + ray_sym_intern("part_absent", 11) + }; + ray_t* set = ray_sym_vec_new(RAY_SYM_W64, 3); + if (!set || RAY_IS_ERR(set)) return NULL; + for (int64_t i = 0; i < 3; i++) + set = ray_vec_append(set, &ids[i]); + ray_op_t* set_op = ray_const_vec(g, set); + ray_release(set); + return ray_in(g, ray_scan(g, "sym"), set_op); +} + +static ray_op_t* pred_eq_part_a(ray_graph_t* g) { + ray_t* lit = ray_sym(ray_sym_intern("part_a", 6)); + ray_op_t* c = ray_const_atom(g, lit); + ray_release(lit); + return ray_eq(g, ray_scan(g, "sym"), c); +} + +static ray_op_t* pred_eq_part_absent(ray_graph_t* g) { + ray_t* lit = ray_sym(ray_sym_intern("part_absent", 11)); + ray_op_t* c = ray_const_atom(g, lit); + ray_release(lit); + return ray_eq(g, ray_scan(g, "sym"), c); +} + /* ─── IN tests ───────────────────────────────────────────────────── */ /* in_hash: hash on k; FILTER(IN(k, [9 5])) → 4 rows; IDX_SITE_IN @@ -1212,6 +1410,96 @@ static test_result_t test_in_hash(void) { PASS(); } +/* A part index should keep dense unions as physical spans. The chosen + * blocks select 2496/4096 rows, which deliberately exceeds the hash route's + * density guard, and exercise both full and partial morsels. */ +static test_result_t test_in_part_sym_dense(void) { + ray_heap_init(); + ray_t* tbl_a = make_part_sym_table(); + TEST_ASSERT_FALSE(RAY_IS_ERR(tbl_a)); + TEST_ASSERT(attach_part_to_col(tbl_a, "sym") == 0, + "attach part (in_part_sym_dense)"); + + uint64_t cons_before = ray_idx_consults[IDX_SITE_IN]; + uint64_t hits_before = ray_idx_hits[IDX_SITE_IN]; + ray_t* ra = run_filter(tbl_a, pred_in_part_syms); + TEST_ASSERT_FALSE(RAY_IS_ERR(ra)); + TEST_ASSERT_EQ_I(ray_table_nrows(ra), 2496); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_consults[IDX_SITE_IN] - cons_before), 1); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_hits[IDX_SITE_IN] - hits_before), 1); + + ray_t* tbl_b = make_part_sym_table(); + TEST_ASSERT_FALSE(RAY_IS_ERR(tbl_b)); + ray_t* rb = run_filter(tbl_b, pred_in_part_syms); + TEST_ASSERT_FALSE(RAY_IS_ERR(rb)); + TEST_ASSERT_EQ_I(ray_table_nrows(rb), 2496); + TEST_ASSERT(v_cols_equal(ra, rb), + "v column mismatch between parted IN and scan"); + + ray_release(ra); ray_release(rb); + ray_release(tbl_a); ray_release(tbl_b); + ray_sym_destroy(); + ray_heap_destroy(); + PASS(); +} + +/* Equality on a dense part must use its physical span directly rather than + * tripping the hash route's density guard. part_a occupies rows [700,2000). */ +static test_result_t test_eq_part_sym_dense(void) { + ray_heap_init(); + ray_t* tbl_a = make_part_sym_table(); + TEST_ASSERT_FALSE(RAY_IS_ERR(tbl_a)); + TEST_ASSERT(attach_part_to_col(tbl_a, "sym") == 0, + "attach part (eq_part_sym_dense)"); + + uint64_t cons_before = ray_idx_consults[IDX_SITE_FILTER_PART]; + uint64_t hits_before = ray_idx_hits[IDX_SITE_FILTER_PART]; + ray_t* ra = run_filter(tbl_a, pred_eq_part_a); + TEST_ASSERT_FALSE(RAY_IS_ERR(ra)); + TEST_ASSERT_EQ_I(ray_table_nrows(ra), 1300); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_consults[IDX_SITE_FILTER_PART] - cons_before), 1); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_hits[IDX_SITE_FILTER_PART] - hits_before), 1); + + ray_t* tbl_b = make_part_sym_table(); + TEST_ASSERT_FALSE(RAY_IS_ERR(tbl_b)); + ray_t* rb = run_filter(tbl_b, pred_eq_part_a); + TEST_ASSERT_FALSE(RAY_IS_ERR(rb)); + TEST_ASSERT_EQ_I(ray_table_nrows(rb), 1300); + TEST_ASSERT(v_cols_equal(ra, rb), + "v column mismatch between parted EQ and scan"); + + ray_release(ra); ray_release(rb); + ray_release(tbl_a); ray_release(tbl_b); + ray_sym_destroy(); + ray_heap_destroy(); + PASS(); +} + +/* A symbol can exist in the shared domain without having a physical part. + * That is still a successful index proof yielding a valid empty rowsel. */ +static test_result_t test_eq_part_sym_absent(void) { + ray_heap_init(); + ray_t* tbl = make_part_sym_table(); + TEST_ASSERT_FALSE(RAY_IS_ERR(tbl)); + (void)ray_sym_intern("part_absent", 11); + TEST_ASSERT(attach_part_to_col(tbl, "sym") == 0, + "attach part (eq_part_sym_absent)"); + + uint64_t cons_before = ray_idx_consults[IDX_SITE_FILTER_PART]; + uint64_t hits_before = ray_idx_hits[IDX_SITE_FILTER_PART]; + ray_t* r = run_filter(tbl, pred_eq_part_absent); + TEST_ASSERT_FALSE(RAY_IS_ERR(r)); + TEST_ASSERT_EQ_I(ray_table_nrows(r), 0); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_consults[IDX_SITE_FILTER_PART] - cons_before), 1); + TEST_ASSERT_EQ_I((int64_t)(ray_idx_hits[IDX_SITE_FILTER_PART] - hits_before), 1); + + ray_release(r); + ray_release(tbl); + ray_sym_destroy(); + ray_heap_destroy(); + PASS(); +} + /* in_dup_set: set [9 9 5] → same 4 rows, no duplicate row ids. */ static test_result_t test_in_dup_set(void) { ray_heap_init(); @@ -2542,8 +2830,14 @@ const test_entry_t idx_route_entries[] = { { "idx_route/range_guard", test_range_guard, NULL, NULL }, { "idx_route/range_f64", test_range_f64, NULL, NULL }, { "idx_route/range_sorted_all_segments", test_range_sorted_all_segments, NULL, NULL }, + { "idx_route/sorted_marker_dense_range", test_sorted_marker_dense_range, NULL, NULL }, + { "idx_route/sorted_marker_chained_range", test_sorted_marker_chained_range, NULL, NULL }, + { "idx_route/sorted_marker_mutation_invalidates", test_sorted_marker_mutation_invalidates, NULL, NULL }, { "idx_route/hash_eq_dense_dups", test_hash_eq_dense_dups, NULL, NULL }, { "idx_route/in_hash", test_in_hash, NULL, NULL }, + { "idx_route/in_part_sym_dense", test_in_part_sym_dense, NULL, NULL }, + { "idx_route/eq_part_sym_dense", test_eq_part_sym_dense, NULL, NULL }, + { "idx_route/eq_part_sym_absent", test_eq_part_sym_absent, NULL, NULL }, { "idx_route/in_dup_set", test_in_dup_set, NULL, NULL }, { "idx_route/in_absent_elems", test_in_absent_elems, NULL, NULL }, { "idx_route/in_float_col_falls_back", test_in_float_col_falls_back, NULL, NULL }, diff --git a/test/test_window.c b/test/test_window.c index ef684588..2965ba9e 100644 --- a/test/test_window.c +++ b/test/test_window.c @@ -4196,13 +4196,9 @@ static test_result_t test_window_i32_plain_order_key(void) { PASS(); } -/* Review 2.10 sub-item 4: an unsupported window frame must be a LOUD error, - * not silently degraded to the running frame. The compute path honors only - * the whole partition (UNBOUNDED PRECEDING .. UNBOUNDED FOLLOWING) and the - * running frame (ROWS, UNBOUNDED PRECEDING .. CURRENT ROW). Any other frame - * (N_PRECEDING / N_FOLLOWING bounds, RANGE-mode CURRENT ROW, a non-unbounded - * start) was previously accepted and IGNORED — yielding the wrong window. - * exec_window now rejects these. */ +/* Unsupported frames must be a LOUD error, not silently degraded to the + * running frame. Whole, running, and fixed trailing ROWS frames are honored; + * other N_FOLLOWING/RANGE/non-trailing shapes remain unsupported. */ static test_result_t test_window_unsupported_frame_errors(void) { ray_heap_init(); (void)ray_sym_init(); @@ -4220,8 +4216,6 @@ static test_result_t test_window_unsupported_frame_errors(void) { /* Helper: run a SUM window with the given frame and expect an error. */ struct { uint8_t ftype, fstart, fend; int64_t sn, en; } bad[] = { - /* ROWS BETWEEN 1 PRECEDING AND CURRENT ROW — sliding, NOT honored */ - { RAY_FRAME_ROWS, RAY_BOUND_N_PRECEDING, RAY_BOUND_CURRENT_ROW, 1, 0 }, /* ROWS BETWEEN UNBOUNDED PRECEDING AND 1 FOLLOWING — NOT honored */ { RAY_FRAME_ROWS, RAY_BOUND_UNBOUNDED_PRECEDING, RAY_BOUND_N_FOLLOWING, 0, 1 }, /* RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW — RANGE not honored */ @@ -4247,7 +4241,33 @@ static test_result_t test_window_unsupported_frame_errors(void) { ray_release(result); ray_graph_free(g); } - /* Sanity: the two HONORED frames still succeed. */ + /* ROWS BETWEEN 1 PRECEDING AND CURRENT ROW is a real sliding frame. */ + { + ray_graph_t* g = ray_graph_new(tbl); + ray_op_t* tbl_op = ray_const_table(g, tbl); + ray_op_t* g_op = ray_scan(g, "g"); + ray_op_t* v_op = ray_scan(g, "v"); + ray_op_t* parts[] = { g_op }; + uint8_t kinds[] = { RAY_WIN_SUM }; + ray_op_t* fins[] = { v_op }; + int64_t params[] = { 0 }; + ray_op_t* w = ray_window_op(g, tbl_op, parts, 1, NULL, NULL, 0, + kinds, fins, params, 1, RAY_FRAME_ROWS, + RAY_BOUND_N_PRECEDING, + RAY_BOUND_CURRENT_ROW, 1, 0); + ray_t* result = ray_execute(g, w); + TEST_ASSERT_FALSE(RAY_IS_ERR(result)); + ray_t* sum_col = ray_table_get_col_idx(result, 2); + TEST_ASSERT_NOT_NULL(sum_col); + int64_t* sums = (int64_t*)ray_data(sum_col); + TEST_ASSERT_EQ_I(sums[0], 10); + TEST_ASSERT_EQ_I(sums[1], 30); + TEST_ASSERT_EQ_I(sums[2], 50); + TEST_ASSERT_EQ_I(sums[3], 70); + ray_release(result); ray_graph_free(g); + } + + /* Sanity: the running frame still succeeds. */ { ray_graph_t* g = ray_graph_new(tbl); ray_op_t* tbl_op = ray_const_table(g, tbl);