Skip to content

fix(avm): avoid data race on shared interaction selector writes#24456

Merged
fcarreiro merged 1 commit into
nextfrom
fc/avm-safer-interactions
Jul 2, 2026
Merged

fix(avm): avoid data race on shared interaction selector writes#24456
fcarreiro merged 1 commit into
nextfrom
fc/avm-safer-interactions

Conversation

@fcarreiro

@fcarreiro fcarreiro commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What was wrong

The interactions tracegen phase runs every lookup/permutation job concurrently: AvmTraceGenHelper::fill_trace_interactions concatenates all builders' jobs and dispatches them with parallel_for. Lookups whose fine-grained destination selector is a shared column (DST_SELECTOR != outer_dst_selector) can resolve to the same dst_row from different jobs, so multiple threads write the same (selector, row) cell at once.

The previous code did a guarded read-modify-write on that shared cell:

if (DST_SELECTOR != outer_dst_selector && trace.get(DST_SELECTOR, dst_row) != 1) {
    trace.set(DST_SELECTOR, dst_row, 1);
}

Both the get and the non-atomic 32-byte set race against the other threads' writes to the same cell — a data race, i.e. undefined behavior. In practice it produced the right value (every writer stores 1), but it is still UB.

The fix

Add an opt-in use_atomic_limbs flag to TraceContainer::set. When set, the field's four 64-bit limbs are written with relaxed atomic stores — four plain movq on x86-64, no lock and no libatomic call. (A whole-field std::atomic_ref<FF> is not an option on the hot path: at 32 bytes it exceeds the hardware lock-free width and falls back to a locked libatomic call.)

The shared selector write now uses it and drops the guard read:

if (DST_SELECTOR != outer_dst_selector) {
    trace.set(DST_SELECTOR, dst_row, 1, /*use_atomic_limbs=*/true);
}

The write is an unconditional, idempotent atomic store of 1. Because every concurrent writer stores the same value, per-limb atomicity is sufficient — no torn value is possible — so this is data-race-free without the cost of whole-field atomicity. The default (non-atomic) set hot path is untouched and keeps its sparse-column "zero = absent" fast path.

Performance (this PR vs baseline)

Mega bulk AVM tx, full proving, HARDWARE_CONCURRENCY=16. Tracegen stage timings, median of runs (baseline n=3, this PR n=6):

Stage baseline (next) this PR
tracegen traces 1,536 ms 1,537 ms
tracegen interactions 350 ms 329 ms
tracegen all 1,917 ms 1,936 ms

No measurable cost — the safety fix is free. The per-limb atomic only fires on the shared selector writes in the interactions stage; the traces stage is byte-for-byte the same hot path as baseline.

Fixes https://linear.app/aztec-labs/issue/AVM-276/interactions-tracegen-does-ub-write-to-destination-selector

The interactions tracegen phase runs every lookup/permutation job via parallel_for (fill_trace_interactions). Lookups that share a destination-selector column can write the same (selector, row) cell from multiple threads, so the previous guarded read-modify-write (get(...) != 1 then set(...)) on that shared cell is a data race (undefined behavior).

Add an opt-in use_atomic_limbs flag to TraceContainer::set that stores the field's four limbs with relaxed atomic writes (4 plain movq on x86-64, no lock -- unlike a whole-field std::atomic_ref<FF>, whose 32 bytes exceed the lock-free width). Use it for the shared selector write and drop the guard read: the write is now an unconditional idempotent atomic store of 1. Because every writer stores the same value, per-limb atomicity is sufficient (no torn value is possible), and the hot non-atomic set() path is unchanged.
if (BB_UNLIKELY(use_atomic_limbs)) {
store_per_limb(shard->rows[offset], zero);
} else {
shard->rows[offset] = FF::zero();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use the constant zero.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm yeah, it probably would make sense to use either zero or FF::zero for both. I'll leave it like this because CI is already running but otherwise I'd probably use FF::zero for both (since it gives the compiler more information than a reference).


// Set the fine grained inner selector if it's not already one.
if (LookupSettings::DST_SELECTOR != this->outer_dst_selector &&
trace.get(LookupSettings::DST_SELECTOR, dst_row) != 1) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you remove the second boolean condition because it is not worth it performance-wise or because it would not be thread-safe?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not safe even if it's a read.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thanks for the pointer.

@fcarreiro fcarreiro added this pull request to the merge queue Jul 2, 2026
Merged via the queue into next with commit e3f3a14 Jul 2, 2026
23 checks passed
@fcarreiro fcarreiro deleted the fc/avm-safer-interactions branch July 2, 2026 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants