Performance [P3]: Index sandbox activity overlap validation - #205
Merged
berndverst merged 4 commits intoJul 27, 2026
Conversation
`build_sandbox_worker_profiles()` validated activity ownership by scanning every previously registered activity with `activities_overlap()` for each new activity, making sandbox worker profile construction O(n^2) in the number of registered activities. Replace the linear scan with `_ActivityOwnerIndex`, which keys owners on the normalized (casefolded) activity name and on the exact (name, version) pair, with `None` representing an unversioned registration. Overlap semantics are unchanged: an unversioned activity overlaps every version of the same name, identical name+version pairs overlap, and the same name with different explicit versions does not. Registration order is retained so the reported conflicting profile is the same one a linear scan would report, keeping the exception type and message byte-identical. No public API or observable behavior change, so no changelog entry. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1bb48ba8-473f-4632-9551-9c8d3c6615de
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes sandbox worker profile construction in durabletask.azuremanaged by replacing a quadratic activity-overlap validation scan with a private indexed lookup structure, preserving existing overlap semantics and error reporting while improving cold-start control-plane performance for large activity catalogs.
Changes:
- Replaced the linear
activities_overlap()scan inbuild_sandbox_worker_profiles()with_ActivityOwnerIndexfor O(1) per-activity overlap lookups. - Added
_ActivityOwnerSlottracking to preserve “first conflicting owner” ordering to keep the raisedValueErrormessage identical to the legacy behavior. - Expanded unit tests to assert exact overlap error messages and added a differential test to ensure the index matches the legacy linear scan across shuffled registrations.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| durabletask-azuremanaged/durabletask/azuremanaged/preview/sandboxes/profile_builder.py | Introduces _ActivityOwnerIndex/_ActivityOwnerSlot and switches overlap validation to indexed lookups while keeping conflict semantics and messaging stable. |
| tests/durabletask-azuremanaged/test_sandboxes_extension.py | Adds targeted overlap regression tests (including exact-message assertions) plus a differential test ensuring indexed behavior matches the legacy scan. |
…box-activity-overlap-validatio
added 2 commits
July 27, 2026 10:52
…box-activity-overlap-validatio # Conflicts: # tests/durabletask-azuremanaged/test_sandboxes_extension.py
…box-activity-overlap-validatio
andystaples
approved these changes
Jul 27, 2026
berndverst
deleted the
berndverst-index-sandbox-activity-overlap-validatio
branch
July 27, 2026 19:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
build_sandbox_worker_profiles()validated activity ownership with a linearnext(...)scan over every previously accumulated activity, callingactivities_overlap()for each candidate. That made sandbox worker profileconstruction O(n²) in the number of registered activities — startup-only
control-plane work, but it degrades badly for large generated or versioned
sandbox catalogs.
This replaces the scan with
_ActivityOwnerIndex, a private index inprofile_builder.pythat gives O(1) overlap lookups per activity:_by_name— keyed on the casefolded activity name, used when the incomingactivity is unversioned (an unversioned activity overlaps all versions of
the same name).
_by_name_and_version— keyed on(casefolded name, version)whereNonerepresents an unversioned registration, used when the incoming activity has
an explicit version (it overlaps the same version plus any unversioned
registration of that name).
Semantics preserved exactly
The index mirrors
activities_overlap()byte-for-byte, including itsnormalization rules:
Registration order is tracked (
_ActivityOwnerSlotretains the first owner plusthe first owner that differs from it) so the profile named in the error is the
same one a linear scan would have reported. The exception type (
ValueError)and message text are unchanged:
Profile ordering in the returned list, the same-profile overlap allowance, and
the truthiness check on the conflicting profile id are all unchanged.
activities_overlap()remains exported fromhelpers.py— no public APIsignature changes, no renames, no removed symbols, no changed import paths.
This is an internal performance refactor with no observable behavior change,
so per repo convention there is no changelog entry.
Fixes #190
Verification
Measured improvement (synthetic catalogs, legacy scan vs. index):
Legacy timings scale ~16x per 4x input (quadratic); indexed timings scale
linearly.
Tests — 4 new tests in
tests/durabletask-azuremanaged/test_sandboxes_extension.py:..._rejects_versioned_activity_overlapping_unversioned_owner— unversionedowner vs. versioned candidate, asserts the exact conflict message.
..._rejects_unversioned_activity_overlapping_versioned_owner— versionedowner vs. unversioned candidate, asserts the exact conflict message.
..._rejects_identical_activity_versions— same name + same explicit version,asserts the exact conflict message.
test_activity_owner_index_matches_linear_overlap_scan— differential testthat replays 48 registrations across 25 seeded shuffles and asserts the index
agrees with the original
activities_overlap()linear scan at every step.The existing overlap test also gained an exact-message assertion. The
pre-existing same-name/different-version allowance test still passes unchanged.
The differential test was mutation-checked: a variant of the index that drops
its second-owner tracking is caught on the first seed, confirming the test has
teeth.
Commands run (session-local venv, so the shared interpreter was untouched):
python -m pytest tests/durabletask-azuremanaged/test_sandboxes_extension.py tests/durabletask-azuremanaged/test_durabletask_grpc_interceptor.py -q→ 46 passed (42 before this change + 4 new)python -m pytest tests/durabletask -q --ignore-glob="*_e2e.py"→ 681 passed, 7 skippedflake8 .indurabletask-azuremanaged/andtests/durabletask-azuremanaged/(matching CI) → cleanpyrightwith the repo's strictpyrightconfig.json→ 0 diagnostics inprofile_builder.pyBaseline was captured before the change; the only difference was a flaky
pre-existing core test (
test_sync_client_recreate_cooldown_prevents_immediate_repeated_recreation)that passed on re-run. Remaining
tests/durabletask-azuremanagedmodules requirea live Durable Task Scheduler endpoint (
TASKHUBunset) and are out of scope here.