Performance [P2]: Reuse sandbox registration transport on retries

[berndverst]

Summary

SandboxWorker._run_sandbox_registration_loop() constructed a brand new
SandboxActivitiesGrpcTransport on every retry attempt. Each construction created a
new gRPC channel, a new stub, and a new DTSDefaultClientInterceptorImpl, which in
turn acquires an Entra token in its constructor. A transient registration failure
therefore cost a full channel setup plus a token acquisition per retry, amplifying
startup and recovery latency exactly when the service was already degraded.

The registration loop now builds the transport once and reuses it across retriable
stream failures, closing it once when the loop exits so channels are not leaked.

What changed

• durabletask-azuremanaged/durabletask/azuremanaged/preview/sandboxes/worker.py
  • The transport is created lazily on the first attempt and reused for subsequent
    attempts, including the reconnect path where the server ends the registration
    stream while the worker is still running.
  • Added _requires_new_registration_transport(), which replaces the transport only
    when a failure leaves the channel unusable: a channel that has been shut down
    (CANCELLED with Channel closed!) or any non-gRPC failure, such as an error
    raised while the channel itself is being created. gRPC channels reconnect on their
    own after transient RPC failures, so those reuse the existing transport.
  • Added _close_sandbox_registration_transport(), which closes the transport
    defensively and is invoked from a finally block covering the whole loop.

Everything else is unchanged: the exponential backoff schedule, responsiveness to the
stop signal, thread join semantics, log messages, and the exception types and terminal
handling for non-retriable failures.

No public API, signature, import path, or observable behavior changes, so per the
repository changelog policy this internal performance fix does not get a changelog
entry.

Verification

• New tests in tests/durabletask-azuremanaged/test_sandboxes_extension.py:
  • test_sandbox_registration_reuses_one_transport_across_retriable_failures uses a
    fake transport factory to assert that 3 retriable UNAVAILABLE failures result in
    exactly 1 transport construction and 1 close, and that the backoff windows
    are still 1.0, 2.0, 4.0.
  • test_sandbox_registration_rebuilds_transport_after_channel_shutdown asserts a
    shut-down channel still produces a replacement transport, and that the discarded
    transport is closed.
  • test_sandbox_registration_loop_exits_promptly_on_stop_signal runs the loop on a
    thread and asserts it exits promptly once the stop event is set, with the transport
    closed exactly once.
  • test_sandbox_registration_recreates_transport_only_when_channel_is_unusable
    covers the reuse-versus-replace predicate directly.
• The reuse test fails against the pre-fix implementation (4 transports instead of 1),
  confirming it guards the regression.
• pytest tests/durabletask-azuremanaged/test_sandboxes_extension.py tests/durabletask-azuremanaged/test_durabletask_grpc_interceptor.py -q -> 46 passed (42 before, plus the 4 new tests).
• pytest tests/durabletask -q --ignore-glob="*_e2e.py" -> 681 passed, 7 skipped.
• flake8 clean on both changed files.
• pyright reports 0 errors for the changed source file under the repository's strict
  configuration.

Out of scope

The issue's cold-start note about sharing channel or authentication state with the
parent DurableTaskSchedulerWorker is intentionally not addressed here; it overlaps
with the deferred token acquisition work tracked in #187.

Fixes #188

[Copilot]

Pull request overview

This PR improves the Azure Managed sandbox worker’s resiliency and startup/recovery performance by reusing the sandbox registration gRPC transport across retriable registration failures, avoiding repeated channel + interceptor + token-acquisition setup during retry loops.

Changes:

• Reworked SandboxWorker._run_sandbox_registration_loop() to lazily create and then reuse a single SandboxActivitiesGrpcTransport across retriable stream failures, closing it once when the loop exits.
• Added _requires_new_registration_transport() to decide when a failure implies the underlying channel/transport is no longer reusable (e.g., channel shutdown) and should be recreated.
• Added targeted tests validating reuse vs rebuild behavior, correct backoff progression, and prompt loop shutdown on stop signal.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
durabletask-azuremanaged/durabletask/azuremanaged/preview/sandboxes/worker.py	Reuses registration transport across retries; adds transport-close helper and “reuse vs rebuild” predicate.
tests/durabletask-azuremanaged/test_sandboxes_extension.py	Adds regression tests covering transport reuse, rebuild-on-channel-shutdown, stop responsiveness, and predicate behavior.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.