ChannelDbConnectionPool replace connection - #4429
Conversation
There was a problem hiding this comment.
Pull request overview
Implements connection replacement support for the channel-based connection pool, including a new slot-level atomic replace operation and expanded unit test coverage around replacement behavior.
Changes:
- Implement
ChannelDbConnectionPool.ReplaceConnection(...)and introduce internal replacement acquisition logic (idle-preferred, otherwise create new). - Add
ConnectionPoolSlots.TryReplace(...)to atomically swap a connection within an existing reserved slot. - Update/extend unit tests, including a new dedicated
ChannelDbConnectionPoolReplaceConnectionTestsuite and an updated existing replacement test.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ConnectionPoolSlotsTest.cs | Adds new tests validating ConnectionPoolSlots.TryReplace behaviors. |
| src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTest.cs | Updates the existing replace-connection test to exercise the new implementation. |
| src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolReplaceConnectionTest.cs | Adds a new test file covering replacement scenarios (idle preference, capacity behavior, failure propagation). |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/IDbConnectionPool.cs | Changes ReplaceConnection return type to nullable. |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ConnectionPoolSlots.cs | Adds TryReplace helper to atomically swap a connection in-place. |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs | Implements ReplaceConnection, adds replacement acquisition logic, and updates PrepareConnection to accept an optional transaction. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4429 +/- ##
==========================================
- Coverage 64.77% 62.88% -1.90%
==========================================
Files 288 283 -5
Lines 44011 66985 +22974
==========================================
+ Hits 28510 42126 +13616
- Misses 15501 24859 +9358
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Introduce an optional System.Threading.RateLimiting policy that throttles new physical connection opens in the channel pool: when a permit is denied the caller waits for a returned connection instead of forcing a create, and leases are always released (including on failure) to avoid starvation. Adds NoOpAcquiredLease, wires the RateLimiting package into the product and test projects, and includes the 006-pool-rate-limiting spec. Also repairs two pre-existing build breaks in ChannelDbConnectionPoolTest (a dropped CountingSuccessfulConnectionFactory declaration and DbConnectionPoolGroupOptions calls missing the new idleTimeout argument).
The connection pool only needs a concurrency limiter (pooling against on-prem SQL Server), so change ChannelDbConnectionPool to take a concrete System.Threading.RateLimiting.ConcurrencyLimiter? instead of the abstract RateLimiter base. The limiter remains optional (null = no limiting), and AttemptAcquire(1)/RateLimitLease usage is unchanged (both inherited). Rework the three rate-limiter unit tests to use real ConcurrencyLimiter instances and assert via GetStatistics() (CurrentAvailablePermits, TotalFailedLeases) instead of the now-removed TestRateLimiter double. Update the spec and diagram to describe a concurrency limiter specifically, noting other limiter types can be added later if needed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the two "options to consider" TODOs above the AttemptAcquire call and replace them with a comment explaining why non-blocking fast-fail was chosen over failing immediately or blocking on the limiter. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Drop the two "options to consider" TODOs above the AttemptAcquire call. The rationale for choosing non-blocking fast-fail lives in the PR discussion rather than in code. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…nel-rate-limiting
Remove the redundant leaseAcquired local; read lease.IsAcquired directly in the early-return guard and the finally-block poke condition. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
RateLimiter_SuccessfulCreate_ReleasesLeaseForNextCreate exercises a single-permit ConcurrencyLimiter with two sequential opens against distinct owners. A leaked lease on the success path would deny the second open, so asserting both create physical connections (CreateCount == 2) guards the release-on-success behavior at the behavioral level rather than only via the permit counter. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover the previously untested concurrency behavior where a caller blocked purely by rate limiting is woken by another caller's lease release (the finally-block null poke) and then creates its own physical connection. RateLimiter_LeaseReleaseWakesRateLimitedWaiter_CreatesPhysicalConnection is a [Theory] over the sync and async idle-channel wait mechanisms. It uses a new GatedSuccessfulConnectionFactory that blocks the first physical create so the permit is held in-flight while a second caller is denied and parks on the idle channel; releasing the gate triggers the release poke that must wake and satisfy the waiter. Verified the test fails (waiter times out) when the poke is disabled. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…osal - Exclude OperationCanceledException from the creation-failure catch so a caller's own timeout/cancellation no longer poisons the pool blocking period. - Gate the finally idle-channel poke to non-faulted completion via a faulted flag, avoiding a redundant double wake on exception paths (cleanupCallback already writes a wake). - Document that the pool does not own the injected ConcurrencyLimiter and never disposes it (caller owns its lifetime). - Fix comment typo (rather then -> rather than) and trailing whitespace. - Reword spec User Story 1 / FR-002 from strict FIFO to best-effort idle-channel wait, matching the non-blocking AttemptAcquire implementation. - Dispose ConcurrencyLimiter instances in tests (using var) and drop the unused System.Collections.Generic using. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…cit method parameters.
- Localize the "slot could not be replaced" guard in ChannelDbConnectionPool.ReplaceConnection: replace the hard-coded InvalidOperationException with ADP.InternalError using a new InternalErrorCode.ConnectionSlotReplacementFailed (still an InvalidOperationException, so behavior is unchanged). - Correct the TryOpenInner forceNewConnection XML remarks: the flag is also valid when the connection was previously opened and is now disconnected (the reconnect path via DbConnectionClosedPreviouslyOpened / DbConnectionClosedConnecting), not only when already open. Also removes a stray blank doc line by using <para> blocks. - Fix the activation-failure test so its name, summary, and inline comment match the implementation: the new connection is disposed (never slotted) and the old connection is left intact, so pool count is unchanged. - Remove unused usings (Microsoft.Data.Common, Microsoft.Data.Common.ConnectionString) from the ReplaceConnection tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlConnection.cs:1619
- This call mixes a named argument (
forceNewConnection: false) with a following positional argument (overrides), which is a C# compile-time error (positional arguments cannot follow named arguments). Name theoverridesargument as well (or revert to fully positional arguments).
if (!(IsProviderRetriable ? TryOpenWithRetry(null, forceNewConnection: false, overrides) : TryOpen(null, forceNewConnection: false, overrides)))
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:324
- The PR description is still the template and does not describe what changed or what testing was performed. Please update the PR description with a concise summary of the behavior change (ReplaceConnection implementation + slot replacement) and which unit tests were added/updated.
SqlClientEventSource.Log.TryPoolerTraceEvent(
"<prov.DbConnectionPool.ReplaceConnection|RES|CPOOL> {0}, replacing connection.", Id);
Replace the opaque ADP.InternalError(ConnectionSlotReplacementFailed) at the ReplaceConnection !replaced guard with a localized InvalidOperationException (SQL_ConnectionPoolReplaceConnectionFailed). Removes the now-unused InternalErrorCode.ConnectionSlotReplacementFailed enum value. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Document that the ReplaceConnection create branch intentionally skips _connectionCreationRateLimiter: a replacement is a 1-for-1 swap (not pool growth) and must make forward progress for an already checked-out caller's reconnect, so the limiter's fast-fail-then-wait-for-idle contract does not apply. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.
Files not reviewed (1)
- src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (1)
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:377
- In ReplaceConnection’s create-new branch, the catch block always calls newConnection.DeactivateConnection(). If ActivateConnection(...) throws, EnterActiveConnection() was never executed (it runs after Activate returns), so DeactivateConnection() will decrement the active-connection metric without a matching increment, potentially driving the counter negative. Consider tracking whether ActivateConnection completed successfully and only calling DeactivateConnection() in that case; otherwise just Dispose() the newly created connection.
newConnection.ClearGeneration = _clearGeneration;
lock (newConnection)
{
// PostPop requires a lock on the connection.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.
Files not reviewed (1)
- src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (1)
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:398
- In the create-new branch, the catch block unconditionally calls newConnection.DeactivateConnection() before Dispose(). If the exception is thrown by PostPop (before an owner is set / before the connection is activated), DeactivateConnection can be an invalid operation and may throw, masking the original failure. Guard the deactivation so it only runs when PostPop succeeded (e.g., Owner != null), while still always disposing the new connection.
catch
{
newConnection.DeactivateConnection();
newConnection.Dispose();
throw;
The new ReplaceConnection tests built pools on TimeProvider.System, letting time-driven background maintenance (idle-timeout pruning, warmup/replenishment, blocking-period expiry) advance in real time and potentially race the assertions. Thread a frozen FakeTimeProvider through the replacement test helper (default) and the TestReplaceConnection case so the pool clock only moves when a test drives it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.
Files not reviewed (1)
- src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (1)
src/Microsoft.Data.SqlClient/src/Resources/Strings.resx:2162
- The new resource key
SQL_ConnectionPoolReplaceConnectionFailedwas added only to the invariant (default)Strings.resx. All localized satellite .resx files (e.g.,Strings.de.resx,Strings.fr.resx, etc.) are missing this key, so non-invariant cultures will fall back to the English text for this message (and some localization validation workflows may fail due to key mismatches). Consider adding this key to eachStrings.<culture>.resx(even if the value is temporarily the English string) to keep resource sets in sync.
<data name="SQL_ConnectionPoolReplaceConnectionFailed" xml:space="preserve">
<value>Could not replace the connection because it is no longer in the connection pool.</value>
</data>
Brings in the completed connection-pool pruning work (Story 2/3/4, #4463), which reworks PoolPruner to be driven by Connection Idle Timeout and only constructs a Pruner when IdleTimeout != 0. The single overlapping file, ChannelDbConnectionPool.cs, auto-merged cleanly: main's constructor pruner block coexists with this branch's ReplaceConnection additions. Also pulls in #4460 (unobserved-exception repro), #4347 (vector test refactor), and #4459 (pool benchmark coverage). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.
Files not reviewed (1)
- src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (1)
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:400
- In ReplaceConnection's create-new branch, the catch block unconditionally calls newConnection.DeactivateConnection(). If ActivateConnection throws (or anything throws before ActivateConnection completes), DbConnectionInternal.ActivateConnection never reaches Metrics.EnterActiveConnection, but DeactivateConnection always calls Metrics.ExitActiveConnection (DbConnectionInternal.cs:519-538). That can drive the ActiveConnections counter negative and misreport pool usage.
Consider restructuring so DeactivateConnection is only invoked after a successful ActivateConnection (e.g., a nested try/catch around ActivateConnection that disposes on failure, or a boolean flag set after ActivateConnection returns).
catch
{
newConnection.DeactivateConnection();
newConnection.Dispose();
throw;
Mirror WaitHandleDbConnectionPool: when the physical open of a replacement connection fails, enter the blocking-period error state so subsequent opens fast-fail until it expires. Activation failures are excluded (the server proved reachable), matching the WaitHandle pool where PrepareConnection runs outside CreateObject's error-state catch. Adds two tests and updates the creation-failure retry test to reflect that the failed open now enters the blocking period. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.
Files not reviewed (1)
- src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (2)
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:378
- Same as above: this TODO is misleading because the code already activates the new connection with the old connection’s enlisted transaction. Rewording the comment helps keep the intent clear.
// TODO: Full transaction enlistment support (Story 2).
newConnection.ActivateConnection(oldConnection.EnlistedTransaction);
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:336
- The TODO about "Full transaction enlistment support" is now misleading: this branch already passes the enlisted transaction into PrepareConnection/ActivateConnection. Consider replacing the TODO with an accurate comment describing what the code is doing (or removing it) to avoid confusion.
This issue also appears on line 375 of the same file.
if (newConnection is not null)
{
// TODO: Full transaction enlistment support (Story 2).
PrepareConnection(owningObject, newConnection, oldConnection.EnlistedTransaction);
oldConnection.DeactivateConnection();
| // _connectionCreationRateLimiter. This mirrors the behavior in WaitHandleDbConnectionPool.ReplaceConnection. | ||
| try | ||
| { | ||
| newConnection = ConnectionFactory.CreatePooledConnection(owningObject, this, timeout); |
There was a problem hiding this comment.
We call CreatePooledConnection directly to skip rate limiting and because we want to swap the new connection into a currently occupied slot.
This pull request implements the connection replacement logic in the
ChannelDbConnectionPool, adds atomic slot replacement support toConnectionPoolSlots, and thoroughly tests the new behavior. It also improves XML documentation and error handling for connection replacement scenarios.This functionality is used to replace broken connections during command execution.