Skip to content

ChannelDbConnectionPool replace connection - #4429

Open
mdaigle wants to merge 41 commits into
mainfrom
dev/mdaigle/replace-conn-2
Open

ChannelDbConnectionPool replace connection#4429
mdaigle wants to merge 41 commits into
mainfrom
dev/mdaigle/replace-conn-2

Conversation

@mdaigle

@mdaigle mdaigle commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

This pull request implements the connection replacement logic in the ChannelDbConnectionPool, adds atomic slot replacement support to ConnectionPoolSlots, and thoroughly tests the new behavior. It also improves XML documentation and error handling for connection replacement scenarios.

This functionality is used to replace broken connections during command execution.

Copilot AI review requested due to automatic review settings July 8, 2026 01:50
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Jul 8, 2026
@mdaigle mdaigle changed the title Dev/mdaigle/replace conn 2 ChannelDbConnectionPool replace connection Jul 8, 2026
@mdaigle mdaigle added this to the 7.1.0-preview3 milestone Jul 8, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements connection replacement support for the channel-based connection pool, including a new slot-level atomic replace operation and expanded unit test coverage around replacement behavior.

Changes:

  • Implement ChannelDbConnectionPool.ReplaceConnection(...) and introduce internal replacement acquisition logic (idle-preferred, otherwise create new).
  • Add ConnectionPoolSlots.TryReplace(...) to atomically swap a connection within an existing reserved slot.
  • Update/extend unit tests, including a new dedicated ChannelDbConnectionPoolReplaceConnectionTest suite and an updated existing replacement test.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ConnectionPoolSlotsTest.cs Adds new tests validating ConnectionPoolSlots.TryReplace behaviors.
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolTest.cs Updates the existing replace-connection test to exercise the new implementation.
src/Microsoft.Data.SqlClient/tests/UnitTests/ConnectionPool/ChannelDbConnectionPoolReplaceConnectionTest.cs Adds a new test file covering replacement scenarios (idle preference, capacity behavior, failure propagation).
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/IDbConnectionPool.cs Changes ReplaceConnection return type to nullable.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ConnectionPoolSlots.cs Adds TryReplace helper to atomically swap a connection in-place.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs Implements ReplaceConnection, adds replacement acquisition logic, and updates PrepareConnection to accept an optional transaction.

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.49123% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.88%. Comparing base (f5be62b) to head (8736037).

Files with missing lines Patch % Lines
...qlClient/ConnectionPool/ChannelDbConnectionPool.cs 95.74% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4429      +/-   ##
==========================================
- Coverage   64.77%   62.88%   -1.90%     
==========================================
  Files         288      283       -5     
  Lines       44011    66985   +22974     
==========================================
+ Hits        28510    42126   +13616     
- Misses      15501    24859    +9358     
Flag Coverage Δ
CI-SqlClient ?
PR-SqlClient-Project 62.88% <96.49%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI review requested due to automatic review settings July 10, 2026 00:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

mdaigle and others added 19 commits July 13, 2026 16:38
Introduce an optional System.Threading.RateLimiting policy that throttles new physical connection opens in the channel pool: when a permit is denied the caller waits for a returned connection instead of forcing a create, and leases are always released (including on failure) to avoid starvation. Adds NoOpAcquiredLease, wires the RateLimiting package into the product and test projects, and includes the 006-pool-rate-limiting spec. Also repairs two pre-existing build breaks in ChannelDbConnectionPoolTest (a dropped CountingSuccessfulConnectionFactory declaration and DbConnectionPoolGroupOptions calls missing the new idleTimeout argument).
The connection pool only needs a concurrency limiter (pooling against
on-prem SQL Server), so change ChannelDbConnectionPool to take a concrete
System.Threading.RateLimiting.ConcurrencyLimiter? instead of the abstract
RateLimiter base. The limiter remains optional (null = no limiting), and
AttemptAcquire(1)/RateLimitLease usage is unchanged (both inherited).

Rework the three rate-limiter unit tests to use real ConcurrencyLimiter
instances and assert via GetStatistics() (CurrentAvailablePermits,
TotalFailedLeases) instead of the now-removed TestRateLimiter double.

Update the spec and diagram to describe a concurrency limiter specifically,
noting other limiter types can be added later if needed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the two "options to consider" TODOs above the AttemptAcquire call and
replace them with a comment explaining why non-blocking fast-fail was chosen
over failing immediately or blocking on the limiter.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Drop the two "options to consider" TODOs above the AttemptAcquire call. The
rationale for choosing non-blocking fast-fail lives in the PR discussion
rather than in code.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the redundant leaseAcquired local; read lease.IsAcquired directly in
the early-return guard and the finally-block poke condition.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
RateLimiter_SuccessfulCreate_ReleasesLeaseForNextCreate exercises a
single-permit ConcurrencyLimiter with two sequential opens against distinct
owners. A leaked lease on the success path would deny the second open, so
asserting both create physical connections (CreateCount == 2) guards the
release-on-success behavior at the behavioral level rather than only via the
permit counter.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover the previously untested concurrency behavior where a caller blocked
purely by rate limiting is woken by another caller's lease release (the
finally-block null poke) and then creates its own physical connection.

RateLimiter_LeaseReleaseWakesRateLimitedWaiter_CreatesPhysicalConnection is a
[Theory] over the sync and async idle-channel wait mechanisms. It uses a new
GatedSuccessfulConnectionFactory that blocks the first physical create so the
permit is held in-flight while a second caller is denied and parks on the idle
channel; releasing the gate triggers the release poke that must wake and
satisfy the waiter. Verified the test fails (waiter times out) when the poke
is disabled.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…osal

- Exclude OperationCanceledException from the creation-failure catch so a
  caller's own timeout/cancellation no longer poisons the pool blocking period.
- Gate the finally idle-channel poke to non-faulted completion via a faulted
  flag, avoiding a redundant double wake on exception paths (cleanupCallback
  already writes a wake).
- Document that the pool does not own the injected ConcurrencyLimiter and never
  disposes it (caller owns its lifetime).
- Fix comment typo (rather then -> rather than) and trailing whitespace.
- Reword spec User Story 1 / FR-002 from strict FIFO to best-effort idle-channel
  wait, matching the non-blocking AttemptAcquire implementation.
- Dispose ConcurrencyLimiter instances in tests (using var) and drop the unused
  System.Collections.Generic using.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

- Localize the "slot could not be replaced" guard in
  ChannelDbConnectionPool.ReplaceConnection: replace the hard-coded
  InvalidOperationException with ADP.InternalError using a new
  InternalErrorCode.ConnectionSlotReplacementFailed (still an
  InvalidOperationException, so behavior is unchanged).
- Correct the TryOpenInner forceNewConnection XML remarks: the flag is
  also valid when the connection was previously opened and is now
  disconnected (the reconnect path via DbConnectionClosedPreviouslyOpened /
  DbConnectionClosedConnecting), not only when already open. Also removes a
  stray blank doc line by using <para> blocks.
- Fix the activation-failure test so its name, summary, and inline comment
  match the implementation: the new connection is disposed (never slotted)
  and the old connection is left intact, so pool count is unchanged.
- Remove unused usings (Microsoft.Data.Common,
  Microsoft.Data.Common.ConnectionString) from the ReplaceConnection tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 27, 2026 19:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlConnection.cs:1619

  • This call mixes a named argument (forceNewConnection: false) with a following positional argument (overrides), which is a C# compile-time error (positional arguments cannot follow named arguments). Name the overrides argument as well (or revert to fully positional arguments).
                    if (!(IsProviderRetriable ? TryOpenWithRetry(null, forceNewConnection: false, overrides) : TryOpen(null, forceNewConnection: false, overrides)))

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:324

  • The PR description is still the template and does not describe what changed or what testing was performed. Please update the PR description with a concise summary of the behavior change (ReplaceConnection implementation + slot replacement) and which unit tests were added/updated.
            SqlClientEventSource.Log.TryPoolerTraceEvent(
                "<prov.DbConnectionPool.ReplaceConnection|RES|CPOOL> {0}, replacing connection.", Id);

Replace the opaque ADP.InternalError(ConnectionSlotReplacementFailed) at the
ReplaceConnection !replaced guard with a localized InvalidOperationException
(SQL_ConnectionPoolReplaceConnectionFailed). Removes the now-unused
InternalErrorCode.ConnectionSlotReplacementFailed enum value.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 28, 2026 15:18
Document that the ReplaceConnection create branch intentionally skips
_connectionCreationRateLimiter: a replacement is a 1-for-1 swap (not pool
growth) and must make forward progress for an already checked-out caller's
reconnect, so the limiter's fast-fail-then-wait-for-idle contract does not apply.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (1)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:377

  • In ReplaceConnection’s create-new branch, the catch block always calls newConnection.DeactivateConnection(). If ActivateConnection(...) throws, EnterActiveConnection() was never executed (it runs after Activate returns), so DeactivateConnection() will decrement the active-connection metric without a matching increment, potentially driving the counter negative. Consider tracking whether ActivateConnection completed successfully and only calling DeactivateConnection() in that case; otherwise just Dispose() the newly created connection.
                    newConnection.ClearGeneration = _clearGeneration;

                    lock (newConnection)
                    {
                        // PostPop requires a lock on the connection.

Copilot AI review requested due to automatic review settings July 28, 2026 15:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (1)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:398

  • In the create-new branch, the catch block unconditionally calls newConnection.DeactivateConnection() before Dispose(). If the exception is thrown by PostPop (before an owner is set / before the connection is activated), DeactivateConnection can be an invalid operation and may throw, masking the original failure. Guard the deactivation so it only runs when PostPop succeeded (e.g., Owner != null), while still always disposing the new connection.
                catch
                {
                    newConnection.DeactivateConnection();
                    newConnection.Dispose();
                    throw;

The new ReplaceConnection tests built pools on TimeProvider.System, letting
time-driven background maintenance (idle-timeout pruning, warmup/replenishment,
blocking-period expiry) advance in real time and potentially race the
assertions. Thread a frozen FakeTimeProvider through the replacement test
helper (default) and the TestReplaceConnection case so the pool clock only
moves when a test drives it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 28, 2026 15:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (1)

src/Microsoft.Data.SqlClient/src/Resources/Strings.resx:2162

  • The new resource key SQL_ConnectionPoolReplaceConnectionFailed was added only to the invariant (default) Strings.resx. All localized satellite .resx files (e.g., Strings.de.resx, Strings.fr.resx, etc.) are missing this key, so non-invariant cultures will fall back to the English text for this message (and some localization validation workflows may fail due to key mismatches). Consider adding this key to each Strings.<culture>.resx (even if the value is temporarily the English string) to keep resource sets in sync.
  <data name="SQL_ConnectionPoolReplaceConnectionFailed" xml:space="preserve">
    <value>Could not replace the connection because it is no longer in the connection pool.</value>
  </data>

Brings in the completed connection-pool pruning work (Story 2/3/4, #4463),
which reworks PoolPruner to be driven by Connection Idle Timeout and only
constructs a Pruner when IdleTimeout != 0. The single overlapping file,
ChannelDbConnectionPool.cs, auto-merged cleanly: main's constructor pruner
block coexists with this branch's ReplaceConnection additions. Also pulls in
#4460 (unobserved-exception repro), #4347 (vector test refactor), and #4459
(pool benchmark coverage).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 28, 2026 23:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (1)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:400

  • In ReplaceConnection's create-new branch, the catch block unconditionally calls newConnection.DeactivateConnection(). If ActivateConnection throws (or anything throws before ActivateConnection completes), DbConnectionInternal.ActivateConnection never reaches Metrics.EnterActiveConnection, but DeactivateConnection always calls Metrics.ExitActiveConnection (DbConnectionInternal.cs:519-538). That can drive the ActiveConnections counter negative and misreport pool usage.

Consider restructuring so DeactivateConnection is only invoked after a successful ActivateConnection (e.g., a nested try/catch around ActivateConnection that disposes on failure, or a boolean flag set after ActivateConnection returns).

                catch
                {
                    newConnection.DeactivateConnection();
                    newConnection.Dispose();
                    throw;

Mirror WaitHandleDbConnectionPool: when the physical open of a
replacement connection fails, enter the blocking-period error state so
subsequent opens fast-fail until it expires. Activation failures are
excluded (the server proved reachable), matching the WaitHandle pool
where PrepareConnection runs outside CreateObject's error-state catch.

Adds two tests and updates the creation-failure retry test to reflect
that the failed open now enters the blocking period.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 28, 2026 23:50
@mdaigle
mdaigle marked this pull request as ready for review July 28, 2026 23:52
@mdaigle
mdaigle requested a review from a team as a code owner July 28, 2026 23:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file
Comments suppressed due to low confidence (2)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:378

  • Same as above: this TODO is misleading because the code already activates the new connection with the old connection’s enlisted transaction. Rewording the comment helps keep the intent clear.

                    // TODO: Full transaction enlistment support (Story 2).
                    newConnection.ActivateConnection(oldConnection.EnlistedTransaction);

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs:336

  • The TODO about "Full transaction enlistment support" is now misleading: this branch already passes the enlisted transaction into PrepareConnection/ActivateConnection. Consider replacing the TODO with an accurate comment describing what the code is doing (or removing it) to avoid confusion.

This issue also appears on line 375 of the same file.

            if (newConnection is not null)
            {
                // TODO: Full transaction enlistment support (Story 2).
                PrepareConnection(owningObject, newConnection, oldConnection.EnlistedTransaction);
                oldConnection.DeactivateConnection();

// _connectionCreationRateLimiter. This mirrors the behavior in WaitHandleDbConnectionPool.ReplaceConnection.
try
{
newConnection = ConnectionFactory.CreatePooledConnection(owningObject, this, timeout);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We call CreatePooledConnection directly to skip rate limiting and because we want to swap the new connection into a currently occupied slot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

4 participants