Get/Return pooled connections

[mdaigle]

Description

This PR adds Get and Return functionality to the new ChannelDbConnectionPool class.
A channel and corresponding channel reader and writer are added to underpin these operations.
An array is added to hold references to all connections managed by the pool.
Logic is included to respect max pool size in all cases.

Not included: pool warm up (including respecting min pool size), pool pruning, transactions, tracing/logs. These will come in follow-up PRs.

Also includes extensive unit test coverage for the implemented functionality. Integration testing is reserved for a later state when more of the pool functionality is available.

Issues

#3356

Testing

Tests focus on the Get and Return flows with special focus on the following:

• async behavior
• max pool size respected
• connections successfully reused
• requests successfully queued and queue order respected
• stress testing with many parallel operations

[mdaigle]

@roji tagging you here if you have any time to review. Thanks!

[benrr101]

In general, it looks good. A bunch of things I'd like addressed, but you know I'll accept valid arguments against fixing them :)

[paulmedynski]

I have reviewed the ChannelDbConnectionPool changes. I will look at the tests once we have converged on an implementation. It might be a good idea to host another meeting to walk through the commentary here.

[imasud00]

ChannelDbConnectionPool

it might be good to add some documentation on how this class works and how this is different than the WaitHandleDbConnectionPool.

[paulmedynski]

We're getting closer!

[codecov]

Codecov Report

❌ Patch coverage is 89.57529% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.78%. Comparing base (5354e83) to head (cf418ab).
⚠️ Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
...qlClient/ConnectionPool/ChannelDbConnectionPool.cs	89.28%	21 Missing ⚠️
...icrosoft/Data/ProviderBase/DbConnectionInternal.cs	33.33%	2 Missing ⚠️
...ta/SqlClient/ConnectionPool/ConnectionPoolSlots.cs	95.91%	2 Missing ⚠️
...c/Microsoft/Data/SqlClient/SqlConnectionFactory.cs	80.00%	2 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (5354e83) and HEAD (cf418ab). Click for more details.

HEAD has 2 uploads less than BASE

Flag	BASE (5354e83)	HEAD (cf418ab)
netcore	1	0
addons	1	0

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3404      +/-   ##
==========================================
- Coverage   69.75%   63.78%   -5.97%     
==========================================
  Files         276      201      -75     
  Lines       62192    44159   -18033     
==========================================
- Hits        43381    28169   -15212     
+ Misses      18811    15990    -2821     

Flag	Coverage Δ
addons	?
netcore	?
netfx	63.78% <89.57%> (-5.64%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[paulmedynski]

Still looking for some changes from previous reviews. Good changes in these last 4 commits though.

[paulmedynski]

Code looks good. Will look at tests shortly. Going to loop through older comments first.

[paulmedynski]

I had a look at the slot tests. I'll spin through the pool tests once this set of comments is resolved.

[Copilot]

Pull Request Overview

This PR implements core Get and Return functionality for the new ChannelDbConnectionPool class, establishing the foundation for channel-based connection pooling. The implementation introduces a novel approach using System.Threading.Channels for managing idle connections and a custom ConnectionPoolSlots class for thread-safe connection tracking.

• Adds channel-based connection pooling with Get/Return operations that respect max pool size limits
• Implements ConnectionPoolSlots for thread-safe connection management with reservation-based slot allocation
• Includes comprehensive unit test coverage for both connection pooling logic and slot management

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tools/specs/Microsoft.Data.SqlClient.nuspec	Adds System.Threading.Channels dependency for both .NET 6+ and .NET 8+ target frameworks
src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/ConnectionPool/ConnectionPoolSlotsTest.cs	New comprehensive unit tests for ConnectionPoolSlots covering concurrent operations, capacity limits, and edge cases
src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPoolTest.cs	Extensive unit tests for ChannelDbConnectionPool covering sync/async operations, timeout handling, connection reuse, and stress testing
src/Microsoft.Data.SqlClient/src/System/Runtime/CompilerServices/IsExternalInit.netfx.cs	Adds explanatory comment for C# 9.0 init accessor polyfill
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlConnectionFactory.cs	Makes constructor protected and refactors connection creation methods to support pooling scenarios
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/WaitHandleDbConnectionPool.cs	Updates ReturnInternalConnection signature and removes redundant validation logic
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/IDbConnectionPool.cs	Updates interface signatures for nullable references and standardizes parameter types
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/DbConnectionPoolState.cs	Removes Initializing state from enum
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ConnectionPoolSlots.cs	New thread-safe collection with fixed capacity using reservation-based slot allocation
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/ChannelDbConnectionPool.cs	Implements core connection pooling with channel-based idle connection management
src/Microsoft.Data.SqlClient/src/Microsoft/Data/ProviderBase/DbConnectionInternal.cs	Exposes CreateTime property and updates PrePush signature for stronger typing
Various project files	Adds System.Threading.Channels package references across different target frameworks

[benrr101]

Delays in unit tests are always a bit of a smell ... but ehhhhhhhhhhhhhh

[paulmedynski]

A couple of new suggestions. I'll look at the remaining tests after 6.1.1 has simmered down.

[paulmedynski]

Does this function throw? The docs don't say much, and the code doesn't mention which calls within the function, if any, could throw. It would be bad for this to throw and then we lose the original exception.

[paulmedynski]

Just noticed this. You can capture the state info directly within the delegate lambda expressions in OpenNewInternalConnection():

return _connectionSlots.Add(
    createCallback: () =>
    {
         return ConnectionFactory.CreatePooledConnection(
             owningConnection,
             this,
             PoolGroup.PoolKey,
             PoolGroup.ConnectionOptions,
             userOptions);
    },
    cleanupCallback: static (newConnection) =>
    {
        _idleConnectionWriter.TryWrite(null);
        newConnection?.Dispose();
    });

This changes the delegates from static to not-static, so that may have performance implications, but I'd be surprised. The old state arguments weren't freezing-in-time any instance values, so there shouldn't be any functionality change in the above.

[paulmedynski]

Duplicate check.

[paulmedynski]

Should you check that ReservationCount is 1 here?