Use swift-native "SocketAddress" facilities

[MahdiBM]

I'd assume the NIO team is interested in using (or merging) a library like swift-endpoint instead of accessing the platform C APIs.
I've heard from NIO maintainers that they don't like how the current SocketAddress leaks some C APIs.

This could either wait for the next major NIO version, or we could deprecate some of the current APIs in favor of those in swift-endpoint.

The advantages for using swift-endpoint would be:

• No behavior discrepancies across platforms.
  • See this comment in AHC as an example of why that matters.
• Better performance compared to using the C APIs.
  • See this section of the swift-endpoint README for more info.
• All in Swift.
  • No need to expose or interact with C APIs.

I can't really think of any disadvantages, but I'm all ears if you know of something that would degrade if you were to use swift-endpoint.
One issue is swift-endpoint currently relies on ByteBuffer for the DomainName implementation so we'll need to make sure that doesn't result in a circular dependency.
Another issue is that I Just went with requiring Swift 6.2, which is a requirement that could be lowered although might take some library downgrades, since the library uses "the latest and greatest" like the new Span APIs (including some parts that were not back-deployed).

The library is also well-tested and well-benchmarked.
I'm open to discussions if you think something would need to change if NIO were to adopt this library.

[Lukasa]

Thanks for filing this @MahdiBM.

I think some changes are likely to be a good idea here, but I think we'd like them to be a considered set of changes, rather than jumping immediately on swift-endpoint.

Firstly, a note: having excessive C API surface is sub-optimal, but we can't get away with none. It is a hard requirement of SocketAddress at the moment that it must be able to convert itself to something we can bind or connect to, and those APIs rely on getting a sockaddr pointer. So at some stage the transformation must take place. We can decide it takes place in NIO if swift-endpoint (or a similar library) decides it is out of scope, but in that context we need sufficient information to correctly reconstruct the relevant address type.

As another note, for NIO to adopt swift-endpoint any time before the release of Swift 6.4 you would absolutely have to drop the minimum Swift version. More broadly, any library that NIO takes as a dependency is going to need to commit to a compatibility policy at least as broad as NIO's itself. This doesn't require any action from you if you're happy to let this sit and wait to some future release though, and I don't want you to jump ahead and do this prior to any kind of firm commitment from us.

We will also want unix domain socket support, as the existing SocketAddress type supports them already.

However, I think a more important step is to get broader community feedback, including from the SSWG. NIO has a very minimal dependency set, currently depending only on things that are standard library adjacent. Widening that dependency set is not unreasonable, but we are pretty conservative about doing it. However, I don't think NIO should just merge these types either. What we want is a broader set of currency types that can be used in a wide range of contexts, including those where NIO needn't appear at all. That is something the NIO project itself is not really set up to do, and so wants to be part of a broader conversation with the community.

[MahdiBM]

@Lukasa Sure, we'll need to discuss this at length before it comes to happen (if at all), I know NIO is rightfully strict about these stuff.

We can decide it takes place in NIO if swift-endpoint (or a similar library) decides it is out of scope, but in that context we need sufficient information to correctly reconstruct the relevant address type.

I'm not against having those in swift-endpoint.

As another note, for NIO to adopt swift-endpoint any time before the release of Swift 6.4 you would absolutely have to drop the minimum Swift version.

That's what I thought, and it makes sense.

What I actually wonder about though, is what do you think about the macOS requirements?
IIRC I can lower the macOS requirement to macOS 13, for the most part by simply using a pair of UInt64s instead of a UInt128 in IPv6, but is macOS 13 even enough?

However, I think a more important step is to get broader community feedback, including from the SSWG.

Yup, we can let everyone know in the Swift forums, if we can first come to a conclusion ourselves and if we think such a integration is possible at all.

For example I'll need to confirm I can lower the macOS requirements enough. I'm fine with the possibility of having to wait for Swift 6.4, but the macOS requirements are not going to solve themselves over time, unless NIO decides to drop support for those old macOS versions some time in the future.

[Lukasa]

Yeah, I'd forgotten the platform requirements.

Right now SwiftPM makes those a hard limitation: NIO can't raise its minimum deployment target without a semver major, and we aren't doing one of those any time soon. The minimums that SwiftPM supports do slowly raise over time, and so that is having the impact of lifting NIO's minimums, so this is another problem that either time will solve, or SwiftPM will fix, but right now even macOS 13 is too high.

[MahdiBM]

To be clear I'm not talking about SwiftPM-level package requirements. I'm not using those.
I'm simply using #/@available attributes in the code. Given that knowledge, do you still think that macOS 13 is too high?
I think so tbh, looking at NIO, but want to be sure.

[Lukasa]

Yeah, NIO would need to keep using SocketAddress widely, and so the availability constraints are still too high.

[FranzBusch]

However, I think a more important step is to get broader community feedback, including from the SSWG. NIO has a very minimal dependency set, currently depending only on things that are standard library adjacent. Widening that dependency set is not unreasonable, but we are pretty conservative about doing it. However, I don't think NIO should just merge these types either. What we want is a broader set of currency types that can be used in a wide range of contexts, including those where NIO needn't appear at all. That is something the NIO project itself is not really set up to do, and so wants to be part of a broader conversation with the community.

Personally speaking not on behalf of the SSWG here. I think having standardized currency types for networking such as IPAddress and friends is an important gap to fill. I agree that NIO should keep a minimal set of dependencies but taking on such a dependency is IMO worth it. Having said that, we certainly need to find an "official" place for those currency types to land with the community. I personally would prefer if we could land them in the swiftlang organization but we need a steering group such as the ESG to sponsor such an effort. This would also include some kind of official evolution process that might just be PR reviews but it needs to be documented. I can bring up the topic in the SSWG and ESG meetings.

[MahdiBM]

This would also include some kind of official evolution process that might just be PR reviews but it needs to be documented. I can bring up the topic in the SSWG and ESG meetings.

Let's go ahead with those then, please. I'm fine with some kind of evolution process / posting on the forums. In fact I've already done that, although not in an official-enough manner which might be needed 😅

[MahdiBM]

TL;DR: Just merged this to lower the macOS requirements to 13:
Introduce UnsignedInt128 as an alternative to UInt128 to lower macOS requirements to 13

If you're on macOS 15+ this type won't make much of a difference compared to UInt128. You're encouraged to simply do UInt128(myUnsignedInt128) and use that instead, just incase.
In the future UnsignedInt128 can be reduced to a typealias of UInt128 with no API breakages.
I've put an exhaustive test suite together (other than for the Codable conformance) that verifies the implemented functions against UInt128.
This should be more than sufficient.

[MahdiBM]

As we discussed earlier, I do realize even macOS 13 is not enough to get things into NIO.
It should be more than enough to have swift-endpoint as a "standard" package of the ecosystem though.
I am, however, going to look and see if I can get rid of the macOS 13 requirement altogether. I don't have an accurate idea what it would require, off-hand.

[MahdiBM]

TL;DR: Supporting down to SwiftPM's lowest supported-macOS looks ... improbable to do.

Hmmm that didn't look too good. If I understand correctly Span requires macOS 10.14.4 while SwiftPM's base is even lower than that.

In hindsight it would have been kind of cool for SwiftPM to raise the base to 10.14.4 in Swift 6.2 just so Span APIs can be used more widely.
I couldn't find a quote on what is the base macOS version that SwiftPM supports.
I find it questionable that SwiftPM supports such low macOS versions at all but I could be missing some info / insight. It's just that I assume those macOS versions are kind of extinct now, specially for developers to use. Do we really need to inflict such pain on ourselves and support such low macOS versions?

Anyway, this was more like a rant since we can't go in the past to modify SwiftPM for the Swift 6.2 release. I guess even if such a change could make it to 6.3 that'd be nice, but that would mean swift-endpoint adoption would require Swift 6.5 to be out, not just 6.4, which is fine but not exciting. That is all assuming I don't just remove the Span usages, which is not off the table.

[MahdiBM]

Another update:

TL;DR: swift-endpoint (and swift-idna) will support down to macOS 10.15 now.

I went for supporting down to macOS 10.15 for now, since it didn't require too much work.
For the most part only String(unsafeUninitializedCapacity:initializingUTF8With:) is not available on macOS 10.15 (requires macOS 11), which is not hard to work around.

Result is, swift-endpoint and swift-idna (which swift-endpoint relies on) support down to macOS 10.15.
swift-dns which is mainly where I'm personally using swift-endpoint myself, can't support macOS < 13 easily since it uses the connection pool module of postgres-nio which uses Clock and Duration which require macOS 13+.

I'm waiting on CI to update the benchmark thresholds, then I'll merge the PRs:
swift-endpoint PR, swift-idna commit, swift-idna followup PR, swift-dns PR

Unfortunately the backward-compatible String(unsafeUninitializedCapacity:initializingUTF8With:) that I've introduced results in up to 20% regressions in some benchmarks. I have yet to investigate too much further, I know that on Linux where the benchmarks are run, the compiler is smart enough to not consider the useless code-path in if #availables, so not sure what is causing the regressions. Will have to profile things on Linux to be able to tell.

The regressions are not a big deal though. They mostly affect the code paths which are ~8-10x faster than the inet APIs anyway.

EDIT: Managed to mitigate most of the regressions. Just so you know how weird the regression was: I was forced to use @_transparent.

[MahdiBM]

Related to #3448 (comment):

https://forums.swift.org/t/can-swiftpm-not-support-ridiculously-low-macos-versions/83441

[MahdiBM]

I had not noticed these following issues/PRs, so just linking the issues/PRs to each other since they are related:

#1650 ("Provide IPAddress type")
#1673 ("C system types are exposed, which may be incompatible [with Windows]")
#1910 ("Fix up SocketAddress")
#2054 ("Provide IPAddress wrapper type")

I found out about them when searching issues for "Windows", since I was trying to make a library Windows compatible but the CI was (and is) failing.