Skip to content

Support NIO Transport Services #135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

Yasumoto
Copy link
Contributor

This is the investigation for #127

This builds on apple/swift-nio#1253 and apple/swift-nio-transport-services#65, mostly motivated by soto-project/soto-core#141.

Caveats:

  1. I wasn't able to see how to support ChannelOptions.maxMessagesPerRead; we'll need this to make sure we don't block in testUploadStreamingBackpressure.
  2. As mentioned in narrower bootstrap ELG requirements apple/swift-nio#674, we're really pushing back on our users to always use NIOTSEventLoopGroup if it's available. We should likely also add a check if we're passed in a .shared() EventLoop that we use the corresponding bootstrap, but figured I'd post this up here for an initial review first.

@swift-server-bot
Copy link

Can one of the admins verify this patch?

2 similar comments
@swift-server-bot
Copy link

Can one of the admins verify this patch?

@swift-server-bot
Copy link

Can one of the admins verify this patch?

@Yasumoto Yasumoto force-pushed the yasumoto-transport-services branch from 6a53611 to 11dacad Compare November 19, 2019 17:39
self.eventLoopGroup = MultiThreadedEventLoopGroup(numberOfThreads: 1)
#if canImport(Network)
if #available(OSX 10.14, iOS 12.0, tvOS 12.0, watchOS 6.0, *) {
self.eventLoopGroup = NIOTSEventLoopGroup()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weissi do you think it's a good idea to default to TS on darwin?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

var bootstrap: ClientTransportBootstrap

if #available(OSX 10.14, iOS 12.0, tvOS 12.0, watchOS 6.0, *) {
bootstrap = NIOTSConnectionBootstrap(group: self.eventLoopGroup)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not channelEL ?? delegateEL?

let loopGroup: EventLoopGroup
let external: EventLoopGroup

#if canImport(Network)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should extract this to a test helper function (that returns one ELG)?

@artemredkin artemredkin requested a review from weissi December 5, 2019 13:57
@weissi
Copy link
Contributor

weissi commented Mar 24, 2020

ping @Yasumoto this is now ready :)

@weissi
Copy link
Contributor

weissi commented Mar 24, 2020

how this works is: You only need one #if ... and then you can

#if canImport(Network)
// Creating the "universal bootstrap" with the `NIOSSLClientTLSProvider`.
let tlsProvider = NIOTSClientTLSProvider(tlsOptions: .init())
let bootstrap = NIOTSClientTLSProvider(NIOTSEventLoopGroup(group: group), tls: tlsProvider)
#else
// TLS setup.
let configuration = TLSConfiguration.forClient()
let sslContext = try NIOSSLContext(configuration: configuration)

// Creating the "universal bootstrap" with the `NIOSSLClientTLSProvider`.
let tlsProvider = NIOSSLClientTLSProvider<ClientBootstrap>(context: sslContext, serverHostname: "example.com")
let bootstrap = NIOClientTCPBootstrap(ClientBootstrap(group: group), tls: tlsProvider)
#endif

// Bootstrapping a connection using the "universal bootstrapping mechanism"
let connection = bootstrap.enableTLS()
                     .connect(host: "example.com", port: 12345)
                     .wait()

For AsyncHTTPClient, you'll want to cache two bootstraps:

let tlsBootstrap: NIOClientTCPBootstrap = makeBootstrapForPlatform().enableTLS()
let plainTextBootstrap: NIOClientTCPBootstrap = makeBootstrapForPlatform()

and then you can just use the right one to bootstrap connections from depending on if that's TLS or not.

@Yasumoto
Copy link
Contributor Author

Oh heck yah 💪

@adam-fowler
Copy link
Member

Hey guys,
I have a version of this working on another branch. There are still a few issues: a couple of tests fail and TLS testing with NIOTS fails but that looks like it is a problem with the TLS testing, not the code.

I have tested it with aws-sdk-swift and it works on both macOS and iOS. Should I open another PR?

@fabianfett
Copy link
Member

I have tested it with aws-sdk-swift and it works on both macOS and iOS. Should I open another PR?

+1 the sooner we get the special Darwin treatment out of aws-sdk-swift the better. I can only imagine @Yasumoto would be supporting this.

@Yasumoto
Copy link
Contributor Author

@adam-fowler amazing! If you’ve already got a branch ready for review let’s close this out and transition to a new PR 🙇

weissi added a commit that referenced this pull request Apr 18, 2020
This is the continuation of the good work of @Yasumoto and @weissi in #135

The following code adds support for NIO Transport services. When the ConnectionPool asks for a connection bootstrap it is returned a NIOClientTCPBootstrap which wraps either a NIOTSConnectionBootstrap or a ClientBootstrap depending on whether the EventLoop we are running on is NIOTSEventLoop.

If you initialize an HTTPClient with eventLoopGroupProvider set to .createNew then if you are running on iOS, macOS 10.14 or later it will provide a NIOTSEventLoopGroup instead of a EventLoopGroup.

Currently a number of tests are failing. 4 of these are related to the NIOSSLUncleanShutdown error the others all seem related to various race conditions which are being dealt with on other PRs. I have tested this code with aws-sdk-swift and it is working on both macOS and iOS.

Things look into:

The aws-sdk-swift NIOTS HTTP client had issues with on Mojave. We should check if this is the case for async-http-client as well.

Co-authored-by: Joe Smith <yasumoto7@gmail.com>
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants