Adopt TypeScript & Refocus Core APIs

[viglucci]

This issue proposes that an effort should be made to re-write the core set of rsocket-js packages from Flow to TypeScript, with a focus on adopting a "small core" mentality when designing the projects public API.

This issue is to serve as:

1. Documentation of the proposal that was discussed and decided upon
2. A means of tracking progress towards implementing the proposal

Motivation

TypeScript in Favor of Flow

TypeScript is a proven solution for authoring JavaScript libraries for use in NodeJs and browser environments, and will provide a number of benefits to the project:

• Continued development & support from Microsoft
• Large community adoption
• Easier and more familiar tooling
• Types can be published from source without authoring additional @types

In addition to the above, in a recent post on the Flow Type blog, the Flow team described a refocusing of the projects priorities towards Facebook's internal needs. This is a signal that Flow may not be a suitable solution for projects outside of Facebook's internal infrastructure and ecosystem.

Small Core

New APIs and paradigms should be introduced to adopt a "small core" mentality, with core rsocket-js packages providing only the basic building blocks and implementations for the RSocket protocol. APIs should be easily extendable/composable to support additional packages that can later provide functionality outside of implementing the RSocket protocol, for example, an RxJS interface.

This "small core" approach is intended to assist with maintainability of the project, as well as ease of support for satisfying feature requests through extensions and "addon" packages.

Other RSocket implementations, such as RSocket-Swift, have benefited to a degree from this approach.

Serialization and Encoding

In conformance with the "small core" goal of this initiative, support for payload data encoding should be reduced down to transmitting instances of Buffer, rather than supporting a wide variety of data and metadata payload serialization and encodings. This stance is in contrast to the current support provided by @rsocket packages, where support for serializing objects to JSON and other formats is provided by interfaces such as JsonSerializer, and encoding is provided by interfaces such as BufferEncoders.

Support for user friendly translation of objects and other structures to/from JSON and other serialization formats can be accomplished via extension packages, rather than as a core feature of rsocket-js.

Reactive APIs

rsocket-js has historically leveraged a Reactive Streams implementation (rsocket-flowable) both internally, and as its public API. This implementation has historically been published under @rsocekt/rsocket-flowable. Moving forward, rsocket-js will move away from @rsocket/rsocket-flowable as a key feature of its public API, and instead focus on exposing a core set of Stream APIs that satisfy the core requirements of the RSocket protocol, while also promoting composition and extension with other Reactive Streams implementations, such as RxJS and reactive-streams-js.

Interfaces example:

export interface Cancellable {
  cancel(): void;
}

export interface Requestable {
  request(requestN: number): void;
}

export interface OnExtensionSubscriber {
  onExtension(
    extendedType: number,
    content: Buffer | null | undefined,
    canBeIgnored: boolean
  ): void;
}

export interface OnNextSubscriber {
  onNext(payload: Payload, isComplete: boolean): void;
}

export interface OnTerminalSubscriber {
  onError(error: Error): void;
  onComplete(): void;
}

The intention of this change is two fold:

• Simplify the internal implementation details of the core rsocket-js libraries
• Allow for consumers and integrators to easily extend rsocket-js with their Reactive Streams implementation of choice

Desired solution

Core RSocket protocol feature support, and APIs implementations needed:

• RSocketConnector API
• Transports
  • TCP Client
  • TCP Server
  • WebSocket Client
    • Browser compatibility
    • Non-browser compatibility
  • WebSocket Server
• Request Operators
  • Fire and Forget
  • Request Response
  • Request Stream
  • Request Channel
  • Metadata Push
• Fragmentation
• Keepalive
• Composite Metadata
  • Routing
  • AuthMetadata
• Resumption
• Leasing
  • Requester side leasing
  • Responder side leasing
• Requester Side Acceptor
• Encoding/Decoding API
• Loadbalancing
• Improved browser environment support
  • Clear support/guidance on Buffer API in Browser environments
    • issue: LiteBuffer overwrites Buffer import in React Native #122
    • issue: rsocket-js can not work in browser #195
    • issue: RFC Browser Support & Buffer Module #213

TODO: complete list of necessary core protocol features

Considered alternatives

N/A

Additional context

Major Version Change

Because of the breaking API changes that this effort will introduce, the version of all @rsocket scoped packages who's APIs are modified will be incremented to an initial major version release of 1.0.

Issues & pull requests related to this work should be/are tagged with the 1.0 tag.

Work in progress

Work for this effort is ongoing in the dev branch.

API Example

const connector = new RSocketConnector({
  transport: new TcpClientTransport({
    connectionOptions: {
      host: "127.0.0.1",
      port: 9090,
    },
  }),
});

const rsocket = await connector.bind();

await new Promise((resolve, reject) =>
  rsocket.requestResponse(
    {
      data: Buffer.from("Hello World"),
    },
    {
      onError: (e) => reject(e),
      onNext: (payload, isComplete) => {
        console.log(
          `payload[data: ${payload.data}; metadata: ${payload.metadata}]|${isComplete}`
        );
        resolve(payload);
      },
      onComplete: () => { },
      onExtension: () => { },
      cancel: () => { },
      request: () => { },
    }
  )
);

[sdeleuze]

Thank you so much for working on this.

[Polve]

wonderful, thanks

[LifeIsStrange]

Maybe that support for interceptors could be added to the desired solution checklist #192 ?

[lachenmayer]

Hi @viglucci, thank you so much for your efforts on this - this would make our lives so much easier. If you'd like us to "beta test" any of your changes, let me know when you think it's ready to look at.

I'm particularly excited about the removal of Flowable - we currently have a RxJS<->Flowable interop layer which pretty much completely disables the back-pressure features of Flowable. It would be awesome if we could just use regular RxJS operators on Flowables. This could be achieved by making a Flowable implement the Observable interface, or by defining generic helper functions which could model some common back-pressure mechanisms.

For example, we currently define toObservable<T>: (flowable: Flowable<T>, options: { batchSize: number }) => Observable<T>, which requests batchSize entries at a time once the subscription is active.

To implement toFlowable<T>: (observable: Observable<T>) => Flowable<T>, we have created BufferingSubscription and DroppingSubscription classes which buffer or drop emitted values on the given observable if the subscriber hasn't requested any more values.

As API inspiration, I found the Apple Combine Subscription class a quite nice request API. One nice feature is that you can just request unlimited, effectively disabling back-pressure. (It's also possible to request none, but I'm not sure how useful that is - maybe for the fire and forget request type?).

[viglucci]

Hi @lachenmayer, and others.

Many thanks for your kind words and encouragement.

In regards to beta testings the changes that have been made, we are hoping to have a 0.1.0 release, or another semantic versioning appropriate preview version available ahead of the initial 1.0.0 release, hopefully sometime between now and the end of Q1 2022 (ideally sooner rather than later, but of course these timelines are fluid and subject to change). We'll be sure to broadcast its availability once that has happened.

In regards to RxJS interoperability, one of our goals is definitely improved interoperability with existing reactive libraries, and to support future implementations (such as a Reactive Streams compliant library in JavaScript). We believe this will likely take the form of an "extension" library which will wrap/compose with the core RSocket APIs, rather than as a core feature of RSocket-JS. I discussed this approach a bit more here. You'll see in the linked example of an RxJS extension that the API is pretty simple, though the batching approach you mention should likely be something we consider.

The finer details of how the extensions will take shape, and which we'll implement immediately are all still very much in the works though, so I do appreciate the details and examples you've provided above, as they will likely serve well as inspiration.

[viglucci]

Providing a status update on this effort:

We believe we are at a point with the 1.0.x branch where we would like to publish some preview versions to npm. We are working through some outstanding requirements to be able to actually publish on npm and will do so once those blockers are cleared. Once those preview versions are available we'll be sure to add a release on GitHub and provide an update referencing them.

Additionally, there have been some conversations around how to best support browser environments that require a buffer polyfill, and the current strategy we are leaning towards is to require consumers to polyfill buffer via their build tooling, and provide ample documentation to make that as clear and painless as possible. If you are interested in that conversation or would like to provide insights or alternatives, please refer to #213 .

[viglucci]

We are excited to share that we've released the first alpha versioned artifacts from this effort. 🎉 😄

Please see the 1.0.0-alpha.1 release for an overview.