SIP-26: Unsigned Integers

[sjrd]

Imported from scala/slip#30

By: Sébastien Doeraene @sjrd and Denys Shabalin @densh

tl;dr: We propose the addition of 4 "primitive" types to represent unsigned integers: UByte, UShort, UInt and ULong.

A prototype implementation of this proposal, with unit tests and benchmarks, can be found at scala/scala@2.12.x...sjrd:uints

For the non-tl;dr, read the complete text.

[Ichoran]

Rather than go through in detail point by point, I'll start with a higher-level concern: this creates a new category of thingy which is neither a fully supported primitive nor an object. In particular we have

• No unsigned literals
• No unsigned arrays
• Broken overloaded methods (def x(i: Int); def x(u: UInt))
• No specialization
• Slower equality on everything numeric
• Awkward Java interop (boxing/erasure issues)

This means that rather than being a helpful stepping stone, an intermediate proposal creates the need for a variety of workarounds, which then will either fix in place this awkward not-fully-integrated scheme, or require a bunch of code changes again when things are no longer awkward.

Because of this, and because of a lack of motivating examples for the extent of the integration so far, I'd recommend an even less integrated version of unsigned integer types: they are just AnyVal wrappers, and you give a UArray AnyVal type that presents the components as their unsigned counterparts.

If it is uncomfortable to live with the limitations of AnyVal types, well, everyone else trying to use AnyVal types also has to live with those uncomfortable limitations. If those are fixed in a generally useful way, then everyone benefits.

[sjrd]

@Ichoran Thanks for your summary of the concerns you had expressed earlier. I'm almost on my way to vacation, so I'll come back to this end of August.

[Ichoran]

@sjrd - I thought it bore repeating since the draft doesn't entirely address the issues even if it does acknowledge some of them.

[soc]

I agree with @Ichoran. While having unsigned numbers might be nice, I fear that the current technical limitations prevent us from having an implementation on the JVM that would make people even remotely happy.

I feel that it would be better to wait until the JVM ships with value types and revisit this. The worst case would be to standardize something now, and then realize that it won't "upgrade" to JVM value types without breaking semantics and user code.

[sjrd]

Well ... I was trying to address the == performance problem by any means necessary, but despite working on it for days--and going as far as inspecting the assembly generated by Hotspot to understand the issues--I have not been able to reach a solution with 0% overhead for existing codebases.

My efforts can be found here: scala/scala@2.12.x...sjrd:improve-equals-hashcode

The closest I got with a complete implementation is a 6% performance hit on the foosHashMap benchmark.

3 of those 6% are due to the changes in ##. They can be reclaimed if UByte and UShort use an Int or a Char as underlying value rather than Byte/Short respectively. That is because with an Int or a Char, the generated hashCode() is already correct, and Statics.anyHash does not need any change (it already does not need change for UInt and ULong).

The remaining 3% are due to the changes to BoxesRunTime.equals2 (not the other methods). Note that the 4 instanceof tests on the specific classes are faster than 1 instanceof test against a common interface UnsignedInteger. Those 3% could be reclaimed if we changed the compiler to hack into the unsigned int classes, so that they extends java.lang.Number at run-time, because those 4 tests would be added at the end of equalsNumObject, which is currently a code path that never happens.

I guess this is the end of this SIP, given that:

1. We can't have the cooperative equality without overhead on the JVM, and,
2. Cooperative equality is essential to the main motivation (interop with JS unsigned integers).

Since I couldn't fix those performance, I haven't spent time trying to address the other comments.

I'll leave this PR open until the next SIP meeting (Sep 13), since it's scheduled for discussion, at which point I guess the committee will officially reject it.

[jvican]

Thanks for describing all your findings and results, @sjrd. We all appreciate the work you've put into this.

[jvican]

The following proposal was unanimously rejected by the SIP Committee, to the request of his author @sjrd. For more information, check the minutes.