Comparing arrays of different dtype kinds

[asmeurer]

In array-api-strict, I've had this behavior

>>> import array_api_strict as xp
>>> xp.asarray(0, dtype=xp.int64) < xp.asarray(1, dtype=xp.uint64)
TypeError: array_api_strict.int64 and array_api_strict.uint64 cannot be type promoted together

However, this is very annoying, as it makes it basically impossible to compare uint64 and int64. However, I noticed that the standard doesn't actually say anywhere that the inputs to greater/__gt__ need to be type-promotable, only that they should be real numeric. So from my reading of the standard, even comparing integers and floats should be allowed (but not complex numbers or booleans).

That's all good and fine, and I'm going to update array-api-strict. The only question there is whether the standard should be more explicit about this, or if the current wording is OK. It might also be a good idea to update the wording to explicitly state that the comparison should be based on the full values of the operands. There was an issue in NumPy pre-2.0 where < sometimes gave wrong answers because it would downcast float64 to float32, leading to a wrong answer if the rounding went wrong. In other words, the types should promote (or virtually promote) internally before comparing. I actually don't know if all libraries handle the uint64/int64 case correctly since it isn't tested in the test suite. I know NumPy has historically promoted those types to float64, which is dumb since float64 can't even represent the maximum int64. I suppose one could also consider cases of Python int scalars that are very large which should probably compare without doing an overflowing cast, but this could also be treated as unspecified.

The main question I had is with equal/== and not_equal/!=. For those functions, the standard only says the inputs can be "any data type" https://data-apis.org/array-api/latest/API_specification/generated/array_api.equal.html#equal. This implies, by the above reasoning, that comparing any two arrays of any two data types should be allowed. But this implies things like comparing complex arrays to real arrays, and comparing booleans to integers. It might be a good idea to clarify either whether this should not be allowed (or implementation defined), or if it is allowed, how these different types compare to one another. We can probably reuse some other text, for instance, the text at https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__bool__.html#bool for comparing with a bool array and the text at https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__complex__.html#complex for comparing with a complex array.

[asmeurer]

The agreement from the meeting Thursday is that cross-kind comparison should be undefined.

Perhaps unsurprisingly, PyTorch does not allow comparing int64 and uint64:

>>> torch.asarray(9223372036854775807, dtype=torch.int64) < torch.asarray(18446744073709551615, dtype=torch.uint64)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: Promotion for uint16, uint32, uint64 types is not supported, attempted to promote Long and UInt64

PyTorch only has partial support for uint64 anyways. So it's maybe not the best point of comparison. I wonder if CuPy and other libraries do the right thing.

[asmeurer]

#822

[ev-br]

FWIW the RuntimeError of #819 (comment) seems to be a torch bug / artifact of incomplete uintN (N> 8) support. (until very recently, torch only had uint8).

With uint8, pytorch behavior is consistent with numpy and the rest of the ecosystem:

>>> torch.asarray([9223372036854775807], dtype=torch.int64) < torch.asarray([1], dtype=torch.uint8)
tensor([False])
>>> np.asarray(9223372036854775807, dtype=np.int64) < np.asarray(18446744073709551615, dtype=np.uint64)
np.True_

Mixed comparisons with integer/fp types are also OK in torch:

>>> torch.asarray([9223372036854775807], dtype=torch.int64) < torch.asarray([1], dtype=torch.float16)
tensor([False])
>>> 
>>> torch.asarray([9223372036854775807], dtype=torch.uint64) < torch.asarray([1], dtype=torch.float16)
tensor([False])

So all in all disallowing this in array_api_strict on the grounds of a pytorch definiciency seems too strict IMO.