RFC: add support for querying whether an array library/device supports subnormal numbers

[kgryte]

Overview

The handling of subnormal numbers has come up a few times since the beginning of the Consortium:

• Should finfo have smallest normal/subnormal attributes? #131
• Improve subnormal expectations #339
• Add note concerning subnormal numbers #341
• Add hypot specification for computing the square root of the sum of squares #703

and is more generally specified as part of IEEE 754, the standard for floating-point numbers.

In short, support for subnormal numbers is likely to vary across conforming array libraries due to hardware constraints and/or configuration. For example, some devices support flushing subnormal numbers to zero for increased performance or for improving security by thwarting timing attacks.

Given this variability, array library consumers don't have a standardized mechanism for determining whether a library and/or device supports subnormal numbers. Knowing whether a library has subnormal support can be beneficial for those libraries concerned about accuracy in the computation of transcendental functions and other domains.

Proposal

This RFC proposes to add support to the inspection API for querying whether an array library/device supports subnormal numbers. Specifically, this RFC proposes updating info.capabilities to support an optional device argument (similar to info.default_dtypes):

def capabilities(*, device: Optional[device] = None) -> Capabilities

Currently, the info.capabilities API is a nullary function returning a fixed object. This RFC proposes allowing that to remain the default behavior, while extending it to support device-specific capabilities.

This RFC proposes adding the following field to the dictionary returned by info.capabilities():

{
  ...
  "max rank": Optional[int],
+ "subnormals": bool
}

When device is None, the value assigned to subnormals should correspond to the default device. When a device is specified, the value should indicate whether subnormals are supported on that particular device.

With this API in place, downstream libraries could query whether a conforming array library and a particular device support subnormal numbers.

Questions

1. In Improve subnormal expectations #339 (comment), another proposal was adding a field to finfo along the lines of is_subnormal_zero to handle the case where smallest_normal and smallest_subnormal are equal. Are the changes in this RFC a better alternative?

2. Another way for array consuming libraries to check for subnormal support is to perform a calculation which should return a subnormal number (e.g., 2.25e-308/100.0 > 0.0). If the result is greater than zero, you have your answer. While not elegant, certainly doable, and array consuming libraries can always write a small helper function to query whenever creating arrays assigned to particular devices. Would the ability to use a dedicated inspection API as proposed in this RFC be more desirable?

[rgommers]

One question that came to mind: what about partial support? If you vendor or depend on some third-party libraries for example, one of which has a "flush to zero" flag (-ftz) in its build config and the other doesn't?

Similarly, what about dtype-specific support? That seems to be the default on CUDA; https://docs.nvidia.com/cuda/floating-point/index.html#compiler-flags says that -ftz affects float32 but not float64.

Another way for array consuming libraries to check for subnormal support is to perform a calculation which should return a subnormal number

Given that the behavior may be device/dtype/function-specific, doing a calculation for the actual operation that is relevant seems more robust.