design for checking something is an array

[rgommers]

This is a question that we may have covered but not clearly enough because it keeps coming up in conversations: "how do I check if something is a compliant array object?" People have asked me, and it has also come up in the Dask tracking issue and the NEP 47 review:

• Python Array API Standard dask/community#109 (comment)
• https://numpy.org/neps/nep-0047-array-api-standard.html#the-asarray-asanyarray-pattern

The current answer is:

• we do not have/want a "reference library" that people need to depend on, hence we cannot do something like create an array ABC that libraries can inherit from to enable an isinstance check.
• the one distinguishing feature of a compliant array object is: it has an __array_namespace__ attribute, so at runtime the check to do is hasattr(x, '__array_namespace__').
• each library that wants to do this check should probably implement a standard function like:

def is_arrayobj(x):
    return hasattr(x, '__array_namespace__')

• we can recommend this somewhere, and give an implementation for people to vendor.
• for static type checking we can do better, design an Array typing Protocol: https://data-apis.org/array-api/latest/design_topics/static_typing.html. This is still to be implemented, but isn't hard to do. Again people should vendor this.

Note that in NEP 47 we (I) messed something up: the initial thought was to make numpy.ndarray the object that was the standards-compliant one - and therefore have a __array_namespace__ attribute. However we decided later to create a separate array object. numpy.ndarray is not compliant, and therefore should not have that attribute. However, that also means there is no way to retrieve the compliant namespace from a regular numpy ndarray instance. Hence it should probably be special-cased:

def is_arrayobj_or_ndarray(x):
    # This is just a sketch
    try:
        import numpy as np
        if isinstance(x, np.ndarray):
            x_new = np.array_api.asarray(x)
            # What the caller will need to do is convert back to ndarray at the end ....
            return True, x_new
    except ImportError:
        pass

    return hasattr(x, '__array_namespace__')

The alternative would be two separate checks; either way we need a design pattern that does (at least for existing libraries like SciPy, which must also support numpy.ndarray):

• check for compliant array object
• check for numpy.ndarray instance
• if ndarray instance, convert to compliant array object
• retrieve namespace
• (do stuff)
• if it was an ndarray instance, convert back to ndarray at the end

[leofang]

Is the consideration here mainly for NEP-47, or for any Array API compliant libraries (or libraries that will accept Array API)? I think for the latter class it is odd to single out numpy.ndarray in the checks as a special case. By definition numpy.ndarray is not a compliant Array, so it should be ok that we don't consider it is. Maybe I am missing something here...

[rgommers]

I'm mainly thinking about libraries that now depend on NumPy (e.g., SciPy). If we have functions in such a library that now accept array_like (= numpy.ndarray, scalars, sequences, generators, anything with __array__ attribute, etc.), they'll in the future have to accept both array_like and objects with __array_namespace__.

My expectation when writing the above comment was that NumPy will have 2 array objects, but other libraries have only 1 (or at least, their main array/tensor object is compliant).