ENH: add testing support utilities

[ev-br]

SciPy has two sets of testing support utilities:

• {skip_xfail}_xp_backends skips, https://github.com/scipy/scipy/blob/main/scipy/conftest.py#L215 . These are convenient for testing across the range of array api backends. Note that these are pytest-specific.
• assertions, xp_assert_{equal,close}, https://github.com/scipy/scipy/blob/main/scipy/_lib/_array_api.py#L299. These wrap backend-specific assertions, and scipy effectively transitions from np.testing to these ones. Quite a bit of effort went into crafting these ones, and ISTM these can be useful across the ecosystem.

[lucascolley]

These utilities are specific to the libraries we test with in SciPy, so I don't think they would fit into the current scope of this library.

I think it may be possible to generalise {skip_xfail}_xp_backends over devices and skip based on https://data-apis.org/array-api/draft/API_specification/generated/array_api.info.capabilities.html rather than individual backends. But we would probably still want something separate in SciPy which is able to deal with specific implementations.

Since there are no xp.testing primitives in the standard, I don't see how we would add xp_assert_{equal,close} in an agnostic way.

I agree that these utilities should be pulled out of SciPy into some package which other consuming libraries can re-use. But I think that it would be useful to keep them separate from array-api-extra to avoid confusion, by keeping this library as purely building on top of the standard.

[asmeurer]

I discussed this with Evgeni a few weeks ago, and based on his description of the SciPy tooling, I suggested that these utilities be split out into a separate package that is a pytest extension (e.g., called something like pytest-arrays or pytest-array-api).

[ev-br]

To gauge interest of other downstream adopters: @betatim would you consider using scipy's @skip_xp_backends machinery and/or backend-agnostic assertions, xp_assert_{close,equal} in your scikit-learn work if they were in a separate package, as suggested above?
(We can take this discussion elsewhere if that'd be more convenient)

[betatim]

A bit tricky to judge in the abstract.

Right now we convert all outputs back to Numpy arrays and then use the "standard" all close, equal, etc tooling. So in principle having a xp_assert_close could be nice, but in practice we have already solved this problem. And the current implementation isn't causing us any trouble - don't change what ain't broken?

For skipping particular namespaces: we use a decorator to generate combinations of xp, dtype, device and for now all tests run on all combinations (I think). So for now maybe there is no need for skip_xp_backends for scikit-learn?

[crusaderky]

#88 adds xp_assert_equal and xp_assert_close to array-api-extra, for private use only, at least for now.

The xpx version misses a bunch of bells and whistles that the scipy version has:

• no default_ctx context
• no ability to cherry-pick disabling shape, dtype, and/or namespace validation
• no special casing for 0d

I was keen to avoid scope creep in #88, so I stuck to the mvp for array-api-extra itself.
Feel free to expand them to be able to replace those in scipy or move the whole thing to a separate package (which would become a test dependency of xpx, I guess).

[lucascolley]

also, since I commented above, the scope of xpx has expanded, so these are probably no longer out of scope.

[ev-br]

If anything, ISTM it'd be best to move xp_assert functions wholesale, as they are. 0d versions can either move or stay, of course. Not having two nearly but not quite identical functions in two packages is best avoided though.