RFC: rm array, add buffer #115

ev-br · 2024-12-26T09:58:01Z

Try getting rid of __array__, replace with the buffer protocol's __buffer__.

cross-ref #67, #69

This does not seem to fix gh-102. scipy tests pass locally.
Replacing __array__ with __buffer__ makes code marginally simpler. Effectively bumps the python requirement to 3.12+ though.
So maybe this is a temp solution until dlpack matures.
¯\_(ツ)_/¯

ev-br · 2024-12-26T10:00:11Z

The matching SciPy PR: scipy/scipy#22189

ev-br · 2025-01-24T13:04:01Z

The current behavior: np.asarray(...) just works on python 3.12+, and errors out otherwise.
I'm going to merge this for the upcoming release, unless there are suggestions to the contrary.

On python 3.12:

In [3]: import sys; print(sys.version)
3.12.4 | packaged by conda-forge | (main, Jun 17 2024, 10:23:07) [GCC 12.3.0]

In [4]: np.array(xp.ones(3))
Out[4]: array([1., 1., 1.])

In [5]: np.ones(3) + xp.ones(3)     # can't do anything about
Out[5]: array([2., 2., 2.])

On python 3.11:

In [1]: import sys; print(sys.version)
3.11.0 | packaged by conda-forge | (main, Jan 14 2023, 12:27:40) [GCC 11.3.0]

In [2]: import array_api_strict as xp

In [3]: import numpy as np

In [4]: np.array(xp.ones(3))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[4], line 1
----> 1 np.array(xp.ones(3))

File ~/repos/array-api-strict/array_api_strict/_array_object.py:170, in Array.__array__(self, *args, **kwds)
    168 def __array__(self, *args, **kwds):
    169     # a stub for python < 3.12; otherwise numpy silently produces object arrays
--> 170     raise TypeError(
    171         f"Interoperation with NumPy requires python >= 3.12. Please upgrade."
    172     )

TypeError: Interoperation with NumPy requires python >= 3.12. Please upgrade.

In [5]: xp.ones(3) + np.ones(3)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[5], line 1
----> 1 xp.ones(3) + np.ones(3)

File ~/repos/array-api-strict/array_api_strict/_array_object.py:170, in Array.__array__(self, *args, **kwds)
    168 def __array__(self, *args, **kwds):
    169     # a stub for python < 3.12; otherwise numpy silently produces object arrays
--> 170     raise TypeError(
    171         f"Interoperation with NumPy requires python >= 3.12. Please upgrade."
    172     )

TypeError: Interoperation with NumPy requires python >= 3.12. Please upgrade.

ev-br · 2025-02-04T09:21:41Z

PSA: The timeline for this PR is "land in a couple of month's time". The current thinking is that short-term we work on the 2024.12 support in version 2.3, let the dust settle, and then merge this PR. Downstream testing with array-api-strict will then need to use python >= 3.12.

Otherwise, on python 3.11 and below, np.array(array_api_strict_array) becomes a 0D object array.

ev-br · 2025-06-10T13:28:59Z

Branch protection strikes again: could you please flip testing on python 3.10 and 3.11 to optional @kgryte @rgommers ?

ev-br · 2025-06-10T14:18:35Z

Alternatively, we can make a 2.4 release with what is now in main (#156), and postpone the __array__ story to a dedicated release.

Locally, this branch causes no problems for SciPy (scipy/scipy#23144 adds small fixes from local testing), and scikit-learn/scikit-learn#31517 fixes scikit-learn.

betatim · 2025-06-11T12:58:28Z

array_api_strict/_array_object.py

    # This was implemented historically for compatibility, and removing it has
    # caused issues for some libraries (see
    # https://github.com/data-apis/array-api-strict/issues/67).


Should we remove this comment as well or edit it to mention that it is referring to __array__ - because "this was implemented ..." will be weird. Not sure we need to keep it/combine it with the comment below for __buffer__, so delete?

betatim · 2025-06-11T13:14:44Z

array_api_strict/_array_object.py

+        if self._device != CPU_DEVICE:
+            raise RuntimeError(f"Can not convert array on the '{self._device}' device to a Numpy array.")
+        return memoryview(self._array)
+    def __release_buffer(self, buffer):


Suggested change

def __release_buffer(self, buffer):

def __release_buffer__(self, buffer):

betatim · 2025-06-11T13:32:48Z

array_api_strict/_array_object.py

+    def __buffer__(self, flags):
+        if self._device != CPU_DEVICE:
+            raise RuntimeError(f"Can not convert array on the '{self._device}' device to a Numpy array.")
+        return memoryview(self._array)


For my education: why not use return self._array.__buffer__(flags) - aka delegate to numpy for this? And the same for __release_buffer__

ev-br requested review from asmeurer, rgommers and betatim December 26, 2024 09:58

ev-br mentioned this pull request Dec 26, 2024

CI: array api: test arrays-api-strict getting rid of __array__ scipy/scipy#22189

Draft

ev-br force-pushed the buffer_protocol branch 2 times, most recently from df7b4d1 to 4fcb9e6 Compare December 26, 2024 10:42

ev-br changed the title ~~WIP: rm __array__, add __buffer__~~ RFC: rm __array__, add __buffer__ Dec 26, 2024

ev-br mentioned this pull request Jan 13, 2025

Should we be implementing __array__? #67

Open

ev-br force-pushed the buffer_protocol branch from 4fcb9e6 to a821bb9 Compare January 24, 2025 12:55

ev-br force-pushed the buffer_protocol branch 2 times, most recently from 8115901 to ede45af Compare January 24, 2025 13:08

ev-br force-pushed the buffer_protocol branch from 6961ee7 to 45befd6 Compare February 4, 2025 08:56

ev-br added 2 commits June 10, 2025 13:16

ENH: rm __array__, add __buffer__

45c2d1f

TST: adapt tests for the lack of __array__

9f232b7

ev-br force-pushed the buffer_protocol branch from 45befd6 to d5397d6 Compare June 10, 2025 11:21

ev-br added 2 commits June 10, 2025 13:28

MAINT: make __array__ raise on python < 3.12

98560b8

Otherwise, on python 3.11 and below, np.array(array_api_strict_array) becomes a 0D object array.

CI: bump python versions on CI

12fe35d

ev-br force-pushed the buffer_protocol branch from d5397d6 to 12fe35d Compare June 10, 2025 11:28

ev-br mentioned this pull request Jun 10, 2025

Preemptively fix incompatibilities with an upcoming array-api-strict release scikit-learn/scikit-learn#31517

Open

MAINT: linter

41606bb

ev-br mentioned this pull request Jun 10, 2025

WIP: Release 2.4 branch #156

Open

4 tasks

betatim reviewed Jun 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

RFC: rm array, add buffer #115

RFC: rm array, add buffer #115

Uh oh!

ev-br commented Dec 26, 2024 •

edited

Loading

Uh oh!

ev-br commented Dec 26, 2024

Uh oh!

ev-br commented Jan 24, 2025

Uh oh!

ev-br commented Feb 4, 2025

Uh oh!

ev-br commented Jun 10, 2025

Uh oh!

ev-br commented Jun 10, 2025 •

edited

Loading

Uh oh!

betatim Jun 11, 2025

Uh oh!

betatim Jun 11, 2025

Uh oh!

betatim Jun 11, 2025

Uh oh!

Uh oh!

	def __release_buffer(self, buffer):
	def __release_buffer__(self, buffer):

RFC: rm __array__, add __buffer__ #115

Are you sure you want to change the base?

RFC: rm __array__, add __buffer__ #115

Uh oh!

Conversation

ev-br commented Dec 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ev-br commented Dec 26, 2024

Uh oh!

ev-br commented Jan 24, 2025

Uh oh!

ev-br commented Feb 4, 2025

Uh oh!

ev-br commented Jun 10, 2025

Uh oh!

ev-br commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

betatim Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

betatim Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

betatim Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RFC: rm array, add buffer #115

RFC: rm array, add buffer #115

ev-br commented Dec 26, 2024 •

edited

Loading

ev-br commented Jun 10, 2025 •

edited

Loading