Skip to content

RFC: rm __array__, add __buffer__ #115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

RFC: rm __array__, add __buffer__ #115

wants to merge 5 commits into from

Conversation

ev-br
Copy link
Member

@ev-br ev-br commented Dec 26, 2024 •
edited
Loading

Try getting rid of __array__, replace with the buffer protocol's __buffer__.

cross-ref #67, #69

This does not seem to fix gh-102. scipy tests pass locally.
Replacing __array__ with __buffer__ makes code marginally simpler. Effectively bumps the python requirement to 3.12+ though.
So maybe this is a temp solution until dlpack matures.
¯\_(ツ)_/¯

@ev-br ev-br mentioned this pull request Dec 26, 2024
Draft
@ev-br
Copy link
Member Author

ev-br commented Dec 26, 2024

The matching SciPy PR: scipy/scipy#22189

@ev-br ev-br force-pushed the buffer_protocol branch 2 times, most recently from df7b4d1 to 4fcb9e6 Compare December 26, 2024 10:42
@ev-br ev-br changed the title WIP: rm __array__, add __buffer__ RFC: rm __array__, add __buffer__ Dec 26, 2024
@ev-br ev-br mentioned this pull request Jan 13, 2025
Open
@ev-br
Copy link
Member Author

ev-br commented Jan 24, 2025

The current behavior: np.asarray(...) just works on python 3.12+, and errors out otherwise.
I'm going to merge this for the upcoming release, unless there are suggestions to the contrary.

On python 3.12:

In [3]: import sys; print(sys.version)
3.12.4 | packaged by conda-forge | (main, Jun 17 2024, 10:23:07) [GCC 12.3.0]

In [4]: np.array(xp.ones(3))
Out[4]: array([1., 1., 1.])

In [5]: np.ones(3) + xp.ones(3)     # can't do anything about
Out[5]: array([2., 2., 2.])

On python 3.11:

In [1]: import sys; print(sys.version)
3.11.0 | packaged by conda-forge | (main, Jan 14 2023, 12:27:40) [GCC 11.3.0]

In [2]: import array_api_strict as xp

In [3]: import numpy as np

In [4]: np.array(xp.ones(3))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[4], line 1
----> 1 np.array(xp.ones(3))

File ~/repos/array-api-strict/array_api_strict/_array_object.py:170, in Array.__array__(self, *args, **kwds)
    168 def __array__(self, *args, **kwds):
    169     # a stub for python < 3.12; otherwise numpy silently produces object arrays
--> 170     raise TypeError(
    171         f"Interoperation with NumPy requires python >= 3.12. Please upgrade."
    172     )

TypeError: Interoperation with NumPy requires python >= 3.12. Please upgrade.

In [5]: xp.ones(3) + np.ones(3)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[5], line 1
----> 1 xp.ones(3) + np.ones(3)

File ~/repos/array-api-strict/array_api_strict/_array_object.py:170, in Array.__array__(self, *args, **kwds)
    168 def __array__(self, *args, **kwds):
    169     # a stub for python < 3.12; otherwise numpy silently produces object arrays
--> 170     raise TypeError(
    171         f"Interoperation with NumPy requires python >= 3.12. Please upgrade."
    172     )

TypeError: Interoperation with NumPy requires python >= 3.12. Please upgrade.

@ev-br ev-br force-pushed the buffer_protocol branch 2 times, most recently from 8115901 to ede45af Compare January 24, 2025 13:08
@ev-br
Copy link
Member Author

ev-br commented Feb 4, 2025

PSA: The timeline for this PR is "land in a couple of month's time". The current thinking is that short-term we work on the 2024.12 support in version 2.3, let the dust settle, and then merge this PR. Downstream testing with array-api-strict will then need to use python >= 3.12.

@ev-br ev-br force-pushed the buffer_protocol branch from 45befd6 to d5397d6 Compare June 10, 2025 11:21
ev-br added 2 commits June 10, 2025 13:28
@ev-br
MAINT: make __array__ raise on python < 3.12
98560b8 
Otherwise, on python 3.11 and below, np.array(array_api_strict_array)
becomes a 0D object array.
@ev-br
CI: bump python versions on CI
12fe35d
@ev-br ev-br force-pushed the buffer_protocol branch from d5397d6 to 12fe35d Compare June 10, 2025 11:28
@ev-br ev-br mentioned this pull request Jun 10, 2025
Open
@ev-br
Copy link
Member Author

ev-br commented Jun 10, 2025

Branch protection strikes again: could you please flip testing on python 3.10 and 3.11 to optional @kgryte @rgommers ?

@ev-br ev-br mentioned this pull request Jun 10, 2025
Open
4 tasks
@ev-br
Copy link
Member Author

ev-br commented Jun 10, 2025 •
edited
Loading

Alternatively, we can make a 2.4 release with what is now in main (#156), and postpone the __array__ story to a dedicated release.

Locally, this branch causes no problems for SciPy (scipy/scipy#23144 adds small fixes from local testing), and scikit-learn/scikit-learn#31517 fixes scikit-learn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@asmeurer asmeurer Awaiting requested review from asmeurer

@rgommers rgommers Awaiting requested review from rgommers

@betatim betatim Awaiting requested review from betatim

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Arithmetic operations accept numpy arrays
1 participant
@ev-br