Summary

When I try to run numpy functions on arrays with more than 2**31 -1 elements, I get the following error:

Intel oneMKL ERROR: Parameter 1 was incorrect on entry to vdAbs.

To me, this suggests that the size of the numpy array is stored in an int32 variable.

To reproduce

I'm using Python 3.12.2 on Ubuntu 24.04 for all examples shown below.

$ python -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip install --index-url https://software.repos.intel.com/python/pypi numpy
# installed numpy-1.26.4-18 and some other mkl libraries

>>> import numpy as np
>>> n = 2**31
>>> data = np.ones(n)
>>> np.abs(data)

Intel oneMKL ERROR: Parameter 1 was incorrect on entry to vdAbs.
array([0., 0., 0., ..., 0., 0., 0.])
>>> np.abs(data[:-1])
array([1., 1., 1., ..., 1., 1., 1.])

Behaviour with "regular" (non-MKL) numpy

$ python -m venv .venv_no_mkl
$ source .venv_no_mkl/bin/activate
(.venv_no_mkl) $ pip install numpy
# installed numpy 2.2.5
# I also tried numpy==1.26.4 to match the version of the OneMKL numpy. The behaviour was the same

>>> import numpy as np
>>> n = 2**31
>>> data = np.ones(n)
>>> np.abs(data)
array([1., 1., 1., ..., 1., 1., 1.], shape=(2147483648,))
>>> np.abs(data[:-1])
array([1., 1., 1., ..., 1., 1., 1.], shape=(2147483647,))

Desired Behaviour

I guess ideally the MKL numpy implementation would mirror the behaviour of non-MKL numpy. Storing the actual data does not seem to be a problem, so maybe the data type of the variable containing the array size can be upgraded to an int64?

Should that not be possible for some reason, I think it would be very helpful to raise a Python error (that would have made our debugging much easier 😁 ). Maybe something like:

ValueError: Array-size cannot exceed 2**31-1 elements

or something like that.

Thanks a lot in advance! 🙇 I hope this issue is within the scope of this repository, but I couldn't find a dedicated repo for the MKL numpy implementation.