Openblas wheel #88
Openblas wheel #88
Conversation
| # do not package the static libs, they are ~55MB | ||
| rm local/openblas/*.a | ||
| python -m pip wheel -w dist -vv . | ||
| cp local/openblas/lib/libopenblas64_.so local/openblas/libopenblas_python.so | ||
| # do not package the static libs and symlinks, they are ~55MB | ||
| rm -rf local/openblas/lib/* | ||
| mv local/openblas/libopenblas_python.so local/openblas/lib/ | ||
|
|
||
| cat <<EOF > run_in_docker.sh | ||
| cd /openblas | ||
| patchelf --set-soname libopenblas_python.so local/openblas/lib/libopenblas_python.so |
There was a problem hiding this comment.
symlinks don't play well in wheels, they just end up as copies of the original files.
This hacks the soname & filename just to need the 1 file.
It would probably be better to handle this as an openblas build system patch than hacking this with patchelf.
I didn't take care of any ABI versioning in there.
To get a single file working both for runtime & build time, the soname & filename must be the same and look like lib{whatever}.so. It would be nice for this name not to conflict with a local install of openblas.
There was a problem hiding this comment.
makes sense. Where did I use symlinks?
There was a problem hiding this comment.
Where did I use symlinks?
You don't directly. But openblas build does the generic linux thing here, something along the lines of (I don't remember exactly the actual names):
libopenblas.so => symlink to libopenblas.so.0.3.3
libopenblas.so.3 => symlink to libopenblas.so.0.3.3
libopenblas.so.0.3.3 => the actual binary
| PyObject* | ||
| open_so(PyObject *self, PyObject *args) { | ||
| const char *utf8 = PyUnicode_AsUTF8(args); | ||
| if (utf8 == NULL) { | ||
| return NULL; | ||
| } | ||
| void *handle = dlopen(utf8, RTLD_GLOBAL | RTLD_NOW); | ||
| if (handle == NULL) { | ||
| PyErr_SetString(PyExc_ValueError, "Could not open SO"); | ||
| return NULL; | ||
| } | ||
| Py_RETURN_TRUE; | ||
| } | ||
|
|
There was a problem hiding this comment.
It's not necessary to dlopen this globally for numpy to find the symbols.
Just importing openblas is enough.
There was a problem hiding this comment.
Just importing openblas is enough.
How does this work with NumPy? Could you link to your numpy branch?
There was a problem hiding this comment.
Ahh, got it
git clone https://github.com/mayeut/numpy.git numpy
cd numpy
git checkout openblas-wheel
|
Adopted parts of this in #87, thanks. Closing. |
I wanted to check some of the discussions that happened in the auditwheel repo regarding the
--excludeoption so I hacked something together based on @mattip's work in #87I don't plan to do anything more than this but thought I'd share what I've done in a draft PR as it differs a bit from the work in #87.
The PR also builds numpy from a branch in my fork just for a quick & dirty PoC.
Feel free to just close or re-use anything in here.