error importing Pandas in python-3.4 unix environment - undefined symbol: pyobject_cmp

[dylkot]

I was using Pandas version 0.16.1 on a Red Hat Enterprise Linux Server release 6.6 system and it was working fine but I wanted to update Pandas to version 0.16.2 so I uninstalled it and reinstalled it using

pip uninstall pandas
pip install pandas

The installation seemed to work fine. However, when I attempt to import pandas

import pandas

I get the error:

Traceback (most recent call last):
File "path_to_python/python3.4/site-packages/pandas/__init__.py", line 7, in <module>
from pandas import hashtable, tslib, lib
ImportError: path_to_python/python3.4/site-packages/pandas/hashtable.cpython-34m.so: undefined symbol: pyobject_cmp

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "path_to_python/python3.4/site-packages/pandas/__init__.py", line 13, in <module>
"extensions first.".format(module))
ImportError: C extension: path_to_python/python3.4/site-packages/pandas/hashtable.cpython-34m.so:
undefined symbol: pyobject_cmp not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace' to build the C extensions first.

However:

cd path_to_python/python3.4/site-packages/pandas/
python setup.py build_ext --inplace

doesn't work either as there is no setup.py to be found. The error is:

Fatal Python error: Py_Initialize: can't initialize sys standard streams
AttributeError: 'module' object has no attribute 'OpenWrapper'
Aborted (core dumped)

I also tried to install Pandas by downloading the source and then:

python setup.py install
python setup.py build_ext --inplace

This time the second command generated a lot of output and successfully completed. However, I still get the same error when I try to import pandas

[jreback]

pls show your os / version.

[dylkot]

Using Red Hat Enterprise Linux Server release 6.6

[jreback]

if you are using the system level python (and maybe numpy), then pip install may not always work. Generally recommend to look at Anaconda/miniconda to prevent issues like this.

[dylkot]

I see. I hadn't had a problem at all before using virtualenv and the pip that comes with that. Using Anaconda/miniconda is a bit of a challenge on my work's computing server where I am not a root user.

[jreback]

conda does NOT require you be root, that's part of the point.

[dylkot]

So is pip no longer supported for Pandas? I can switch to conda if necessary.

[jreback]

of course pip is supported. I suspect you have something wonky going on in your 3.4 build. (the python and/or numpy)

[g2boojum]

I just hit this on Gentoo w/ python 3.5.1. I'm using the system python, but I don't have system versions of numpy or pandas. Instead, I had pandas installed in a virtualenv, along with a considerable number of other packages. I noticed the problem when re-running a jupyter notebook crashed on the pandas import.

So, I killed the pip cache directory, created a clean virtualenv, and updated pip & wheel, and installed cython, numpy, python-dateutil, pytz, and pandas. No joy:

(foo)feynman Downloads > python
Python 3.5.1 (default, Jan 20 2016, 11:02:25) 
[GCC 5.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
  File "/home/grant/Envs/foo/lib/python3.5/site-packages/pandas/__init__.py", line 7, in <module>
    from pandas import hashtable, tslib, lib
ImportError: /home/grant/Envs/foo/lib/python3.5/site-packages/pandas/hashtable.cpython-35m-x86_64-linux-gnu.so: undefined symbol: pyobject_cmp

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/grant/Envs/foo/lib/python3.5/site-packages/pandas/__init__.py", line 13, in <module>
    "extensions first.".format(module))
ImportError: C extension: /home/grant/Envs/foo/lib/python3.5/site-packages/pandas/hashtable.cpython-35m-x86_64-linux-gnu.so: undefined symbol: pyobject_cmp not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace' to build the C extensions first.

I've also tried building pandas from source, also to no avail.

I'm not necessarily opposed to using miniconda (that's what I'm doing now, so I can get some actual work done), but I've been using pandas within a virtualenv for quite some time now, and I'm used to it. I'd love to have a better idea of what is going wrong so that I can try to fix it.

Regards, and thanks for an awesome, awesome package.

[jreback]

looks like a pathing issue. pip should work for you (make sure you have an updated version). make sure that sys.path shows correct for you.

[g2boojum]

Well, darn. The thought that it might be a pathing problem was really exciting. I did, in fact, have some junk in my PYTHONPATH that didn't need to be there. Getting rid of it didn't solve the problem, though.

(foo)feynman ~ > python
Python 3.5.1 (default, Jan 20 2016, 11:02:25) 
[GCC 5.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/grant/Envs/foo/lib64/python35.zip', '/home/grant/Envs/foo/lib64/python3.5', '/home/grant/Envs/foo/lib64/python3.5/plat-linux', '/home/grant/Envs/foo/lib64/python3.5/lib-dynload', '/usr/lib64/python3.5', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-linux', '/home/grant/Envs/foo/lib/python3.5/site-packages']

That certainly looks reasonable. Shrug

My user account is pretty crufty, though, so I created a new user with an unsullied account, and created a new virtualenv to see what would happen. I still get the undefined symbol error for pyobject_cmp. So, whatever is going on, I think it's not a simple pathing issue. (It might be a complicated one, however.)

I have to get some real work done, but I'll try looking at this some more later in the week.

Thanks!

[KetothXupack]

Same problem here.

I've tried to install pandas using --global-option='build_ext' --global-option='--inplace' but it did not work either.

[KetothXupack]

I found that my problem was related to clang compiler flags. So CFLAGS='-fgnu89-inline' nailed it.

I also can confirm that fix from #10510 is not enough because I reproduced the same problem on Linux.

[jreback]

can u build master - #12123 seems to fix for the FreeBSD guys

[KetothXupack]

I've tested b26fac8 and it works!