Test failures on debian sid

[avalentino]

Hi, I'm testing the current pandas master on debian sid with updated numexpr and PyTables from experimental and I'm experimenting some test failures:

$ env PYTHONPATH=. nosetests pandas
.................................EES.FSSS......................................................S.............F..........................SSSS...SS.......................S........................................................................................................................................................................................................................................................................................................................................................S.......................................................................................................................SSSS..........................S.............................................................SSSSSSSSSS....................SS...................................................................................................................................................S...............................S.......................SSSSSSS.........................................F..............................................S....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................can't invoke "event" command:  application has been destroyed
    while executing
"event generate $w <<ThemeChanged>>"
    (procedure "ttk::ThemeChanged" line 6)
    invoked from within
"ttk::ThemeChanged"
E...................................................................................................................................................................................................................................................S........................................................................................................................................................................S..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.................................................................................S.S...............S.S............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.............................................................................................S..............................................................................................................................................................................S...S............SSS.........S............SS............SS.....SSSS.................................................................................................................................................................................................................S.......................................................................................................................................................................................................................
======================================================================
ERROR: test_get_goog_volume (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/util/testing.py", line 857, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas/pandas/io/tests/test_data.py", line 55, in test_get_goog_volume
    self.assertEqual(df.Volume.ix['OCT-08-2010'], 2863473)
  File "/home/antonio/pandas/pandas/core/indexing.py", line 54, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/home/antonio/pandas/pandas/core/indexing.py", line 631, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/home/antonio/pandas/pandas/core/indexing.py", line 1273, in _get_label
    return self.obj[key]
  File "/home/antonio/pandas/pandas/core/series.py", line 920, in __getitem__
    return self.index.get_value(self, key)
  File "/home/antonio/pandas/pandas/tseries/index.py", line 1184, in get_value
    raise KeyError(key)
KeyError: 'OCT-08-2010'
-------------------- >> begin captured stdout << ---------------------
Failed: KeyError('OCT-08-2010',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_multi1 (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/util/testing.py", line 857, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas/pandas/io/tests/test_data.py", line 68, in test_get_multi1
    testit()
  File "/home/antonio/pandas/pandas/io/tests/test_data.py", line 64, in testit
    self.assertEquals(ts[0].dayofyear, 96)
  File "/home/antonio/pandas/pandas/tseries/index.py", line 1298, in __getitem__
    val = arr_idx[key]
IndexError: index 0 is out of bounds for axis 0 with size 0
-------------------- >> begin captured stdout << ---------------------
Failed: IndexError('index 0 is out of bounds for axis 0 with size 0',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_axis_limits (pandas.tseries.tests.test_plotting.TestTSPlot)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/tseries/tests/test_plotting.py", line 344, in test_axis_limits
    _test(ax)
  File "/home/antonio/pandas/pandas/tseries/tests/test_plotting.py", line 308, in _test
    ax.get_figure().canvas.draw()
  File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 349, in draw
    tkagg.blit(self._tkphoto, self.renderer._renderer, colormode=2)
  File "/usr/lib/pymodules/python2.7/matplotlib/backends/tkagg.py", line 20, in blit
    tk.call("PyAggImagePhoto", photoimage, id(aggimage), colormode, id(bbox_array))
TclError: this isn't a Tk application

======================================================================
FAIL: test_google (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/util/testing.py", line 857, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas/pandas/io/tests/test_data.py", line 42, in test_google
    13.68)
AssertionError: 17.66 != 13.68
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('17.66 != 13.68',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: test_invalid_url (pandas.io.tests.test_html.TestReadHtmlBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/util/testing.py", line 857, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas/pandas/io/tests/test_html.py", line 299, in test_invalid_url
    'http://www.a23950sdfa908sd.com')
AssertionError: No tables found
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('No tables found',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: testWLS (pandas.stats.tests.test_ols.TestOLS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/stats/tests/test_ols.py", line 111, in testWLS
    self._check_wls(X, Y, weights)
  File "/home/antonio/pandas/pandas/stats/tests/test_ols.py", line 134, in _check_wls
    self.checkMovingOLS('rolling', x, y, weights=weights)
  File "/home/antonio/pandas/pandas/stats/tests/test_ols.py", line 208, in checkMovingOLS
    result_index=n)
  File "/home/antonio/pandas/pandas/stats/tests/test_ols.py", line 247, in compare
    assert_almost_equal(ref, res)
  File "/home/antonio/pandas/pandas/util/testing.py", line 198, in assert_almost_equal
    assert_almost_equal(a[i], b[i], check_less_precise)
  File "/home/antonio/pandas/pandas/util/testing.py", line 227, in assert_almost_equal
    1, a / b, decimal=decimal, err_msg=err_msg(a, b), verbose=False)
  File "/usr/lib/pymodules/python2.7/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 5 decimals expected 6.61194 but got 7.66003
-------------------- >> begin captured stdout << ---------------------
Make sure you're using statsmodels 0.5.0.dev-cec4f26 or later.

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 3933 tests in 407.389s

FAILED (SKIP=64, errors=3, failures=3)

Some cfg info:

PyTables version:  3.0.0
HDF5 version:      1.8.11
NumPy version:     1.7.1
Numexpr version:   2.2 (not using Intel's VML/MKL)
Python version:    2.7.5+ (default, Sep 17 2013, 15:31:50)  [GCC 4.8.1]
Platform:          linux2-x86_64

[jreback]

@avalentino try with PyTables 2.4...!

[cpcloud]

@avalentino Any reason why you're setting PYTHONPATH to the current directory?

[cpcloud]

@avalentino Can you show the output of

ci/print_versions.py

[avalentino]

@avalentino try with PyTables 2.4...!

@jreback e are working to push PyTables 3.0 (and numexpr 2.2) in sid and my purpose is to check if the new version of pytables breaks some of the dependent packages.

@avalentino Any reason why you're setting PYTHONPATH to the current directory?

@cpcloud no, mainly a question of habit

[avalentino]

@avalentino Any reason why you're setting PYTHONPATH to the current directory?

@cpcloud no, mainly a question of habit

well, I'm running tests from a checkout built with

$  python setup.py build_ext --inplace

[avalentino]

@avalentino Can you show the output of
ci/print_versions.py

$ ci/print_versions.py 

INSTALLED VERSIONS
------------------
Python: 2.7.5.final.0
OS: Linux 3.10-3-amd64 #1 SMP Debian 3.10.11-1 (2013-09-10) x86_64
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8

pandas: Not installed
Cython: 0.19.1
Numpy: 1.7.1
Scipy: 0.12.0
statsmodels: 0.4.2
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 1.5
pytz: 2012c
bottleneck: Not installed
PyTables: 3.0.0
    numexpr: 2.2
matplotlib: 1.3.0
openpyxl: 1.5.8
xlrd: 0.9.2
xlwt: 0.7.4
sqlalchemy: Not installed
lxml: Not installed
bs4: 4.2.1
html5lib: 0.95-dev

[cpcloud]

Okay now can you show the output of git describe? I'm guess the fact that pandas isn't "installed" might have something do with it, although I don't see why since you should be able to test it if you've built things....

If you do python setup.py develop nothing except a link to the pandas directory will be added to your site-packages, and pandas will show up as installed....is that a problem?

[avalentino]

Okay now can you show the output of git describe? I'm guess the fact that pandas isn't "installed" might have something do with it, although I don't see why since you should be able to test it if you've built things....

$ git describe 
v0.12.0-501-g1675c2a

If you do python setup.py develop nothing except a link to the pandas directory will be added to your site-packages, and pandas will show up as installed....is that a problem?

$ python setup.py clean --all
$ python setup.py install --user

[...]

$ ci/print_versions.py 

INSTALLED VERSIONS
------------------
Python: 2.7.5.final.0
OS: Linux 3.10-3-amd64 #1 SMP Debian 3.10.11-1 (2013-09-10) x86_64
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8

pandas: 0.12.0-501-g1675c2a
Cython: 0.19.1
Numpy: 1.7.1
Scipy: 0.12.0
statsmodels: 0.4.2
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 1.5
pytz: 2012c
bottleneck: Not installed
PyTables: 3.0.0
    numexpr: 2.2
matplotlib: 1.3.0
openpyxl: 1.5.8
xlrd: 0.9.2
xlwt: 0.7.4
sqlalchemy: Not installed
lxml: Not installed
bs4: 4.2.1
html5lib: 0.95-dev

Unfortunately now I'm no more able to run the test suite:

$ nosetests pandas
E
======================================================================
ERROR: Failure: ImportError (cannot import name hashtable)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 413, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/antonio/pandas/pandas/__init__.py", line 6, in <module>
    from . import hashtable, tslib, lib
ImportError: cannot import name hashtable

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

$ cd ~/tmp
$ nosetests pandas

----------------------------------------------------------------------
Ran 0 tests in 0.001s

OK

[cpcloud]

@avalentino Would you be willing to set up a virtualenv? Things will progress much quicker if you can isolate your Python version, packages, etc....I can help you set one up if necessary

[avalentino]

@cpcloud OK, I managed to setup a virtual environment with:

$ virtualenv --system-site-packages ssp

and then I installed pandas from the check-out directory (updated this mourning).
Now I have a larger umber of tests that fail.

(ssp)antonio@mac4:~/ssp$ ../pandas/ci/print_versions.py 

INSTALLED VERSIONS
------------------
Python: 2.7.5.final.0
OS: Linux 3.10-3-amd64 #1 SMP Debian 3.10.11-1 (2013-09-10) x86_64
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8

pandas: 0.12.0-513-g5b0db94
Cython: 0.19.1
Numpy: 1.7.1
Scipy: 0.12.0
statsmodels: 0.4.2
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 1.5
pytz: 2012c
bottleneck: Not installed
PyTables: 3.0.0
    numexpr: 2.2
matplotlib: 1.3.0
openpyxl: 1.5.8
xlrd: 0.9.2
xlwt: 0.7.4
sqlalchemy: Not installed
lxml: Not installed
bs4: 4.2.1
html5lib: 0.95-dev

$ nosetests lib/python2.7/site-packages/pandas
................................................................................S................S........................................................................................................................................................EES.FSSS......................................................S.............F..........................SSSS...SS.........E........E....S.........E................................E.............................................................................................................................................................................................................................................................................................................S.......................................................................................................................SSSS..........................S.............................................................SSSSSSSSSS....................SS...................................................................................................................................................S...............................S.......................SSSSSSS.........................................F..............................................S........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S........................................................................................................................................................................S..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.................................................................................S.S...............S.S............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.............................................................................................S..............................................................................................................................................................................S...S............SSS.........S............SS............SS.....SSSS.................................................................................................................................................................................................................S.......................................................................................................................................................................................................................
======================================================================
ERROR: test_get_goog_volume (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_data.py", line 55, in test_get_goog_volume
    self.assertEqual(df.Volume.ix['OCT-08-2010'], 2863473)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/core/indexing.py", line 54, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/core/indexing.py", line 631, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/core/indexing.py", line 1273, in _get_label
    return self.obj[key]
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/core/series.py", line 920, in __getitem__
    return self.index.get_value(self, key)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/tseries/index.py", line 1184, in get_value
    raise KeyError(key)
KeyError: 'OCT-08-2010'
-------------------- >> begin captured stdout << ---------------------
Failed: KeyError('OCT-08-2010',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_multi1 (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_data.py", line 68, in test_get_multi1
    testit()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_data.py", line 64, in testit
    self.assertEquals(ts[0].dayofyear, 96)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/tseries/index.py", line 1298, in __getitem__
    val = arr_idx[key]
IndexError: index 0 is out of bounds for axis 0 with size 0
-------------------- >> begin captured stdout << ---------------------
Failed: IndexError('index 0 is out of bounds for axis 0 with size 0',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_frame_from_json_precise_float (pandas.io.tests.test_json.test_pandas.TestPandasContainer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_json/test_pandas.py", line 391, in test_frame_from_json_precise_float
    result = read_json(df.to_json(), precise_float=True)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 180, in read_json
    date_unit).parse()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 248, in parse
    self._parse_no_numpy()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 466, in _parse_no_numpy
    loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Unexpected character in found when decoding object value

======================================================================
ERROR: test_series_from_json_precise_float (pandas.io.tests.test_json.test_pandas.TestPandasContainer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_json/test_pandas.py", line 386, in test_series_from_json_precise_float
    result = read_json(s.to_json(), typ='series', precise_float=True)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 187, in read_json
    date_unit).parse()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 248, in parse
    self._parse_no_numpy()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 401, in _parse_no_numpy
    loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Unexpected character in found when decoding object value

======================================================================
ERROR: testFloatMax (pandas.io.tests.test_json.test_ujson.NumpyJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_json/test_ujson.py", line 926, in testFloatMax
    assert_approx_equal(np.float(ujson.decode(ujson.encode(num, double_precision=15))), num, 15)
ValueError: Trailing data

======================================================================
ERROR: test_decimalDecodeTestPrecise (pandas.io.tests.test_json.test_ujson.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_json/test_ujson.py", line 95, in test_decimalDecodeTestPrecise
    decoded = ujson.decode(encoded, precise_float=True)
ValueError: Unexpected character in found when decoding object value

======================================================================
FAIL: test_google (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_data.py", line 42, in test_google
    13.68)
AssertionError: 17.390000000000001 != 13.68
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('17.390000000000001 != 13.68',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: test_invalid_url (pandas.io.tests.test_html.TestReadHtmlBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_html.py", line 299, in test_invalid_url
    'http://www.a23950sdfa908sd.com')
AssertionError: No tables found
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('No tables found',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: testWLS (pandas.stats.tests.test_ols.TestOLS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/stats/tests/test_ols.py", line 111, in testWLS
    self._check_wls(X, Y, weights)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/stats/tests/test_ols.py", line 134, in _check_wls
    self.checkMovingOLS('rolling', x, y, weights=weights)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/stats/tests/test_ols.py", line 208, in checkMovingOLS
    result_index=n)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/stats/tests/test_ols.py", line 247, in compare
    assert_almost_equal(ref, res)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 225, in assert_almost_equal
    assert_almost_equal(a[i], b[i], check_less_precise)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 254, in assert_almost_equal
    1, a / b, decimal=decimal, err_msg=err_msg(a, b), verbose=False)
  File "/usr/lib/pymodules/python2.7/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 5 decimals expected 1.76645 but got 1.72056
-------------------- >> begin captured stdout << ---------------------
Make sure you're using statsmodels 0.5.0.dev-cec4f26 or later.

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 4150 tests in 764.420s

FAILED (SKIP=66, errors=6, failures=3)

[jreback]

@avalentino can you show the latest entry in the git log

[jreback]

cc @Komnomnomnom, this is on mac, so maybe something new for the json issues?

[cpcloud]

are u sure it's on a mac? mac paths usually don't look like that

[cpcloud]

@avalentino can you run the tests from the cloned directory (instead of nosetests lib/python2.7/site-packages/pandas do nosetests /path/to/pandas)?

[cpcloud]

@avalentino so to recap:

# assuming you're in the virtualenv
git clone git://github.com/pydata/pandas.git
python setup.py develop
nosetests -w pandas