TST: assert_series/frame not comparing for categoricals

[jreback]

xref dask/partd#7

In [1]: df1 = DataFrame({'A' : Series(list('aba')).astype('category', categories=list('ab'), ordered=True)})

In [2]: df2 = DataFrame({'A' : Series(list('aba')).astype('category', categories=list('ba'), ordered=True)})

In [3]: tm.assert_frame_equal(df1, df2)

In [11]: df2 = DataFrame({'A' : Series(list('aba')).astype('category', categories=list('ab'), ordered=False)})

In [12]: tm.assert_frame_equal(df1, df2)

In [13]: df2 = DataFrame({'A' : Series(list('aba')).astype('category', categories=list('ba'), ordered=False)})

In [14]: tm.assert_frame_equal(df1, df2)

all of these should fail as the ordered categoricals don't compare equal in the categories, and ordered is not being compared (nor categories)

Furthermore the assert_series_equal fail for the same.

assert_categorical_equal DOES the right thing, but not being called here. (not that this should say categories are not equal?)

In [17]: tm.assert_categorical_equal(df1.A.values, df2.A.values)
AssertionError: Index are different

Index values are different (100.0 %)
[left]:  Index([u'a', u'b'], dtype='object')
[right]: Index([u'b', u'a'], dtype='object')

[jreback]

cc @sinhrks (on the assert_categorical_equal question)

[sinhrks]

Correct. In addition, CategoricalIndex should also check internal Categorical.

I've once tried this, and found lots of errors including stata. Needs some time to check it one by one...

• https://github.com/pydata/pandas/compare/master...sinhrks:test_strict?expand=1

[jreback]

maybe lets do in 2 stages
first fix assert_categorical_equal then series/frame

[sinhrks]

Let me clarify... 2 stages are:

1. Fix assert_categorical_equal to show correct assertion message (say categories are different or something)
2. Fix assert_frame/series/index_equal to call assert_categorical_equal internally.

[jreback]

yep might be less invasive that way

and for series/frame could have an option that we gradually turn on so can do the change in pieces (not ideal but could be lots of test comparison issues)