Skip to content

reindex of empty CategoricalIndex sometimes fails if target is not a CategoricalIndex #28690

Closed
@batterseapower

Description

@batterseapower

Code Sample, a copy-pastable example if possible

This fails with AttributeError: 'Index' object has no attribute 'codes':

pd.DataFrame(columns=pd.CategoricalIndex([], categories=['A'])).reindex(columns=pd.Index(['A']))

The backtrace is:

c:\python36\lib\site-packages\pandas\util\_decorators.py in wrapper(*args, **kwargs)
    195         @wraps(func)
    196         def wrapper(*args, **kwargs):
--> 197             return func(*args, **kwargs)
    198 
    199         if not PY2:

c:\python36\lib\site-packages\pandas\core\frame.py in reindex(self, *args, **kwargs)
   3807         kwargs.pop('axis', None)
   3808         kwargs.pop('labels', None)
-> 3809         return super(DataFrame, self).reindex(**kwargs)
   3810 
   3811     @Appender(_shared_docs['reindex_axis'] % _shared_doc_kwargs)

c:\python36\lib\site-packages\pandas\core\generic.py in reindex(self, *args, **kwargs)
   4354         # perform the reindex on the axes
   4355         return self._reindex_axes(axes, level, limit, tolerance, method,
-> 4356                                   fill_value, copy).__finalize__(self)
   4357 
   4358     def _reindex_axes(self, axes, level, limit, tolerance, method, fill_value,

c:\python36\lib\site-packages\pandas\core\frame.py in _reindex_axes(self, axes, level, limit, tolerance, method, fill_value, copy)
   3734         if columns is not None:
   3735             frame = frame._reindex_columns(columns, method, copy, level,
-> 3736                                            fill_value, limit, tolerance)
   3737 
   3738         index = axes['index']

c:\python36\lib\site-packages\pandas\core\frame.py in _reindex_columns(self, new_columns, method, copy, level, fill_value, limit, tolerance)
   3756         new_columns, indexer = self.columns.reindex(new_columns, method=method,
   3757                                                     level=level, limit=limit,
-> 3758                                                     tolerance=tolerance)
   3759         return self._reindex_with_indexers({1: [new_columns, indexer]},
   3760                                            copy=copy, fill_value=fill_value,

c:\python36\lib\site-packages\pandas\core\indexes\category.py in reindex(self, target, method, level, limit, tolerance)
    521             else:
    522 
--> 523                 codes = new_target.codes.copy()
    524                 codes[indexer == -1] = cats[missing]
    525                 new_target = self._create_from_codes(codes)

Problem description

This should not fail, but rather return an 0x1 DataFrame with the new columns. Other, closely-related, expressions do work correctly in this scenario e.g.:

pd.DataFrame(columns=pd.CategoricalIndex([], categories=['A'])).reindex(columns=pd.Index(['B']))
pd.DataFrame(columns=pd.CategoricalIndex([], categories=['A'])).reindex(columns=pd.CategoricalIndex(['A']))
pd.DataFrame(columns=pd.CategoricalIndex([], categories=['A'])).reindex(columns=pd.CategoricalIndex(['B']))

Expected Output

pd.DataFrame(columns=pd.Index(['A']))

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.1.final.0 python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 63 Stepping 2, GenuineIntel byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.None

pandas: 0.24.1
pytest: 3.1.2
pip: 19.2.1
setuptools: 39.0.1
Cython: 0.29.6
numpy: 1.17.0
scipy: 1.2.1
pyarrow: 0.13.0
xarray: 0.12.1
IPython: 6.1.0
sphinx: 2.1.1
patsy: 0.4.1
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: 1.3.0.dev0
tables: None
numexpr: 2.6.9
feather: None
matplotlib: 2.2.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: 4.3.4
bs4: 4.7.1
html5lib: 0.9999999
sqlalchemy: 1.1.11
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: 0.3.1
pandas_gbq: None
pandas_datareader: None
gcsfs: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugCategoricalCategorical Data TypeIndexRelated to the Index class or subclassesIndexingRelated to indexing on series/frames, not to indexes themselves

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions