Skip to content

BUG: Casting string series to category raises TypeError in nightlies #40351

Closed
@siboehm

Description

@siboehm
  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd
import pandas.testing as pdt

y = pd.Series(["A"], dtype="object").astype("category")
x = pd.Series(["A"], dtype="string").astype("category")

pdt.assert_series_equal(x,y)

pandas==1.2.3: runs through, doesn't raise

pandas==1.3.0.dev0+1005.g93c52e4e1d (from the nightly wheels):

Traceback (most recent call last):
  File "/home/simon/Documents/QuantCo/quantcore.thek/test.py", line 5, in <module>
    x = pd.Series(["A"], dtype="string").astype("category")
  File "/home/simon/miniconda3/envs/quantcore.thek/lib/python3.9/site-packages/pandas/core/generic.py", line 5944, in astype
    new_data = self._mgr.astype(dtype=dtype, copy=copy, errors=errors)
  File "/home/simon/miniconda3/envs/quantcore.thek/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 659, in astype
    return self.apply("astype", dtype=dtype, copy=copy, errors=errors)
  File "/home/simon/miniconda3/envs/quantcore.thek/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 524, in apply
    applied = getattr(b, f)(**kwargs)
  File "/home/simon/miniconda3/envs/quantcore.thek/lib/python3.9/site-packages/pandas/core/internals/blocks.py", line 658, in astype
    new_values = astype_array_safe(values, dtype, copy=copy, errors=errors)
  File "/home/simon/miniconda3/envs/quantcore.thek/lib/python3.9/site-packages/pandas/core/dtypes/cast.py", line 1352, in astype_array_safe
    new_values = astype_array(values, dtype, copy=copy)
  File "/home/simon/miniconda3/envs/quantcore.thek/lib/python3.9/site-packages/pandas/core/dtypes/cast.py", line 1297, in astype_array
    values = values.astype(dtype, copy=copy)
  File "/home/simon/miniconda3/envs/quantcore.thek/lib/python3.9/site-packages/pandas/core/arrays/string_.py", line 328, in astype
    elif np.issubdtype(dtype, np.floating):
  File "/home/simon/miniconda3/envs/quantcore.thek/lib/python3.9/site-packages/numpy/core/numerictypes.py", line 419, in issubdtype
    arg1 = dtype(arg1).type
TypeError: Cannot interpret 'CategoricalDtype(categories=None, ordered=None)' as a data type

Problem description

Seems like a regression. Converting the string series to object first works without raising:

x = pd.Series(["A"], dtype="string").astype("object").astype("category")

Expected Output

nightly should behave like pandas=1.2.3, running through and producing two equal series.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 93c52e4
python : 3.9.2.final.0
python-bits : 64
OS : Linux
OS-release : 5.8.0-44-generic
Version : #50-Ubuntu SMP Tue Feb 9 06:29:41 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.3.0.dev0+1005.g93c52e4e1d
numpy : 1.21.0.dev0+934.g6222e283f
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 49.6.0.post20210108
Cython : None
pytest : 6.2.2
hypothesis : None
sphinx : 3.5.2
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.21.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : 0.8.7
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 3.1.0.dev317
pyxlsb : None
s3fs : None
scipy : 1.6.0
sqlalchemy : None
tables : None
tabulate : 0.8.9
xarray : None
xlrd : None
xlwt : None
numba : None

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugCategoricalCategorical Data TypeRegressionFunctionality that used to work in a prior pandas version

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions