Skip to content

fillna is broken for Datetime columns [regression] #27419

Closed
@vnlitvinov

Description

@vnlitvinov

In 823af76 a call to block._try_coerce_args() was incorrectly replaced with call to block._can_hold_element() (I think under assumption that all block types have identical implementations of _try_coerce_args which is not true).

For Datetime blocks this leads to blowing up with RecursionError: maximum recursion depth exceeded in comparison.

Prior to the commit I indicated it worked fine, so I consider this a regression.

Small reproducer

import pandas as pd
import numpy as np
from io import StringIO as sio

d1 = pd.read_csv(sio('2010,\n,\n2000\n'),header=None,parse_dates=[0])
f1 = np.dtype('datetime64[ms]').type('1970-01-01').astype('datetime64[ms]')
d1[0].fillna(f1)

Problem description

When the reproducer is launched, it's output is a long stacktrace caused by RecursionError.

Expected Output

The sample should not raise.

Output of pd.show_versions()

commit : None
python : 3.6.8.final.0
python-bits : 64
OS : Linux
OS-release : 4.15.0-47-generic
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 0.25.0rc0+67.g2ed1f28cf
numpy : 1.16.2
pytz : 2018.4
dateutil : 2.6.0
pip : 10.0.1
setuptools : 39.0.1.post20180504
Cython : 0.29.6
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.1
IPython : None
pandas_datareader: None
bs4 : 4.7.1
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 0.9.0
pytables : None
s3fs : None
scipy : 1.2.1
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None

Proposed solution

Revert the changes to fillna() function here: 823af76?diff=unified#diff-8f51ce8697bdff680b3f71b196c582eeL388

Metadata

Metadata

Assignees

No one assigned

    Labels

    BlockerBlocking issue or pull request for an upcoming releaseDatetimeDatetime data dtypeMissing-datanp.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions