BUG: Inconsistent behavior of `.replace()` in Int64 series with `<NA>`.

[tongr]

Code Sample

This code does work as expected (0 value is replaced by <NA> in an Int64 series).

>>> import pandas as pd
>>> s = pd.Series([0, 1]).astype("Int64")
>>> s.replace(0, pd.NA)
0    <NA>
1       1
dtype: Int64

In contrast, when having <NA> values in the series .replace() does not work as expected (0 is not replaced by <NA>).

>>> import pandas as pd
>>> s = pd.Series([0, None]).astype("Int64")
>>> s.replace(0, pd.NA)
0       0
1    <NA>
dtype: object

However, the code does work if we add .fillna() before replacing:

>>> import pandas as pd
>>> s = pd.Series([0, None]).astype("Int64")
>>> s.fillna(0).replace(0, pd.NA)
0    <NA>
1    <NA>
dtype: Int64

Problem description

When applying the replace on a Int64 series with or without <NA> I expect consistent behavior.
Also the result type of .replace() seems to be inconsistent based on the usage of .fillna() on the series (without fillna() the type changes to object).

Expected behavior

I would expect that s.replace(0, pd.NA) and s.fillna(0).replace(0, pd.NA) on an Int64 series will always lead to the same result.

Output of pd.show_versions()

Details

INSTALLED VERSIONS

commit : 67a3d42
python : 3.8.0.final.0
python-bits : 64
OS : Linux
OS-release : 5.4.0-53-generic
Version : #59~18.04.1-Ubuntu SMP Wed Oct 21 12:14:56 UTC 2020
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.1.4
numpy : 1.19.2
pytz : 2020.4
dateutil : 2.8.1
pip : 20.3
setuptools : 50.3.2.post20201201
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[jorisvandenbossche]

@tongr thanks for the report!

This seems to be working correctly on master (there have been some replace related fixes):

In [2]: >>> import pandas as pd
   ...: >>> s = pd.Series([0, None]).astype("Int64")
   ...: >>> s.replace(0, pd.NA)
Out[2]: 
0    <NA>
1    <NA>
dtype: Int64

In [3]: pd.__version__
Out[3]: '1.2.0.dev0+1441.g65f0463d36'

So this will be fixed in the upcoming 1.2 release

[simonjayhawkins]

This seems to be working correctly on master (there have been some replace related fixes):

This was originally working in 1.0.5

fixed in [3967131] BUG: df.replace with numeric values and str to_replace (#36093)

[rkc007]

take

[rkc007]

@simonjayhawkins I would love to work on this issue. Could you please give a brief description that where I should begin with?

[simonjayhawkins]

This issue just needs a test added to the test suite before closing. pandas\tests\series\methods\test_replace.py could be a good place to start. other tests (in that module) will give an idea of the style.