BUG: unary operators on IntegerArray return shared mask creating inconsistencies when assigning null and non-null values

[simonjayhawkins]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

 
>>> s = pd.Series([1, 2, 3], dtype="Int64")
>>> s
0    1
1    2
2    3
dtype: Int64
>>> 
>>> s2 = -s
>>> 
>>> s2
0    -1
1    -2
2    -3
dtype: Int64
>>> 
>>> s[0] = None # assigning None
>>> 
>>> s
0    <NA>
1       2
2       3
dtype: Int64
>>> 
>>> s2
0    <NA>  # <--- expected -1
1      -2
2      -3
dtype: Int64
>>> 
>>> s[1] = 10  # assigning  a value is ok
>>> s
0    <NA>
1      10
2       3
dtype: Int64
>>> 
>>> s2
0    <NA>
1      -2
2      -3
dtype: Int64
>>> 
>>> s2[2] = None
>>> s2
0    <NA>
1      -2
2    <NA>
dtype: Int64
>>> 
>>> s
0    <NA>
1      10
2    <NA>  # expected 3
dtype: Int64
>>> 
>>> s.values._data is s2.values._data
False
>>> 
>>> s.values._mask is s2.values._mask
True

Problem description

shared mask leads to an inconsistency when assigning a non-null value compared to assigning a null value

also for __invert__ in BaseMaskedArray so will also affect other nullable arrays

Expected Output

Output of pd.show_versions()

[paste the output of pd.show_versions() here leaving a blank line after the details tag]

[simonjayhawkins]

this was working in 1.0.5 so will label as regression

first bad commit: [67aae80] CLN: avoid values_from_object in NDFrame (#32422)

this caused a regression, #36063 and the fix, #36303 introduced this bug. cc @dsaxton

[dsaxton]

Thanks @simonjayhawkins, looks like we should be copying the mask as well. I'll try to put up a PR to fix today.

[simonjayhawkins]

closed in #39971