Skip to content

BUG: Series construction from MaskedArray fails for non-floating, non-object types #2880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 23, 2013

Conversation

stephenwlin
Copy link
Contributor

Currently Series constructor is throwing exceptions or ignoring the mask for non-floating, non-object MaskedArray input:

In [1]: from numpy.ma import MaskedArray as ma
In [2]: from pandas import Series, DataFrame
In [3]: from datetime import datetime as dt

In [4]: mask = [True, False]

In [5]: Series(ma([2, 0], mask=mask, dtype=int))
(exception)

In [6]: Series(ma([True, False], mask=mask, dtype=bool))
Out[6]: 
0     True
1    False

In [7]: Series(ma([dt(2006, 1, 1), dt(2007, 1, 1)],
   ...:        mask=mask, dtype='M8[ns]'))
(exception)

After fix:

In [1]: from numpy.ma import MaskedArray as ma
In [2]: from pandas import Series, DataFrame
In [3]: from datetime import datetime as dt

In [4]: mask = [True, False]

In [5]: Series(ma([2, 0], mask=mask, dtype=int))
Out[5]: 
0   NaN
1     0
Dtype: float64

In [6]: Series(ma([True, False], mask=mask, dtype=bool))
Out[6]: 
0      NaN
1    False
Dtype: object

In [7]: Series(ma([dt(2006, 1, 1), dt(2007, 1, 1)],
   ...:        mask=mask, dtype='M8[ns]'))
Out[7]: 
0                   NaT
1   2007-01-01 00:00:00
Dtype: datetime64[ns]

The upcasting will only be done if necessary (i.e. if the mask has any True elements):

In [8]: Series(ma([2, 3], mask=[False, False], dtype=int))
Out[8]: 
0    2
1    3
Dtype: int32

In [9]: DataFrame(ma([2, 3], mask=[False, False], dtype=int))
Out[9]: 
   0
0  2
1  3

This latter is a minor change to the previous behavior of the DataFrame constructor, which was upcasting correctly but was doing it even if the mask was all False: I've changed it to only upcast when necessary because this is more consistent with how masks are treated elsewhere.

Also made some PEP8 line length fixes

@jreback
Copy link
Contributor

jreback commented Feb 23, 2013

can u rebase to master and see if this still merges cleanly?
I had merged a tiny change that might conflict
otherwise ok to merge?

@stephenwlin
Copy link
Contributor Author

everything looks ok to me

jreback added a commit that referenced this pull request Feb 23, 2013
BUG: Series construction from MaskedArray fails for non-floating, non-object types

thanks!
@jreback jreback merged commit 5d8eb29 into pandas-dev:master Feb 23, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants