Skip to content

Empty dataframe with groupby when nan column #16217

Closed
@atbd

Description

@atbd

Hi,
I encountered a 'problem' when my program tried to groupby a dataframe with an empty (ie full of nan) column. Here a code sample with pandas 0.19.2 :

In [4]: a = pd.DataFrame({'a': [2,2,3,4,5], 
                          'b': [5,5,6,6,7], 
                          'c': [np.nan]*5, 
                          'sum': [1]*5})

In [5]: a
Out[5]: 
   a  b   c  sum
0  2  5 NaN    1
1  2  5 NaN    1
2  3  6 NaN    1
3  4  6 NaN    1
4  5  7 NaN    1

In [6]: a.groupby(['a','b','c']).agg({'sum':'sum'})
Out[6]: 
Empty DataFrame
Columns: [sum]
Index: []

In [7]: a.groupby(['a','b']).agg({'sum':'sum'})
Out[7]: 
     sum
a b     
2 5    2
3 6    1
4 6    1
5 7    1

I think we could throw a warning (or maybe an error) when this case is encountered in addition to return an empty dataframe.

INSTALLED VERSIONS ------------------ commit: None python: 3.6.1.final.0 python-bits: 64 OS: Linux OS-release: 4.4.0-75-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.11.3
scipy: 0.18.1
statsmodels: 0.6.1
xarray: None
IPython: 5.1.0
sphinx: 1.5.1
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: 1.2.0
tables: 3.3.0
numexpr: 2.6.1
matplotlib: 2.0.0
openpyxl: 2.4.1
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.2
bs4: 4.5.3
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.1.5
pymysql: 0.7.9.None
psycopg2: None
jinja2: 2.9.4
boto: 2.45.0
pandas_datareader: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Duplicate ReportDuplicate issue or pull requestError ReportingIncorrect or improved errors from pandasGroupbyMissing-datanp.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions