Skip to content

BUG: groupby.rolling.corr incorrect/fails when grouping columns contain tuples #44078

Closed
@rhshadrach

Description

@rhshadrach

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

df = pd.DataFrame({'a': [(1,), (1,), (1,)], 'b': [4, 5, 6]})
gb = df.groupby(['a'])
print(gb.rolling(2).corr(other=df))

df = pd.DataFrame({'a': [(1,2,), (1,2,), (1,2,)], 'b': [4, 5, 6]})
gb = df.groupby(['a'])
print(gb.rolling(2).corr(other=df))

Issue Description

The first block produces

      a    b
a           
1 0 NaN  NaN
  1 NaN  1.0
  2 NaN  1.0

where the first index level should be (1, ) instead. The second block raises a ValueError. It is due to usage of maybe_make_list in _apply_pairwise (related: #44056). Instead of using this function, we can pack the index keys in lists when self._grouper.nkeys is 1 and otherwise leave them as-is. I.e.

if self._grouper.nkeys == 1:
    gb_pairs = (
        [pair] for pair in self._grouper.indices.keys()
    )
else:
    gb_pairs = (
        pair for pair in self._grouper.indices.keys()
    )

Expected Behavior

          a    b
a           
(1, ) 0 NaN  NaN
      1 NaN  1.0
      2 NaN  1.0

           a    b
a           
(1, 2) 0 NaN  NaN
       1 NaN  1.0
       2 NaN  1.0

Installed Versions

Replace this line with the output of pd.show_versions()

Metadata

Metadata

Assignees

Labels

GroupbyNeeds TestsUnit test(s) needed to prevent regressionsWindowrolling, ewma, expandingcov/corr

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions