Skip to content

Remove internal classes from MultiIndex and Index pickles. #5076

Closed
@jtratner

Description

@jtratner

I added a FrozenNDArray and FrozenList class for MultiIndex internals for 0.13. They shouldn't end up in pickles, so we don't need to support them. Trivial fix but needs to happen before 0.13 is released.

Current less than optimal behavior:

In [2]: mi = MultiIndex.from_tuples(zip(range(10), range(10)))

In [3]: mi
Out[3]:
MultiIndex(levels=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]
           labels=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])

In [4]: mi.__reduce__()
Out[4]:
(<function numpy.core.multiarray._reconstruct>,
 (pandas.core.index.MultiIndex, (0,), 'b'),
 ((1, (0,), dtype('O'), False, []),
  ([Int64Index([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64),
    Int64Index([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64)],
   [FrozenNDArray([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64),
    FrozenNDArray([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int64)],
   None,
   [None, None])))

Correct output (or could just be ndarrays too):

In [2]: mi = MultiIndex.from_tuples(zip(range(10), range(10)))

In [3]: mi
Out[3]:
MultiIndex(levels=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]
           labels=[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])

In [4]: mi.__reduce__()
Out[4]:
(<function numpy.core.multiarray._reconstruct>,
 (pandas.core.index.MultiIndex, (0,), 'b'),
 ((1, (0,), dtype('O'), False, []),
  ([array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),
    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])],
   [array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),
    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])],
   None,
   [None, None])))

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions