Skip to content

Msgpack serialization fails for DatetimeIndex with only two rows #5947

Closed
@acowlikeobject

Description

@acowlikeobject

A rare/corner case, but thought I'd document it.

pandas.read_msgpack() throws a ValueError: Dates do not conform to passed frequency if the index is a DatetimeIndex and there are only two rows of data.

In [66]: df = pd.DataFrame([1, 2], index=pd.date_range('1/1/2013', '1/2/2013'))

In [67]: df
Out[67]: 
            0
2013-01-01  1
2013-01-02  2

[2 rows x 1 columns]

In [68]: pd.read_msgpack(df.to_msgpack())
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-68-fae93928effe> in <module>()
----> 1 pd.read_msgpack(df.to_msgpack())

/home/user/environments/python3/src/pandas/pandas/io/packers.py in read_msgpack(path_or_buf, iterator, **kwargs)
    158         try:
    159             fh = compat.BytesIO(path_or_buf)
--> 160             return read(fh)
    161         finally:
    162             fh.close()

/home/user/environments/python3/src/pandas/pandas/io/packers.py in read(fh)
    136 
    137     def read(fh):
--> 138         l = list(unpack(fh))
    139         if len(l) == 1:
    140             return l[0]

/home/user/environments/python3/src/pandas/pandas/msgpack.cpython-33m.so in pandas.msgpack.Unpacker.__next__ (pandas/msgpack.cpp:7846)()

/home/user/environments/python3/src/pandas/pandas/msgpack.cpython-33m.so in pandas.msgpack.Unpacker._unpack (pandas/msgpack.cpp:6981)()

/home/user/environments/python3/src/pandas/pandas/io/packers.py in decode(obj)
    448     elif typ == 'datetime_index':
    449         data = unconvert(obj['data'], np.int64, obj.get('compress'))
--> 450         result = globals()[obj['klass']](data, freq=obj['freq'], name=obj['name'])
    451         tz = obj['tz']
    452 

/home/user/environments/python3/src/pandas/pandas/tseries/index.py in __new__(cls, data, freq, start, end, periods, copy, name, tz, verify_integrity, normalize, closed, **kwds)
    283                 inferred = subarr.inferred_freq
    284                 if inferred != offset.freqstr:
--> 285                     raise ValueError('Dates do not conform to passed '
    286                                      'frequency')
    287 

ValueError: Dates do not conform to passed frequency

Adding a third row makes the problem go away:

In [13]: df = pd.DataFrame([1, 2, 3], index=pd.date_range('1/1/2013', '1/3/2013'))

In [14]: df
Out[14]: 
            0
2013-01-01  1
2013-01-02  2
2013-01-03  3

[3 rows x 1 columns]

In [15]: pd.read_msgpack(df.to_msgpack())
Out[15]: 
            0
2013-01-01  1
2013-01-02  2
2013-01-03  3

[3 rows x 1 columns]

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions