BUG: Unable to parse monthly-string in datetime.pxd

[sinhrks]

Travis build looks fail with following error.

  File "/home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages/pandas/tseries/tests/test_period.py", line 1976, in test_range_slice_outofbounds
...
 File "datetime.pxd", line 137, in datetime._string_to_dts (pandas/tslib.c:52202)
ValueError: Unable to parse 2013/06

The failure is caused by test in DatetimeIndex, strangely PeriodIndex has no problem.

I don't know where datetime.pxd is derived from, but one assumption is the _string_to_dts uses todays date (31) in parsing process.

import pandas as pd

didx = pd.DatetimeIndex(start='2013/10/01', freq='D', periods=10)
pidx = pd.PeriodIndex(start='2013/10/01', freq='D', periods=10)

for idx in [didx, pidx]:
    df = pd.DataFrame(dict(units=[100 + i for i in range(10)]), index=idx)
    for key in ['2013/{0}'.format(i) for i in range(1, 13)]:
        try:
            df[key:'2013/03']
        except ValueError, e:
            print(type(idx), key, e)

# will prints
(<class 'pandas.tseries.index.DatetimeIndex'>, '2013/2', ValueError('Unable to parse 2013/2',))
(<class 'pandas.tseries.index.DatetimeIndex'>, '2013/4', ValueError('Unable to parse 2013/4',))
(<class 'pandas.tseries.index.DatetimeIndex'>, '2013/6', ValueError('Unable to parse 2013/6',))
(<class 'pandas.tseries.index.DatetimeIndex'>, '2013/9', ValueError('Unable to parse 2013/9',))
(<class 'pandas.tseries.index.DatetimeIndex'>, '2013/11', ValueError('Unable to parse 2013/11',))

[TomAugspurger]

I think you're right about that. Could you try parsing Timestamp('2013/06'). Since June has 30 days that will presumably fail too (It's still the 30th here so I can't test).

@jreback should Timestamp('2013-02') be equivalent to Timestamp('2013/02')?

Looks like dateutil uses the current day for both:

In [65]: from dateutil import parser

In [66]: parser.parse('2013-01')
Out[66]: datetime.datetime(2013, 1, 30, 0, 0)

In [67]: parser.parse('2013/01')
Out[67]: datetime.datetime(2013, 1, 30, 0, 0)

In [68]: parser.parse('2013-02')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-68-53cfc7a2bd41> in <module>()
----> 1 parser.parse('2013-02')

/Users/admin/Envs/pandas-dev/lib/python2.7/site-packages/dateutil/parser.pyc in parse(timestr, parserinfo, **kwargs)
    695         return parser(parserinfo).parse(timestr, **kwargs)
    696     else:
--> 697         return DEFAULTPARSER.parse(timestr, **kwargs)
    698 
    699 

/Users/admin/Envs/pandas-dev/lib/python2.7/site-packages/dateutil/parser.pyc in parse(self, timestr, default, ignoretz, tzinfos, **kwargs)
    308             if value is not None:
    309                 repl[attr] = value
--> 310         ret = default.replace(**repl)
    311         if res.weekday is not None and not res.day:
    312             ret = ret+relativedelta.relativedelta(weekday=res.weekday)

ValueError: day is out of range for month

In [69]: parser.parse('2013/02')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-69-edf06e41be1f> in <module>()
----> 1 parser.parse('2013/02')

/Users/admin/Envs/pandas-dev/lib/python2.7/site-packages/dateutil/parser.pyc in parse(timestr, parserinfo, **kwargs)
    695         return parser(parserinfo).parse(timestr, **kwargs)
    696     else:
--> 697         return DEFAULTPARSER.parse(timestr, **kwargs)
    698 
    699 

/Users/admin/Envs/pandas-dev/lib/python2.7/site-packages/dateutil/parser.pyc in parse(self, timestr, default, ignoretz, tzinfos, **kwargs)
    308             if value is not None:
    309                 repl[attr] = value
--> 310         ret = default.replace(**repl)
    311         if res.weekday is not None and not res.day:
    312             ret = ret+relativedelta.relativedelta(weekday=res.weekday)

ValueError: day is out of range for month

But we do

In [70]: Timestamp('2013-02')
Out[70]: Timestamp('2013-02-01 00:00:00')

In [71]: Timestamp('2013/02')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-71-4dafd0282f13> in <module>()
----> 1 Timestamp('2013/02')

/Users/admin/Envs/pandas-dev/lib/python2.7/site-packages/pandas/pandas/tslib.so in pandas.tslib.Timestamp.__new__ (pandas/tslib.c:5958)()

/Users/admin/Envs/pandas-dev/lib/python2.7/site-packages/pandas/pandas/tslib.so in pandas.tslib.convert_to_tsobject (pandas/tslib.c:15680)()

/Users/admin/Envs/pandas-dev/lib/python2.7/site-packages/pandas/pandas/tslib.so in datetime._string_to_dts (pandas/tslib.c:52040)()

ValueError: Unable to parse 2013/02

[TomAugspurger]

@sinhrks let's see what @jreback has to say, but I think we patch this by changing the /s in test_range_slice_outofbounds to -s to follow the iso 8601 format. I am curious what the underlying issue is though. Want to do a PR?

[jreback]

'2013/02' looks like a date but is not ios compat so it doesn't parse and hence is passed to dateutil for parsing (which as one of its bugs uses the current day since no day is provided)

so easy enough to fix the test as you say tom

it might be reasonable to see if u can allow / in addition to '-' for a sep in the c code for the datetime parsing - not sure what if anything this would break

[sinhrks]

Thanks. I'll Fix the test.

[yarikoptic]

FWIW -- hit this one on
https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=mipsel&ver=0.14.0-1&stamp=1401515396
glad that it was addressed -- will cherry pick that commit in Debian package