Interpolate fails when passed limit argument.

[ianp]

When I try to interpolate time series data it fails with an index error: arrays used as indices must be of integer (or boolean) type.

from pandas import read_csv
from StringIO import StringIO

data = '''2013-09-28,4018.02
2013-10-05,5806.78
2013-10-26,6334.4
2013-11-02,6199.6
2013-11-09,6213.07
2013-11-16,7099.39
2013-11-23,7332.98
2013-11-30,6157.37
2013-12-07,7427.65
2014-01-04,5856.38
2014-01-11,6208.99
2014-01-18,7440.29
2014-01-25,7295.97
2014-02-08,7654.42
'''

df = read_csv(StringIO(data))
df.interpolate(limit=1)

Removing the limit=1 argument works as expected.

Python 2.7.6, pandas 0.14.0, numpy 1.8.1, on OS X.

I'll try it with a Python 3 environment when I get chance later this week.

[jreback]

can you update with the expected output as well; thanks

[ianp]

With limit=1 I'd expect it to fill in the data for 2014-02-01 as 7475.195. The gaps in that data are

• 2013-10-12,2013-10-19 (left blank because of limit)
• 2013-12-14,2013-12-21,2013-12-28 (left blank because of limit)
• 2014-01-25 (should be filled in via linear interpolation.

Unless I'm misunderstanding how the interpolate method works (fairly new to Pandas here!)

[jreback]

what I mean is construct a frame that is the result (just like you did the input frame). Then the tests for this are straightforward.

[jreback]

and your example w/o the interpolate doesn't do anything in any event (as the input is not read correctly). In order to fix a bug, we need a copy-pastable example, then produces incorrect output or exception. Then the expected output for comparison. thanks

[ianp]

Is this better:

from StringIO import StringIO
from pandas import read_csv
from pandas.util.testing import assert_frame_equal

frame_in = read_csv(StringIO('''Date,Sales
2013-09-28,4018.02
2013-10-05,5806.78
2013-10-26,6334.4
2013-11-02,6199.6
2013-11-09,6213.07
2013-11-16,7099.39
2013-11-23,7332.98
2013-11-30,6157.37
2013-12-07,7427.65
2014-01-04,5856.38
2014-01-11,6208.99
2014-01-18,7440.29
2014-01-25,7295.97
2014-02-08,7654.42
'''))

frame_expected = read_csv(StringIO('''Date,Sales
2013-09-28,4018.02
2013-10-05,5806.78
2013-10-26,6334.4
2013-11-02,6199.6
2013-11-09,6213.07
2013-11-16,7099.39
2013-11-23,7332.98
2013-11-30,6157.37
2013-12-07,7427.65
2014-01-04,5856.38
2014-01-11,6208.99
2014-01-18,7440.29
2014-01-25,7295.97
2014-02-01,7475.195
2014-02-08,7654.42
'''))

assert_frame_equal(frame_in.interpolate(method='linear', limit=1), frame_expected)

I'd expect no output (i.e. the test would pass).

Am I doing something wrong when calling read_csv? You say that the input isn't read correctly but it seems OK to me (well, the original version I posted read the first line as a header by mistake).

[jreback]

well you need read_csv(StringIO(....),parse_dates=True,index_col=0). Then you would need to reindex the dates or something. Its not clear what you are trying to interpolate (e.g. I think you want daily interpolation). http://pandas.pydata.org/pandas-docs/stable/missing_data.html#interpolation.

after you set the date and index properly. You may find

frame_in.asfreq('D').interpolate() works for you (and actually limit works). Pls reopen if you are having trouble.

[jreback]

``frame_in = read_csv(StringIO(...),parse_dates=True,index_col=0)

In [14]: pd.set_option('max_rows',10)

In [15]: frame_in
Out[15]: 
              Sales
Date               
2013-09-28  4018.02
2013-10-05  5806.78
2013-10-26  6334.40
2013-11-02  6199.60
2013-11-09  6213.07
...             ...
2014-01-04  5856.38
2014-01-11  6208.99
2014-01-18  7440.29
2014-01-25  7295.97
2014-02-08  7654.42

[14 rows x 1 columns]

In [16]: frame_in.asfreq('D')
Out[16]: 
              Sales
2013-09-28  4018.02
2013-09-29      NaN
2013-09-30      NaN
2013-10-01      NaN
2013-10-02      NaN
...             ...
2014-02-04      NaN
2014-02-05      NaN
2014-02-06      NaN
2014-02-07      NaN
2014-02-08  7654.42

[134 rows x 1 columns]

In [17]: frame_in.asfreq('D').interpolate()
Out[17]: 
                  Sales
2013-09-28  4018.020000
2013-09-29  4273.557143
2013-09-30  4529.094286
2013-10-01  4784.631429
2013-10-02  5040.168571
...                 ...
2014-02-04  7552.005714
2014-02-05  7577.609286
2014-02-06  7603.212857
2014-02-07  7628.816429
2014-02-08  7654.420000

[134 rows x 1 columns]

In [18]: frame_in.asfreq('D').interpolate(limit=1)
Out[18]: 
                  Sales
2013-09-28  4018.020000
2013-09-29  4273.557143
2013-09-30          NaN
2013-10-01          NaN
2013-10-02          NaN
...                 ...
2014-02-04          NaN
2014-02-05          NaN
2014-02-06          NaN
2014-02-07          NaN
2014-02-08  7654.420000

[134 rows x 1 columns]

[TomAugspurger]

This actually is a dupe of #7173, which will be fixed by #8235