Wrong indexing with time zone support

[pkaczynski]

There is something really wrong with ix method while using time support. Simple example:

def test_tz_error(self):
        import pandas, pytz
        from datetime import datetime
        from datetime import timedelta as td
        tz = pytz.timezone('Europe/Warsaw')
        ts_from = tz.localize(datetime(2012, 1, 1, 0, 0))
        ts_to = tz.localize(datetime(2012, 1, 1, 23, 0))
        ttt = pandas.DataFrame(index=pandas.date_range(ts_from, ts_to, freq='H'), 
                               columns=['values'])
        ttt['value'] = range(1,25)
        ts = ts_from
        while ts <= ts_to:
            print ttt.ix[ts]['value']
            ts += td(hours=1)

Should print out number 1,2,3,...,24. However i get:
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 24

Why is it from 2, not 1? And why the last value is repeated twice with value 24??

I am using latest release 0.10.1

[Moisan]

Take a look at this : https://gist.github.com/Moisan/4987028

The problem seems to be coming from the timezone information. It seems that the ix method is doing something wrong since print ttt[ttt.index == ts]['value'][0] will output the values correctly with the timezones.

[stephenwlin]

i believe this is the same issue as #2788 (or #2789, which is related) and fixed in #2785: i tested it using my local copy your example prints 1-24. (I'm slightly confused because I think your case is one that should have already been working before this fix, since you're indexing with tz-aware dates on a tz-aware index, but in any case it seems to be working now...)

[stephenwlin]

Actually, I got it now. The issue is part of #2789: internally the timezone offset was being double counted, so everything was off by one in your case. (The second 24 is a weird corner case thing)

[pkaczynski]

Thanks for the answers. Is this going to be included in next release?

[stephenwlin]

It's already been merged, so yes, barring anything catastrophic

[wesm]

Can this be closed? i.e. has been sufficiently unit tested

[stephenwlin]

i think so