PERF: perf regressions vs 0.14.0

[jreback]

We have many + perf fixes in 0.14.1 yeh!

but need to look at these (resamples could be spurious)

dataframe_resample_mean_string               |   4.0483 |   2.4170 |   1.6749 |
dataframe_resample_max_numpy                 |   3.3394 |   1.7230 |   1.9381 |
dataframe_resample_max_string                |   3.3154 |   1.7047 |   1.9448 |
dataframe_resample_min_numpy                 |   3.3020 |   1.6843 |   1.9604 |
dataframe_resample_min_string                |   3.2830 |   1.6587 |   1.9793 |
timeseries_timestamp_downsample_mean         |  11.4803 |   4.0847 |   2.8106 |
timeseries_period_downsample_mean            |  29.7631 |  10.3510 |   2.8754 |
datetimeindex_normalize                      |  77.5570 |   3.0633 |  25.3183 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [272dae5] : BUG: doc example in groupby.rst (GH7559 / GH7628)
Base   [da0f7ae] : RLS: 0.14.0 final

[jreback]

cc @sinhrks

[cpcloud]

There are some pretty hefty hits for DatetimeIndex-ed repring, so much so that I have to KeyboardInterrupt the repring. I'll see if I can't track this down. It looks like there's some unnecessary vectorization of timezone manipulation.

[jreback]

can you post your example?

[cpcloud]

In [1]: period = 1.0 / 2048 * 1e9

In [2]: freq = pd.datetools.Nano(period)

In [3]: df = DataFrame({'a': np.random.randn(6e6)}, index=pd.date_range('now', periods=6e6, freq=freq, tz='EST'))

In [4]: df.a  # takes a very long time

[cpcloud]

hm maybe repr-ing is not the issue repr(df.a) finishes in a timely manner.

[jreback]

what does ipython actually do after executing, maybe it caches it or something?

[jreback]

the answer is in the format, while I ctrl-c....its in trim_zeros (its prob trimming on the entire frame and not on the displayed portion)

[cpcloud]

hm when i control-c it's in a np.vectorize-d function. vectorize is not really that much faster than just map + lambda which i think is the issue

[cpcloud]

okay maybe not ... just cython'd that and it didn't improve the situation

[cpcloud]

that is where the bottleneck is though

[cpcloud]

In [4]: df.a
> /home/phillip/Documents/code/py/pandas/pandas/tseries/frequencies.py(674)__init__()
    673             import ipdb; ipdb.set_trace()
--> 674             self.values = tslib.tz_convert_safe_dst(self.values, 'UTC',
    675                                                     index.tz)

ipdb> n
> /home/phillip/Documents/code/py/pandas/pandas/tseries/frequencies.py(675)__init__()
    674             self.values = tslib.tz_convert_safe_dst(self.values, 'UTC',
--> 675                                                     index.tz)
    676             #self.values = np.vectorize(f)(self.values)

ipdb> n  # bueller ..... bueller ..... anybody

[jreback]

yes...that is the same problem as in the vbench above

[cpcloud]

ok