Skip to content

PERF: .clip with datetime64[ns, UTC] Series regressed #47955

Closed
@jreback

Description

@jreback

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this issue exists on the latest version of pandas.

  • I have confirmed this issue exists on the main branch of pandas.

Reproducible Example

1.2.5

i = pd.date_range('20220101', periods=100000, freq='s', tz='UTC')
clipper = i[0:1000].repeat(100)
%timeit pd.Series(i).clip(upper=clipper)
3.47 ms ± 53 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

1.3.5

i = pd.date_range('20220101', periods=100000, freq='s', tz='UTC')
clipper = i[0:1000].repeat(100)
%timeit pd.Series(i).clip(upper=clipper)
452 ms ± 3.35 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

1.4.3

i = pd.date_range('20220101', periods=100000, freq='s', tz='UTC')
clipper = i[0:1000].repeat(100)
%timeit pd.Series(i).clip(upper=clipper)
132 ms ± 794 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

This work-around gets close to the original (you can do better by using np.where as we do in the impl of clip, but nbd).

i = pd.date_range('20220101', periods=100000, freq='s', tz='UTC')
clipper = i[0:1000].repeat(100)
%timeit pd.Series(pd.to_datetime(pd.Series(i.view('i8')).clip(upper=clipper.view('i8')), utc=True))
6.08 ms ± 62.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Installed Versions

Replace this line with the output of pd.show_versions()

Prior Performance

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    DatetimeDatetime data dtypePerformanceMemory or execution speed performanceRegressionFunctionality that used to work in a prior pandas versionReshapingConcat, Merge/Join, Stack/Unstack, Explode

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions