Skip to content

BUG: DataFrame.resample raised AmbiguousTimeError at a midnight DST transition #33137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 3, 2020
Merged

BUG: DataFrame.resample raised AmbiguousTimeError at a midnight DST transition #33137

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
72b2baf
BUG: DataFrame.resample raised AmbiguousTimeError at a midnight DST t…
Mar 30, 2020
861fdc0
whatsnew grammar
Mar 30, 2020
62f4fd5
Merge remote-tracking branch 'upstream/master' into resample_dst_midn…
Mar 31, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v1.1.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,7 @@ Groupby/resample/rolling
- Bug in :meth:`GroupBy.apply` raises ``ValueError`` when the ``by`` axis is not sorted and has duplicates and the applied ``func`` does not mutate passed in objects (:issue:`30667`)
- Bug in :meth:`DataFrameGroupby.transform` produces incorrect result with transformation functions (:issue:`30918`)
- Bug in :meth:`DataFrame.groupby` and :meth:`Series.groupby` produces inconsistent type when aggregating Boolean series (:issue:`32894`)

- Bug in :meth:`DataFrame.resample` where an ``AmbiguousTimeError`` would be raised when the resulting timezone aware :class:`DatetimeIndex` had a DST transition at midnight (:issue:`25758`)

Reshaping
^^^^^^^^^
Expand Down
4 changes: 3 additions & 1 deletion pandas/core/resample.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1422,13 +1422,15 @@ def _get_time_bins(self, ax):
# because replace() will swallow the nanosecond part
# thus last bin maybe slightly before the end if the end contains
# nanosecond part and lead to `Values falls after last bin` error
# GH 25758: If DST lands at midnight (e.g. 'America/Havana'), user feedback
# has noted that ambiguous=True provides the most sensible result
binner = labels = date_range(
freq=self.freq,
start=first,
end=last,
tz=ax.tz,
name=ax.name,
ambiguous="infer",
ambiguous=True,
nonexistent="shift_forward",
)

Expand Down
18 changes: 18 additions & 0 deletions pandas/tests/resample/test_datetime_index.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1440,6 +1440,24 @@ def test_downsample_across_dst_weekly():
tm.assert_series_equal(result, expected)


def test_downsample_dst_at_midnight():
# GH 25758
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the DST transition at midnight on the 4th?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct https://www.currenttimeonline.com/dst/dst.do?tz=America/Havana

start = datetime(2018, 11, 3, 12)
end = datetime(2018, 11, 5, 12)
index = pd.date_range(start, end, freq="1H")
index = index.tz_localize("UTC").tz_convert("America/Havana")
data = list(range(len(index)))
dataframe = pd.DataFrame(data, index=index)
result = dataframe.groupby(pd.Grouper(freq="1D")).mean()
expected = DataFrame(
[7.5, 28.0, 44.5],
index=date_range("2018-11-03", periods=3).tz_localize(
"America/Havana", ambiguous=True
),
)
tm.assert_frame_equal(result, expected)


def test_resample_with_nat():
# GH 13020
index = DatetimeIndex(
Expand Down