Skip to content

BUG: incorrect year returned in isocalendar for certain dates #36050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Sep 4, 2020
+21 −2
Merged

BUG: incorrect year returned in isocalendar for certain dates #36050

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d49df7f
add test for bug #36032
asishm Sep 1, 2020
e4c2e11
fix incorrect iso_year for certain dates
asishm Sep 1, 2020
15394c7
pep8
asishm Sep 1, 2020
619affe
pep8 fix
asishm Sep 1, 2020
914baf8
black
asishm Sep 1, 2020
630fc22
added other test cases
asishm Sep 2, 2020
6e54623
add whats new
asishm Sep 2, 2020
c57b9de
Merge branch 'master' into isocalendar
jreback Sep 2, 2020
a2697fa
Merge branch 'master' of github.com:pandas-dev/pandas into isocalendar
asishm Sep 3, 2020
cac5b56
add hypothesis tests for isocalendar
asishm Sep 3, 2020
0d6c974
imports isort
asishm Sep 3, 2020
c18836f
add warn=False in tests
asishm Sep 3, 2020
030bc88
Merge branch 'master' of github.com:pandas-dev/pandas into isocalendar
asishm Sep 3, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.1.2.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Bug fixes
- Bug in :meth:`DataFrame.apply` with ``result_type="reduce"`` returning with incorrect index (:issue:`35683`)
- Bug in :meth:`DateTimeIndex.format` and :meth:`PeriodIndex.format` with ``name=True`` setting the first item to ``"None"`` where it should be ``""`` (:issue:`35712`)
- Bug in :meth:`Float64Index.__contains__` incorrectly raising ``TypeError`` instead of returning ``False`` (:issue:`35788`)
- Bug in :meth:`Series.dt.isocalendar` and :meth:`DatetimeIndex.isocalendar` that returned incorrect year for certain dates (:issue:`36032`)
- Bug in :class:`DataFrame` indexing returning an incorrect :class:`Series` in some cases when the series has been altered and a cache not invalidated (:issue:`33675`)

.. ---------------------------------------------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions pandas/_libs/tslibs/ccalendar.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,10 +201,10 @@ cpdef iso_calendar_t get_iso_calendar(int year, int month, int day) nogil:
iso_week = 1

iso_year = year
if iso_week == 1 and doy > 7:
if iso_week == 1 and month == 12:
iso_year += 1

elif iso_week >= 52 and doy < 7:
elif iso_week >= 52 and month == 1:
iso_year -= 1

return iso_year, iso_week, dow + 1
Expand Down
3 changes: 3 additions & 0 deletions pandas/tests/series/test_datetime_values.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -682,6 +682,9 @@ def test_setitem_with_different_tz(self):
[[pd.NaT], [[np.NaN, np.NaN, np.NaN]]],
[["2019-12-31", "2019-12-29"], [[2020, 1, 2], [2019, 52, 7]]],
[["2010-01-01", pd.NaT], [[2009, 53, 5], [np.NaN, np.NaN, np.NaN]]],
# see GH#36032
[["2016-01-08", "2016-01-04"], [[2016, 1, 5], [2016, 1, 1]]],
[["2016-01-07", "2016-01-01"], [[2016, 1, 4], [2015, 53, 5]]],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there another implementation somewhere (maybe stdlib calendar) we can use to compare against?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess datetime.isocalendar would make sense

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/python/cpython/blob/master/Modules/_datetimemodule.c#L3355

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbrockmendel I didn't want to change the current implementation too much, but I tested with datetime.isocalendar locally.

#36050 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add in tests.tslibs.test_ccalendar (OK for follow-up):

from hypothesis import given, strategies as st

@given(st.datetimes(min_value=pd.Timestamp.min.to_pydatetime(), max_value=pd.Timestamp.max.to_pydatetime())
def test_isocalendar(dt):
    expected = dt.isocalendar()
    result = ccalendar.get_iso_calendar(dt.year, dt.month, dt.day)
    assert result == expected

Copy link
Contributor Author

@asishm asishm Sep 3, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbrockmendel Unsure how to resolve the CI failures. should I just add warn=False in the to_pydatetime call?
https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=41452&view=ms.vss-test-web.build-test-results-tab&runId=2135520&resultId=100016&paneView=debug

],
)
def test_isocalendar(self, input_series, expected_output):
Expand Down
15 changes: 15 additions & 0 deletions pandas/tests/tslibs/test_ccalendar.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
from datetime import date, datetime

from hypothesis import given, strategies as st
import numpy as np
import pytest

from pandas._libs.tslibs import ccalendar

import pandas as pd


@pytest.mark.parametrize(
"date_tuple,expected",
Expand Down Expand Up @@ -48,3 +51,15 @@ def test_dt_correct_iso_8601_year_week_and_day(input_date_tuple, expected_iso_tu
expected_from_date_isocalendar = date(*input_date_tuple).isocalendar()
assert result == expected_from_date_isocalendar
assert result == expected_iso_tuple


@given(
st.datetimes(
min_value=pd.Timestamp.min.to_pydatetime(warn=False),
max_value=pd.Timestamp.max.to_pydatetime(warn=False),
)
)
def test_isocalendar(dt):
expected = dt.isocalendar()
result = ccalendar.get_iso_calendar(dt.year, dt.month, dt.day)
assert result == expected