Skip to content

DateTimeIndex.tz_convert() does not apply DST from 2038 onward #33061

Closed
@obeavers

Description

@obeavers

Code Sample, a copy-pastable example if possible

assert np.all(pd.date_range('1/1/2037', periods=8760, freq='H', tz='EST').time == pd.date_range('1/1/2037', periods=8760, freq='H', tz='US/Eastern').time) == False

assert np.all(pd.date_range('1/1/2038', periods=8760, freq='H', tz='EST').time == pd.date_range('1/1/2038', periods=8760, freq='H', tz='US/Eastern').time) == False # fails

assert np.all(pd.date_range('1/1/2039', periods=8760, freq='H', tz='EST').time == pd.date_range('1/1/2039', periods=8760, freq='H', tz='US/Eastern').time) == False # fails

Problem description

Wow, this one hurt. US/Eastern timezone is DST-adjusted (blend of EST/EDT) whereas EST is just EST.

The second and third assert statements above should both return False.

Surprised this hasn't come up before.

This is apparently related to a UNIX issue: https://en.wikipedia.org/wiki/Year_2038_problem. With that said, it seems the dtype is datetime64 with some pandas customizations on timezone. Supposedly 64 bit should have solved this.

Expected Output

Both of the following should pass:
assert np.all(pd.date_range('1/1/2038', periods=8760, freq='H', tz='EST').time == pd.date_range('1/1/2038', periods=8760, freq='H', tz='US/Eastern').time) == False # fails

assert np.all(pd.date_range('1/1/2039', periods=8760, freq='H', tz='EST').time == pd.date_range('1/1/2039', periods=8760, freq='H', tz='US/Eastern').time) == False # fails

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]
NSTALLED VERSIONS

commit : None
python : 3.8.1.final.0
python-bits : 64
OS : Windows
OS-release : 10
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 10, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 1.0.0
numpy : 1.18.1
pytz : 2019.3
dateutil : 2.8.1
pip : 20.0.2
setuptools : 45.1.0.post20200127
Cython : None
pytest : 5.3.5
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.1
IPython : 7.11.1
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : 2.7.1
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 0.13.0
pytables : None
pytest : 5.3.5
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : 3.6.1
tabulate : None
xarray : None
xlrd : 1.2.0
xlwt : None
xlsxwriter : None
numba : None

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions