Imprecision when converting Period to timestamp

[mkuklik]

Code Sample, a copy-pastable example if possible

pd.Period("2000-12-31 23:59:59.999").to_timestamp()

returns: 2000-12-31 23:59:59.998999999
instead of 2000-12-31 23:59:59.999
It works correctly for other dates, e.g.

pd.Period("2000-12-31 23:59:59.99").to_timestamp()

returns correctly "2000-12-31 23:59:59.990000"

Problem description

incorrect conversion of Period to Timestamp, see above.

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None

pandas: 0.23.4
pytest: None
pip: 18.1
setuptools: 39.1.0
Cython: None
numpy: 1.15.4
scipy: 1.2.0
pyarrow: None
xarray: None
IPython: 7.2.0
sphinx: None
patsy: 0.5.1
dateutil: 2.7.5
pytz: 2018.7
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[jreback]

on master. looks ok to me

In [10]: p = pd.Period('2000-12-31 23:59:59.99')

In [11]: p.freq
Out[11]: <Milli>

In [12]: pd.Period('2000-12-31 23:59:59.99').to_timestamp()
Out[12]: Timestamp('2000-12-31 23:59:59.990000')

In [13]: pd.Period('2000-12-31 23:59:59.99').to_timestamp(how='end')
Out[13]: Timestamp('2000-12-31 23:59:59.990999998')

In [14]: pd.Period('2000-12-31 23:59:59.99').to_timestamp(how='start')
Out[14]: Timestamp('2000-12-31 23:59:59.990000')

In [15]: pd.Period('2000-12-31 23:59:59.99').to_timestamp(how='end') - pd.Period('2000-12-31 23:59:59.99').to_timestamp(how='start')
    ...: 
Out[15]: Timedelta('0 days 00:00:00.000999')

@mkuklik can you explain why you think this should be different

[mkuklik]

That is correct, pd.Period('2000-12-31 23:59:59.99').to_timestamp() works file.
The issue is with (three 9's at the end)

pd.Period('2000-12-31 23:59:59.999').to_timestamp()

[gfyoung]

I wonder if this is symptomatic of an intrinsic Python rounding issue?

[jreback]

this also looks fine to me

In [1]: p = pd.Period('2000-12-31 23:59:59.999')

In [2]: p.to_timestamp()
Out[2]: Timestamp('2000-12-31 23:59:59.998999999')

In [3]: p.to_timestamp().round('1 ms')
Out[3]: Timestamp('2000-12-31 23:59:59.999000')

In [4]: p.to_timestamp(how='end') - p.to_timestamp(how='start')
Out[4]: Timedelta('0 days 00:00:00.001000')

[mkuklik]

How is it fine if 2000-12-31 23:59:59.998999999 != 2000-12-31 23:59:59.999 this conversion should be consistent across all the values without having to round up in some cases.

…

On Sun, Dec 30, 2018 at 1:26 PM Jeff Reback ***@***.***> wrote: this also looks fine to me In [1]: p = pd.Period('2000-12-31 23:59:59.999') In [2]: p.to_timestamp() Out[2]: Timestamp('2000-12-31 23:59:59.998999999') In [3]: p.to_timestamp().round('1 ms') Out[3]: Timestamp('2000-12-31 23:59:59.999000') In [4]: p.to_timestamp(how='end') - p.to_timestamp(how='start') Out[4]: Timedelta('0 days 00:00:00.001000') — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24444 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFjHo6ogXIq-MGZkovc3Dvh_RP4yHu2Nks5u-QVggaJpZM4ZiHl7> .

[jreback]

@mkuklik you are welcome to have a look. I would say we are round tripping thru floats; these are natively int64.

[mroeschke]

Looks to work on master now. Could use a test

In [4]: pd.Period('2000-12-31 23:59:59.999').to_timestamp()
   ...:
Out[4]: Timestamp('2000-12-31 23:59:59.999000')

In [5]: pd.__version__
Out[5]: '1.1.0.dev0+1536.ga7fb88fd5'