Skip to content

UInt64Index: lookup by label gives KeyError for values over 2**63-1 #19399

Closed
@mdickinson

Description

@mdickinson

[From a StackOverflow question]

Lookup in a UInt64Index appears to fail for values larger than 2**63-1 (but smaller than 2**64). Here's a simple example session (Pandas 0.22.0, Python 3.6.)

>>> import pandas as pd
>>> s = pd.Series({0x7fff_ffff_ffff_ffff: 3, 0x8000_0000_0000_0000: 4})
>>> type(s.index)
<class 'pandas.core.indexes.numeric.UInt64Index'>
>>> s.loc[0x7fff_ffff_ffff_ffff]
3
>>> s.loc[0x8000_0000_0000_0000]
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 1505, in _has_valid_type
    if not ax.contains(key):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 1716, in contains
    return key in self._engine
  File "pandas/_libs/index.pyx", line 81, in pandas._libs.index.IndexEngine.__contains__
  File "pandas/_libs/hashtable_class_helper.pxi", line 591, in pandas._libs.hashtable.UInt64HashTable.__contains__
OverflowError: Python int too large to convert to C long

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 1373, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 1626, in _getitem_axis
    self._has_valid_type(key, axis)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 1514, in _has_valid_type
    error()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 1501, in error
    axis=self.obj._get_axis_name(axis)))
KeyError: 'the label [9223372036854775808] is not in the [index]'

I would have expected the second lookup to succeed, returning 4.

Output of pd.show_versions()

>>> pd.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.6.4.final.0 python-bits: 64 OS: Darwin OS-release: 17.3.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8

pandas: 0.22.0
pytest: None
pip: 9.0.1
setuptools: 36.8.0
Cython: 0.27.3
numpy: 1.14.0
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.4
feather: None
matplotlib: 2.1.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0b10
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    IndexingRelated to indexing on series/frames, not to indexes themselvesTestingpandas testing functions or related to the test suite

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions