Skip to content

HDF store not closed on AssertionError #28430

Open
@adimascio

Description

@adimascio

Code Sample, a copy-pastable example if possible

Code adapted from https://github.com/ToucanToco/peakina/blob/master/peakina/cache.py#L84

        try:
            metadata = pd.read_hdf(filepath)
        except Exception:  # catch all, on purpose
            metadata = pd.DataFrame(columns=['key', 'mtime', 'created_at'])
            metadata.to_hdf(filepath, key, mode='w')

        return metadata
        

Problem description

In read_hdf, if the store.select() call throws either a ValueError, a TypeError or a KeyError then the store is closed.

However, if any other exception is thrown (e.g. an AssertionError if there are gaps in blocks ref_loc) , the store is not closed and some client code could end up trying to reopen the store and hit an error like: the file XXX is already opened. Please close it before reopening in write mode.

Expected Output

No reentrant exception.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : None python : 3.6.8.final.0 python-bits : 64 OS : Linux OS-release : 4.19.0-5-amd64 machine : x86_64 processor : byteorder : little LC_ALL : None LANG : fr_FR.UTF-8 LOCALE : fr_FR.UTF-8

pandas : 0.25.1
numpy : 1.17.2
pytz : 2019.2
dateutil : 2.8.0
pip : 19.2.3
setuptools : 41.2.0
Cython : None
pytest : 5.1.2
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : 2.7.0
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 0.14.1
pytables : None
s3fs : 0.3.4
scipy : None
sqlalchemy : None
tables : 3.5.2
xarray : None
xlrd : 1.2.0
xlwt : None
xlsxwriter : None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions