Skip to content

read_json and orient='table' Incorrect For MI And Some Missing Values #19130

Closed
@WillAyd

Description

@WillAyd

Code Sample, a copy-pastable example if possible

df = pd.DataFrame([[1,2,3,4]], columns=['a', 'b', 'c', 'd'], 
    index=pd.MultiIndex.from_product([('a', 'b'), ('c', 'd')],  
                                      names=['n1', None]))
a b c d
n1
a c 1 2 3 4
d 1 2 3 4
b c 1 2 3 4
d 1 2 3 4
df.to_json('test.json', orient='table')
pd.read_json('test.json', orient='table')
a b c d
n1 level_1
a c 1 2 3 4
d 1 2 3 4
b c 1 2 3 4
d 1 2 3 4

level_1 comes back as the second level index name, even though it should be None. The problem is with the below line of code that checks if all of the levels of a MultiIndex have missing names, but doesn't handle correctly when only some are missing.

if all(x.startswith('level_') for x in df.index.names):

Will have to update that line of code and add this to the documentation notes, similar to the callout that is made for the literal string index not being roudtrippable with a normal Index

Output of pd.show_versions()

INSTALLED VERSIONS

commit: 16d0262
python: 3.6.3.final.0
python-bits: 64
OS: Darwin
OS-release: 17.3.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.0.dev0+79.g16d026212
pytest: 3.2.1
pip: 9.0.1
setuptools: 36.5.0.post20170921
Cython: 0.26.1
numpy: 1.13.3
scipy: 1.0.0
pyarrow: 0.8.0
xarray: 0.10.0
IPython: 6.2.1
sphinx: 1.6.3
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.4
feather: 0.4.0
matplotlib: 2.1.1
openpyxl: 2.5.0b1
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.1
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: 1.1.13
pymysql: 0.7.11.None
psycopg2: None
jinja2: 2.10
s3fs: 0.1.2
fastparquet: 0.1.3
pandas_gbq: None
pandas_datareader: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions