Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
import pyarrow as pa
import pyarrow.parquet as pq
fields = [
pa.field('int_column',pa.int32()),
pa.field('float_column', pa.float32()),
pa.field('bool_column', pa.bool_()),
]
schema = pa.schema(fields)
data = {'int_column': [1, 2, None, 4],
'float_column': [1.5, 2.5, 3.5, None],
'bool_column': [True, True, None, False]}
pa_table = pa.table(raw_data, schema=schema)
pq.write_table(pa_table, './example_table.parquet')
df1 = pd.read_parquet('./example_table.parquet')
df2 = pd.read_parquet('./example_table.parquet', dtype_backend='numpy_nullable')
print(df1.dtypes)
print(df2.dtypes)Issue Description
According to the documentation of pd.read_parquet(), the default value for dtype_backend is 'numpy_nullable'.
Data types differ when not passing any argument from when passing 'numpy_nullable'.
Specifically, if not passing any argument the data types are the not nullable ones.
Expected Behavior
not passing an argument should give the same result as passing the default value.
Installed Versions
NSTALLED VERSIONS
commit : d9cdd2e
python : 3.12.4.final.0
python-bits : 64
OS : Linux
OS-release : 6.8.0-38-generic
Version : #38-Ubuntu SMP PREEMPT_DYNAMIC Fri Jun 7 15:25:01 UTC 2024
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 2.2.2
numpy : 1.26.4
pytz : 2024.1
dateutil : 2.9.0.post0
setuptools : 69.5.1
pip : 24.0
Cython : None
pytest : 7.4.4
hypothesis : None
sphinx : 7.3.7
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 5.2.1
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.4
IPython : 8.25.0
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.12.3
bottleneck : 1.3.7
dataframe-api-compat : None
fastparquet : None
fsspec : 2024.3.1
gcsfs : None
matplotlib : 3.8.4
numba : 0.59.1
numexpr : 2.8.7
odfpy : None
openpyxl : 3.1.2
pandas_gbq : None
pyarrow : 17.0.0
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : 2024.3.1
scipy : 1.13.1
sqlalchemy : 2.0.30
tables : 3.9.2
tabulate : 0.9.0
xarray : 2023.6.0
xlrd : None
zstandard : 0.22.0
tzdata : 2023.3
qtpy : 2.4.1
pyqt5 : None