DIS: Should the keyword use_nullable_dtypes use nullable dtypes in the absence of nulls

[lithomas1]

Currently the keyword use_nullable_dtypes(only implemented for read_parquet now) will use nullable dtypes for columns even when that column does not have nulls.

Should this behavior change?

xref discussion #40687 (comment) #42588 (comment)
#42588 (comment)

(Note that always using nullable dtypes always(the current behavior) is a choice made on our end https://github.com/pandas-dev/pandas/blob/master/pandas/io/parquet.py#L215-L227 not by the pyarrow engine)

cc @pandas-dev/pandas-core

[bashtage]

I would vote that it should change to only use when necessary, and to use standard dtypes when there are no nulls. Users can always upcast to nullable should they want to use nulls.

[phofl]

I think we can close this. We implemented this for other functions like for read_parquet, which makes the most sense imo

[MarcoGorelli]

Agreed - personally I'm not keen on value-dependent behaviour such as

only use when necessary, and to use standard dtypes when there are no nulls

so I'd vote for keeping as-is (use_nullable_dtypes uses nullable types - which is clear and simple)

[lithomas1]

We'll have to tell fastparquet then. I think they still do it the other way.

[phofl]

Could you open an issue there?