ENH: New Name for "numpy_nullable" dtype_backend

[WillAyd]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

Many I/O methods today accept a "numpy_nullable" argument for the dtype_backend= parameter. While historically our extension arrays exclusively used NumPy, this is no longer true with the string dtype so the name "numpy_nullable" is a misnomer.

Feature Description

To make for a less confusing API, I would suggest adding "pandas_nullable" or maybe even just "pandas" as an argument. This can have the exact same behavior as "numpy_nullable" today but abstracts and corrects the semantics. "numpy_nullable" can be slowly deprecated over time

Alternative Solutions

n/a

Additional Context

dtype_backend="pandas" would also make for a smoother transition into the logical type system proposed as part of PDEP-13 #58455

...but even if that PDEP is not accepted, I still see value in changing the value "numpy_nullable" to something else

[WillAyd]

@jorisvandenbossche maybe a good follow up to the discussion we had as part of PDEP-14

[WillAyd]

@pandas-dev/pandas-core this wasn't major enough to include as part of PDEP-14, but I think is a logical follow up to clean up semantics. Curious what others may think

[Dr-Irv]

I think it should be pandas_nullable . Keeps options open with respect to the whole pd.NA/np.nan discussion

[chaarvii]

Hey! I’d like to work on this

[chaarvii]

Take

[WillAyd]

Any other team feedback on this? I think would be good to use the new name starting with 3.0

[simonjayhawkins]

We have pandas.api.types.pandas_dtype where we Convert input into a pandas only dtype object ... and this returns np.dtype or a pandas dtype.

Given that the term “pandas dtype” already has a precedent, using dtype_backend="pandas" would indeed align well with existing conventions. It provides clarity and maintains consistency.

[WillAyd]

I also have a slight preference for pandas because it is shorter, and I don't see us every introducing a non-nullable type system, so "_nullable" is superfluous

[jorisvandenbossche]

On the other hand, when specifying dtype_backend="pyarrow", you also get back a "pandas dtype" in that sense (i.e. a pandas ExtensionDtype subclass). And at the same time, some of the non-nullable default dtypes we have are also pandas dtypes.

So I don't think dtype_backend="pandas" is an ideal naming, but I also don't have any better suggestion ..