Skip to content

BUG: EAs with list-like values fail in to_csv (due to to_native_types) #28840

Closed
@jorisvandenbossche

Description

@jorisvandenbossche

From a report at GeoPandas: geopandas/geopandas#1145

Explanation: assume you have an ExtensionArray with list-like values (eg with an object numpy array with lists under the hood). When calling, to_csv, which goes through Block.to_native_types, we convert this to a numpy array and call astype(str):

values = values.astype("<U{size}".format(size=itemsize))

But, in numpy, if you have a numpy array with list like values, doing astype(str) does not work:

In [2]: a = np.array([[1, 2], [3]], dtype=object) 

In [3]: a.astype(str)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-db38302b5f9a> in <module>
----> 1 a.astype(str)

ValueError: setting an array element with a sequence

This could be considered a numpy issue, but, we should also avoid converting to a numpy array while we can directly call astype on the ExtensionArray.

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugExtensionArrayExtending pandas with custom dtypes or arrays.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions