Skip to content

PERF: json should process data column-by-column (and not use .values) #9037

Closed
@jreback

Description

@jreback

xref #9027

So to_json first converts all the a DataFrame by using .values. This converts everything to object dtype (if its a mixed frame) and is pretty expensive perf wise.

If you do this column by column you get excellent perf (and tiny bit more complexity in the code).

cc @Komnomnomnom
@cpcloud

Metadata

Metadata

Assignees

No one assigned

    Labels

    IO JSONread_json, to_json, json_normalizePerformanceMemory or execution speed performance

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions