Skip to content

Do not manually loop over all rows when reading a dataframe #97

Closed
@tswast

Description

@tswast

See: #25 (comment)

Perhaps there is a faster way to construct a dataframe from the results returned by the client library than looping over rows individually?

Note: the client library ends up effectively looping over all rows as well by returning an iterator that does the type conversions / parsing over the actual API results. I imagine some profiling might reveal places where the performance there can also be improved.

P.S. version 0.29.0 of the BigQuery client library (not yet released, as of 2017-12-08) will expose a to_dataframe() method. The actual implementation of this issue may be to just use that method here.

https://github.com/GoogleCloudPlatform/google-cloud-python/blob/061011d0213f82ca5ccaa9dec0a12713faaa2899/bigquery/google/cloud/bigquery/table.py#L1103-L1123

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions