Skip to content

Precalculate and store repository activities #34133

Open
@TheFox0x7

Description

@TheFox0x7

Feature Description

Lately we had some issues like this one which touched on git timing out too soon to calculate something. I'd like to propose precalculating activity stats and storing them instead of making git calls which might be expensive on larger repositories.

The idea I had is to keep the already calculated data (files changed, deletions, additions, etc.) in a iceberg/parquet file, tied to repository.
When user requests activity data from the last year, instead running git and recalculate it all over again, the data would be fetched from the file.
This way things like getExtendedCommitStats can reuse this data and avoid spawning long running git commands.

Ideally it would be append only, with the exception being force push, where a history could be rewritten.

Screenshots

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    performance/speedperformance issues with slow downstype/proposalThe new feature has not been accepted yet but needs to be discussed first.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions