Skip to content

Implement update_index background worker task #4172

Closed
@Turbo87

Description

@Turbo87

We currently have an add_crate background worker task, that gets called when a new crate (or version) is published. This task appends the version metadata to the file in the package index repository which corresponds to the crate.

We currently have another background worker task, yank, which gets called when a version is yanked (or unyanked). This tasks changes the yanked value in the version metadata in the package index repository.

Our background worker implementation (https://github.com/sgrif/swirl/) mostly runs tasks in order, but, due to built-in retry logic, that is not guaranteed. This means we could end up in a situation where the tasks are run in an order like: unyank, yank, yank, publish, unyank.

Instead of telling the background worker explicitly what action should be performed, it would be better to only tell it to update the package index repository according to the current state in the database.

We should implement a new update_index(crate_name, version_num) background worker task, that looks at the state of the index and of the database and then updates the index according to the current state of the database.

In a second step that background worker tasks should ideally then also handle the cases where versions (or even crates) are completely removed from the database due to e.g. GDPR requests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-backend ⚙️C-internal 🔧Category: Nonessential work that would make the codebase more consistent or clear

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions