Skip to content

perf: add shuffle_list function #744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 13, 2024
Merged

perf: add shuffle_list function #744

merged 3 commits into from
Feb 13, 2024

Conversation

mpaulucci
Copy link
Collaborator

@mpaulucci mpaulucci commented Feb 12, 2024

Motivation
Shuffling a whole list should be much more efficient than calling compute_shuffled_index on every single item.

Description
Lighthouse implementations claims it can be up to 250x more efficient than computing compute_shuffled_index through the entire list, see: https://github.com/sigp/lighthouse/blob/c11638c36c25f3af910eecd33a6e45d1caa99f1f/consensus/swap_or_not_shuffle/src/shuffle_list.rs#L55

In my case I've observed "only" 10x improvements so far:

Name                             ips        average  deviation         median         99th %
shuffle_list                  254.79        3.92 ms    ±10.57%        3.90 ms        4.29 ms
compute_shuffled_index         22.14       45.17 ms     ±2.66%       45.24 ms       52.94 ms

Comparison:
shuffle_list                  254.79
compute_shuffled_index         22.14 - 11.51x slower +41.25 ms

TODO: Integrate this function into the code.

Closes #467

@mpaulucci mpaulucci requested a review from a team as a code owner February 12, 2024 18:31
Copy link
Collaborator

@Arkenan Arkenan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with some optional code ideas

@mpaulucci mpaulucci merged commit 5929b33 into main Feb 13, 2024
@mpaulucci mpaulucci deleted the shuffle-list branch February 13, 2024 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

Add function to shuffle an entire list instead of one element at a time
2 participants