Skip to content

feat(ring): add GetShardClients and GetShardClientForKey methods to Ring for shard access #3388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 27, 2025

Conversation

DengY11
Copy link
Contributor

@DengY11 DengY11 commented May 23, 2025

This PR adds two new methods to the Ring client that provide access to individual shard clients, addressing the need to create dedicated connections (e.g., PubSub) for each shard.

When using Redis Ring with PubSub connections, users need to create separate PubSub connections for each shard because PubSub connections enter a special mode and cannot be used for other Redis operations. Currently, there's no way to access the underlying shard clients directly, making it impossible to pre-create and reuse PubSub connections related issue

This PR introduces two new public methods to the Ring struct, users can pre-create and reuse PubSub connections per shard

  1. GetShards() []*Client
  • Returns a list of all currently active shard clients
  • Only includes shards that are marked as "up"
  • Allows users to iterate over all available shards
  1. GetShardByKey(key string) (*Client, error)
  • Returns the shard client that would handle a specific key/channel
  • Uses the same consistent hashing logic as the Ring's internal routing
  • Enables users to determine which shard a particular key would be routed to

Checklist:

  • Code follows project style guidelines
  • Tests pass
  • New functionality is documented
  • Backward compatibility maintained
  • Uses existing patterns and conventions
  • Test coverage includes edge cases
  • No security vulnerabilities introduced

closes #3341

- Add GetShards() method to retrieve a list of active shard clients.
- Add GetShardByKey(key string) method to get the shard client for a specific key.
- These methods enable users to manage Pub/Sub operations more effectively by accessing shard-specific clients.
@ndyakov
Copy link
Member

ndyakov commented May 27, 2025

Hello @DengY11 , thank you for this contribution. Please write tests for the code you are introducing.

On a side note, would you like to share your usecase of Redis Ring and why you decided to use Ring instead of a Redis OSS Cluster?

@DengY11
Copy link
Contributor Author

DengY11 commented May 27, 2025

hi @ndyakov
Thanks for looking at this!
yep, the tests are in ring_test.go (Ginkgo/Gomega) under the Describe("Ring GetShards and GetShardByKey", ...) block. Sorry if that wasn't clear!
Quickly on why ring for us: we use it for a busy Pub/Sub system. Ring's simpler client-side sharding works well with our current standalone Redis setup. We don't need the full Redis OSS Cluster complexity for this specific Pub/Sub part.
These new methods are super helpful because they let us:
Grab all shard clients to set up dedicated Pub/Sub connections upfront.
Figure out which shard's Pub/Sub connection to use for each channel, using Ring's own logic.
This is key for our Pub/Sub performance, and it's tricky to do cleanly without these helpers.

@ndyakov ndyakov self-requested a review May 27, 2025 13:32
@ndyakov
Copy link
Member

ndyakov commented May 27, 2025

@DengY11 now I can see the tests, for some reason the diff was not complete. Anyway, would you mind renaming the new functions to GetShardClients and GetShardClientForKey ? As for the usecase, we are collecting information regarding Ring since we are discussing extracting it from the client since Redis Cluster covers mostly the same usecases. Thank you for providing that information.

@DengY11
Copy link
Contributor Author

DengY11 commented May 27, 2025

@ndyakov Thanks for the quick response and the suggestions!
I've updated the PR with the function names changed as you recommended:
GetShards is now GetShardClients
GetShardByKey is now GetShardClientForKey
The latest commits have been pushed.
I also appreciate you sharing the insight into the discussions about Ring's future. It's helpful to understand the broader context. Our use case, particularly for streamlining Pub/Sub sharding with standalone instances where Ring's simplicity is a benefit, hopefully adds a useful perspective to those discussions.
Thanks again for your time and guidance!

@ndyakov ndyakov changed the title feat: add GetShards and GetShardByKey methods to Ring for shard access feat(ring): add GetShardClients and GetShardClientForKey methods to Ring for shard access May 27, 2025
Copy link
Member

@ndyakov ndyakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looks good, we have flaky tests right now. I will investigate tomorrow. Merging this to include in v9.9.0

@ndyakov ndyakov merged commit cb1968c into redis:master May 27, 2025
29 of 36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make shards in redis.Ring publicly accessible
2 participants