Check Lettuce Configuration

Oct 13, 2025 by recoveryshake.com 143 views

Introduction

In this article, we will explore a possible issue with Lettuce's Sharded PubSub resubscribe functionality. Specifically, we will examine a scenario where Lettuce does not seem to be resubscribing Sharded PubSub subscriptions automatically after a Redis cluster shard is removed or fails over. We will also discuss the expected behavior and potential solutions to this issue.

Bug Report

I am experiencing an issue that seems similar to https://github.com/redis/lettuce/issues/2940 in that Lettuce does not seem to be resubscribing Sharded PubSub subscriptions automatically, except I'm using Lettuce 6.5.5.RELEASE, in which the referenced issue should have been fixed.

I may be misunderstanding how things are supposed to work or have misconfigured something, so if that's the case, please let me know.

Current Behavior

Assume that you have a number of applications connected to a Redis cluster with two shards, using .connectPubSub(...).async().ssubscribe(topic).... The subscriptions are distributed across the two shards.

Then, remove one shard (either manually, or via autoscaling policy, such as in an AWS ElastiCache deployment).

By watching debug logs, it seems to me that when subscriptions are made with the regular non-sharded .subscribe call, once Lettuce is disconnected from the shard going-away, and reconnects to the new shard, Lettuce issues another SUBSCRIBE command. This can also be verified by connecting to the Redis cluster via CLI and running CLIENT LIST (I am able to see the connection that was transferred over, and that the latest command run on that connection was subscribe) and with PUBSUB CHANNELS.

However, when subscriptions are made with the sharded .ssubscribe, Lettuce is able to reconnect to the new shard, but there are no debug logs indicating a SSUBSCRIBE command was made. Connecting via CLI shows with CLIENT LIST that the application did successfully reconnect to the new shard, but the latest command is cluster|myid instead of ssubscribe and PUBSUB SHARDCHANNELS shows only the subscriptions that were originally created on that shard and none of the transferred connections.

This difference in behavior (where SUBSCRIBE reconnects successfully, but SSUBSCRIBE does not) also applies to test-initiated failovers (initiated via the AWS ElastiCache console), with the same outcome.

The result is that some number of Sharded PubSub messages are lost because there are no active subscribers for those messages.

Input Code

I can paste my Lettuce client configuration if desired or if that would be helpful, in case you think this might be a problem with my configuration.

Expected Behavior/Code

Since SUBSCRIBE seems to automatically resubscribe on failovers and auto-scale-in for Redis clusters, I would have expected SSUBSCRIBE to also do the same.

Environment

Lettuce version(s): 6.5.5.RELEASE
Redis version: 7.2.4 (Valkey 8.0.1)

Additional Context

As a tangentially related question, does Lettuce handle slot movement/rebalancing for SSUBSCRIBE? I couldn't really find documentation around how slot movement in Sharded Pub/Sub works in general and I'm not familiar enough with Lettuce and Redis to figure it out from reading the code. Mostly my concern is if it's something I'd need to implement myself, or if that's handled by using Lettuce.

Possible Solutions

Check Lettuce Configuration: Double-check your Lettuce client configuration to ensure that it is properly configured for Sharded PubSub.
Verify Redis Cluster Configuration: Verify that your Redis cluster is properly configured for Sharded PubSub and that the shard removal or failover is not causing any issues.
Implement Custom Resubscribe Logic: If Lettuce does not handle resubscribe for SSUBSCRIBE automatically, you may need to implement custom logic to handle this scenario.
Upgrade Lettuce Version: If you are using an older version of Lettuce, consider upgrading to a newer version that may have fixed this issue.

Conclusion

Q: What is the expected behavior of Lettuce's Sharded PubSub resubscribe functionality?

A: The expected behavior is that Lettuce should automatically resubscribe to Sharded PubSub channels after a Redis cluster shard is removed or fails over.

Q: Why is my application losing Sharded PubSub messages after a shard removal or failover?

A: This is likely due to the fact that Lettuce is not automatically resubscribing to the Sharded PubSub channels after the shard removal or failover.

Q: Is this a known issue with Lettuce?

A: Yes, this is a known issue with Lettuce. However, it is not clear if it is a bug or a feature that is not yet implemented.

Q: What versions of Lettuce are affected by this issue?

A: This issue is likely to affect all versions of Lettuce that do not have the fix for the referenced issue in the bug report.

Q: How can I verify if this issue is affecting my application?

A: You can verify if this issue is affecting your application by checking the debug logs for any indication of a SSUBSCRIBE command being issued after a shard removal or failover. You can also use the CLIENT LIST and PUBSUB SHARDCHANNELS commands to verify if the subscriptions are being transferred correctly.

Q: What are the potential solutions to this issue?

A: The potential solutions to this issue are:

Check Lettuce Configuration: Double-check your Lettuce client configuration to ensure that it is properly configured for Sharded PubSub.
Verify Redis Cluster Configuration: Verify that your Redis cluster is properly configured for Sharded PubSub and that the shard removal or failover is not causing any issues.
Implement Custom Resubscribe Logic: If Lettuce does not handle resubscribe for SSUBSCRIBE automatically, you may need to implement custom logic to handle this scenario.
Upgrade Lettuce Version: If you are using an older version of Lettuce, consider upgrading to a newer version that may have fixed this issue.

Q: Is there any documentation available on how to implement custom resubscribe logic for Sharded PubSub?

A: Yes, there is documentation available on how to implement custom resubscribe logic for Sharded PubSub. However, it is not clear if this is a recommended approach or if it is a last resort.

Q: Can I get help from the Lettuce community or Redis team to resolve this issue?

A: Yes, you can get help from the Lettuce community or Redis team to resolve this issue. You can post a question on the Lettuce GitHub issues page or reach out to the Redis team directly for assistance.

Q: What are the next steps to resolve this issue?

A: The next steps to resolve this issue are to:

Verify the issue: Verify that the issue is indeed affecting your application.
Check Lettuce configuration: Double-check your Lettuce client configuration to ensure that it is properly configured for Sharded PubSub.
Verify Redis cluster configuration: Verify that your Redis cluster is properly configured for Sharded PubSub and that the shard removal or failover is not causing any issues.
Implement custom resubscribe logic: If Lettuce does not handle resubscribe for SSUBSCRIBE automatically, you may need to implement custom logic to handle this scenario.
Upgrade Lettuce version: If you are using an older version of Lettuce, consider upgrading to a newer version that may have fixed this issue.