Using Lettuce with `RedisMessageListenerContainer` leaves open connections

[quaff]

Warning show up after I upgrade spring-data-redis to 2.7.0.

o.s.d.r.c.l.LettucePoolingConnectionProvider WARN LettucePoolingConnectionProvider contains unreleased connections

Leaked connection was released via LettucePoolingConnectionProvider::release caused by RedisMessageListenerContainer::stop in spring-data-redis 2.6.x

Thread [restartedMain] (Suspended (breakpoint at line 163 in LettucePoolingConnectionProvider))	
	owns: Object  (id=281)	
	owns: Object  (id=282)	
	LettucePoolingConnectionProvider.release(StatefulConnection<?,?>) line: 163	
	LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.release(StatefulConnection<?,?>) line: 1664	
	LettuceSubscription.doClose() line: 105	
	LettuceSubscription(AbstractSubscription).close() line: 106	
	RedisMessageListenerContainer$SubscriptionTask.cancel() line: 880	
	RedisMessageListenerContainer.stop() line: 238	
	RedisMessageListenerContainer.stop(Runnable) line: 192	
	DefaultLifecycleProcessor.doStop(Map<String,Lifecycle>, String, CountDownLatch, Set<String>) line: 234	
	DefaultLifecycleProcessor.access$300(DefaultLifecycleProcessor, Map, String, CountDownLatch, Set) line: 54	
	DefaultLifecycleProcessor$LifecycleGroup.stop() line: 373	
	DefaultLifecycleProcessor.stopBeans() line: 206	
	DefaultLifecycleProcessor.onClose() line: 129	
	AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).doClose() line: 1067	
	AnnotationConfigServletWebServerApplicationContext(ServletWebServerApplicationContext).doClose() line: 172	
	AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).close() line: 1021	
	SpringApplication.close(ApplicationContext) line: 1370	
	SpringApplication.exit(ApplicationContext, ExitCodeGenerator...) line: 1357	
	DefaultApplication.start(String[], Consumer<ApplicationContext>) line: 125	
	DefaultApplication.start(String[]) line: 104	
	MainApplication.main(String[]) line: 19	
	NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
	NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62	
	DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
	Method.invoke(Object, Object...) line: 566	
	RestartLauncher.run() line: 49	

I notice that RedisMessageListenerContainer is revised by 431dfc2, maybe the refactor is broken, please check it.

[mp911de]

Can you provide a reproducer? It looks as if there was a shutdown ordering issue where the connection pool is closed before the message listener is closed.

[quaff]

RedisMessageListenerContainer is closed before LettucePoolingConnectionProvider, I'm trying to create a minimal reproducer, It's a bit hard.

[quaff]

@mp911de Is it possible that caused by subscriber not cancelled when RedisMessageListenerContainer::stop?

spring-data-redis/src/main/java/org/springframework/data/redis/listener/RedisMessageListenerContainer.java

Line 133 in 587600a

private @Nullable Subscriber subscriber;

[mp911de]

We introduced synchronization in RedisMessageListenerContainer.stop() to await completion of the unsubscription to avoid concurrency during the shutdown (and startup) phase. Need to check what's happening there.

[mp911de]

I wasn't able to reproduce the issue with a minimal set of beans and connection pooling enabled.

My reproducer looks like:

@SpringBootApplication
public class RedisReproApplication {

	public static void main(String[] args) {
		SpringApplication.run(RedisReproApplication.class, args);
	}


	@Bean
	public RedisMessageListenerContainer mlc(RedisConnectionFactory rcf) {

		RedisMessageListenerContainer container = new RedisMessageListenerContainer();
		container.setConnectionFactory(rcf);
		container.addMessageListener((message, pattern) -> {

			System.out.println("Message : " + new String(message.getChannel()) + ": " + new String(message.getBody()));

		}, new PatternTopic("*"));

		return container;
	}

}

[quaff]

@mp911de I couldn't reproduce it yet, It happens in my actual project. I notice LettucePoolingConnectionProvider.destroy() is called twice, is it intentional?

[quaff]

It breaks TracingRedisConnectionFactory from io.opentracing.contrib:opentracing-redis-spring-data2, here is reproducer example.zip.