Description
StsWebIdentityCredentialsProvider in the default credential provider chain never gets closed. This causes a created STS client to never get closed, which in turn causes the IdleConnectionReaper to keep on to connection managers.
Expected Behavior
Default clients and credential provider chains should clean up any resources it created after being closed.
Current Behavior
DefaultCredentialsProvider#close calls LazyAwsCredentialsProvider#close, which calls IoUtils.closeIfCloseable, but the provided delegate is not an AutoClosable type. It is actually a Lazy<AwsCredentialsProvider>, and Lazy<T> does not implement AutoClosable. So the chain (and consequently the STS client) in the delegate field never receives a close() call.
Screenshot of the debugger
Stack dump from IntelliJ
closeIfCloseable:89, IoUtils (software.amazon.awssdk.utils) [2]
close:50, LazyAwsCredentialsProvider (software.amazon.awssdk.auth.credentials.internal)
close:109, DefaultCredentialsProvider (software.amazon.awssdk.auth.credentials)
closeQuietly:70, IoUtils (software.amazon.awssdk.utils)
closeIfCloseable:87, IoUtils (software.amazon.awssdk.utils) [1]
lambda$close$0:86, AttributeMap (software.amazon.awssdk.utils)
accept:-1, 912038923 (software.amazon.awssdk.utils.AttributeMap$$Lambda$1914)
forEach:981, HashMap$Values (java.util)
close:86, AttributeMap (software.amazon.awssdk.utils)
close:79, SdkClientConfiguration (software.amazon.awssdk.core.client.config)
close:80, HttpClientDependencies (software.amazon.awssdk.core.internal.http)
close:73, AmazonSyncHttpClient (software.amazon.awssdk.core.internal.http)
close:118, BaseSyncClientHandler (software.amazon.awssdk.core.internal.handler)
close:12198, DefaultIamClient (software.amazon.awssdk.services.iam)
discover:37, IamServiceRoleAwsDiscovery (com.planonsoftware.cloud.library.discovery.iamrole)
provideParameters:63, PlanonEnvironmentActivitiesImpl (com.planonsoftware.cloud.pco.orchestrator.controllers.planonenvironment.activities)
invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect)
invoke:567, Method (java.lang.reflect)
execute:214, POJOActivityTaskHandler$POJOActivityImplementation (com.uber.cadence.internal.sync)
handle:190, POJOActivityTaskHandler (com.uber.cadence.internal.sync)
handle:175, ActivityWorker$TaskHandlerImpl (com.uber.cadence.internal.worker)
handle:146, ActivityWorker$TaskHandlerImpl (com.uber.cadence.internal.worker)
lambda$process$0:71, PollTaskExecutor (com.uber.cadence.internal.worker)
run:-1, 640627828 (com.uber.cadence.internal.worker.PollTaskExecutor$$Lambda$1491)
runWorker:1128, ThreadPoolExecutor (java.util.concurrent)
run:628, ThreadPoolExecutor$Worker (java.util.concurrent)
run:830, Thread (java.lang)
Steps to Reproduce
I am running the following on a Kubernetes cluster with the AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE environment variables set. The memory just explodes, until eventually it goes out of memory.
while (true) {
try (S3Client s3 = S3Client.builder().region(Region.EU_WEST_1).build()) {
System.out.println(s3.headBucket(builder -> builder.bucket("some bucket").build()).toString());
}
}Possible Solution
For example, let Lazy implement AutoClosable, and propage close(), like in IoUtils.
Related Issues
I commented here, but its a closed issue and didn't get a response, so I'm opening this issue. #1679
Your Environment
- AWS Java SDK version used: 2.15.25
- JDK version used: 13
- Operating System and version: