Issues with native compilation using S3 sdk

[cmoine]

This ticket is a subset of #1852 (Make SDK easily compileable to GraalVM native image)

Describe the issue

The AWS S3 Client has an awkward behavior with some characters in the URL (character ' ', '@' for instance) with native compilation using GraalVM

Steps to Reproduce

Use the following snippet:

public class Main {

    public static void main(String[] args) {
        S3Client client = S3Client.create();
        // This one works natively
        System.out.println("object size: " + client.headObject(HeadObjectRequest.builder().bucket("mybucket").key("destDir/testProductDto.json").build()).contentLength());
        // This one fails (/!\ only with native compilation)
        System.out.println("object size: " + client.headObject(HeadObjectRequest.builder().bucket("mybucket").key("destDir/commodule-9@2.bin").build()).contentLength());
    }
}

Current Behavior

The second invocation raises this exception:

Exception in thread "main" software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status Code: 403, Request ID: null, Extended Request ID: jyQ8wJCeXzMYD68kQq9Wb9Mml1jvThAaVGWvR7S7CRdstWtflbY6tj1pDfOg8rzUOAJ3PK1K8U8
=)
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:106)
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:84)
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:42)
        at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:94)
        at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$4(BaseClientHandler.java:214)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30)
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:64)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34)
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
        at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
        at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:189)
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:121)
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:147)
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:101)
        at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
        at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
        at software.amazon.awssdk.services.s3.DefaultS3Client.headObject(DefaultS3Client.java:4080)
        at Main.main(Main.java:160)

Our investigation so far regarding this issue:

• We have compared Aws4Signer behavior with JVM and native compilation, both seem to behave the same
• We have compared both URL (there was a warning during native execution (NoSuchMethodException was thrown when disabling normalizeUri. This indicates you are using an old version (< 4.5.8) of Apache http client. It is recommended to use http client version >= 4.5.9 to avoid the breaking change introduced in apache client 4.5.7 and the latency in exception handling. See org.apache.httpcomponents:httpclient:4.5.7 breaks fetching S3 objects with consecutive slashes in the key aws-sdk-java#1919 for more information)", so we added "org/apache/http/client/config/RequestConfig$Builder" to the reflect config, and the warn disappeared, but no différence)

Your Environment

• AWS Java SDK version used:
  v2.13.47
• JDK version used:
  OpenJDK 64-Bit Server VM GraalVM 20.1.1-dev (build 25.252-b09-jvmci-20.1-b02, mixed mode)
• Operating System and version:
  Linux 64bits

[cmoine]

After 3 days of investigation, I finally found the problem: S3 SDK use execution interceptors. The execution interceptor is loaded dynamically, and thus requires to add this in the resources-config.json file:

{"pattern":"\\Qsoftware/amazon/awssdk/services/s3/execution.interceptors\\E"}

After that modification, the classes listed in that file required to be accessible reflectively in "reflect-config.json" file:

{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.AddContentMd5HeaderInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.AsyncChecksumValidationInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.CreateBucketInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.CreateMultipartUploadRequestInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.DecodeUrlEncodedResponseInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.DisableDoubleUrlEncodingInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.EnableChunkedEncodingInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.EnableTrailingChecksumInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.EndpointAddressInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.ExceptionTranslationInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.GetBucketPolicyInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.GetObjectInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.PutObjectInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
},
{
  "name":"software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor",
  "methods":[{"name":"<init>","parameterTypes":[] }]
}

If this can helps other people, I'll be glad about that!

As a conclusion, I have been punished for not using the native agent as described here:
https://www.graalvm.org/reference-manual/native-image/Configuration/#assisted-configuration-of-native-image-builds

[jmahonin]

Just adding a comment here for anyone who has also fallen down this rabbit-hole. The execution.interceptors functionality was removed in AWS SDK 2.17.46
https://github.com/aws/aws-sdk-java-v2/blob/f941b3f7ae1a35e289c10be940757525feed8290/.changes/2.17.46.json