I tried to follow the guide as mentioned by https://github.com/aws-powertools/powertools-lambda-java/blob/v2/GraalVM.md which helped a lot with generating the meta data.

However I do wonder if we need all the metadata generated by the GraalVM agent. In on of my projects where I used v1 of Powertools I only needed the following snippet to get the SecretsProvider to work:

[
  {
    "name" : "software.amazon.lambda.powertools.parameters.SecretsProvider",
    "allDeclaredConstructors" : true,
    "allPublicConstructors" : true,
    "allDeclaredMethods" : true,
    "allPublicMethods" : true,
    "allDeclaredFields" : true,
    "allPublicFields" : true
  }
]

Now we have all sorts of java internal classes or classes coming from other libraries or the JVM, which I don't think we need? Shouldn't they be handled by the corresponding projects? I'm by no means a GraalVM expert, but I always understood it in such a way that metadata in a provided library only contains metadata about its own classes and resources which otherwise can't be reached.

Hi @jreijn,

I just review your changes and you did an amazing job 🚀 – especially cleaning up the test scoped dependencies in the generated metadata files. This helped us a lot.

I just verified that unit tests against the native image pass, which they do ✅

Before we go ahead merging this PR, I would love to do two things:

1. Update the roadmap documentation page with the sub-issues and the progress made in the parameters module thanks to you (I can take care of this).
2. Test this at runtime using the Parameters example we have and update the example to also include GraalVM -> https://github.com/aws-powertools/powertools-lambda-java/tree/v2/examples/powertools-examples-parameters
   • For this I would take this graalvm example as inspiration and provide the user with a template-native.yaml using provided.al2023 as runtime for the Lambda.

I am happy to also update the example and don't want to cause more work on your end. But let me know if you like to do it.

Regarding your comment:

Now we have all sorts of java internal classes or classes coming from other libraries or the JVM, which I don't think we need? Shouldn't they be handled by the corresponding projects? I'm by no means a GraalVM expert, but I always understood it in such a way that metadata in a provided library only contains metadata about its own classes and resources which otherwise can't be reached.

You are absolutely right here. Since we run the metadata generation via unit tests (which is uncommon) there is a risk of including "too much" metadata and this is something we can optimize in the future. This is also why the manual cleanup is needed. The unit tests are good candidate for now since they have pretty good coverage.

One way to address this which I tested was including a filter on the GraalVM tracing agent to only generate metadata for the library itself and not dependencies. However, this led to two issues:

1. It requires customers to include metadata files for transitive dependencies that do not support GraalVM. For example, for X-RAY in the case of the tracing utility. We felt that this is a bad customer experience because the customer did not explicitly choose to include that transitive dependency.
2. In some edge cases when we access something reflectively ourselves that is outside of the filter it would not be included. Hence, a lot of manual fine-tuning on the filter is necessary which we didn't do (yet).

Having said that, there is a lot of potential on improving this and before going to production with v2 we will do some improvements (e.g. running integ tests for both JVM and GraalVM native images #1805).

It seems like in the parameters example that you also tested in v1 using GraalVM, we can definitely reduce the generated metadata files. For now, I would like to be consistent with the other modules and improve this consistently in the future. What are your thoughts on this? Happy to have a discussion, also in a short meeting if you like.

@phipag thanks for the feedback!

With regards to the two open items:

1. it would be great if you can do that

2. I will pick that up and update the example. I'll see if I can find some time over the next couple of days.

With regard to the reflection metadata I see your point. I'm fine with leaving it for now and improving it later. Once we get to that point I would love to brainstorm a bit about it and see how I can contribute.

Thanks @jreijn, this sounds like a good plan. Awesome that you would like to update the example as well! I'll be waiting for your update here and try to review it promptly.

Regarding the roadmap update, I will take care of this as part of a separate PR since I am already working on a different documentation update.

Next steps: Wait for powertools-examples-parameters update and review it to close this PR.

Hey @jreijn,
I would love to close this PR by the end of the week. Let me know if you are still working on this, otherwise I am happy to take over updating the example. Thanks again for your contributions so far!

@phipag I'm sorry for the delay. I will pick it up tomorrow! Sorry I had a busy couple of weeks!

@jreijn, no problem, we all know these busy weeks. Thanks for your quick reply. I'll be waiting for your updates.

@phipag I'm close to pushing my changes. Found some missing reflection config for the transformer that were not picked up by the tests. I'll try to push them tomorrow morning with working examples.

@phipag I've added the working example and tested it (it took me a bit longer then expected). I did find something in parameter module that wasn't obvious. If you have some more time before the release I was thinking of moving the powertools-parameters/src into powertools-parameters-core (as it's used by all parameters modules) and merge that with powertools-parameters/powertools-parameters-tests as those seem to be the tests for those classes as far as I can tell. That would allow for easy generation of graalvm metadata in the core module itself and. It also would make the maven structure a bit more standard. WDYT?

Thanks for your update @jreijn ! I will review it today.

Regarding your suggestion, this is an interesting point. Especially the fact that the tests have a dedicated package but not the core logic. I don't see a reason (right now) why not do it. I will test this and might update it before the final v2 release (target for end of Q2).

That would allow for easy generation of graalvm metadata in the core module itself

What do you mean by "easy generation". Is it easier to generate the GRM files with that new structure?

Edit: Ignore the failing workflows. It seems that GitHub is experiencing problems. https://www.githubstatus.com/

What do you mean by "easy generation". Is it easier to generate the GRM files with that new structure?

@phipag Right now I've added the graalvm maven profiles, to generate the metadata, to the powertools-parameters-test maven module and moved the generated metadata to the core sources META-INF folder. As the powertools-parameters/powertools-parameters-tests module is not used as a dependency (which they should not) for parameters-ssm, etc.

Merging tests and the src folder would allow for the same proces as in any other submodule of the project.

Hey @jreijn,

I sent a few very small commits addressing some typos and other cosmetic things. I tested your example and deployed it in my AWS account and it works perfectly. I tested it both from Mac using podman compilation and also natively from an Amazon Linux machine.

See my comment above regarding the SecretsParams file. Why did you create this file? I believe it was by accident but let me know if I missed something. Feel free to delete it otherwise and then we can merge this PR.

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@phipag that's awesome! Can't wait to try it out! Will start working on the next GraalVM module!

Looking forward to this! 🚀