Maintenance: run e2e tests in on bigger runner

[dreamorosi]

Summary

Our e2e test suite is growing and is expected to grow even more as we add multiple features and utilities.

Currently we have ~35 parallel jobs that run in a chunks of 30 at the time. This means we're not effectively parallelizing as much as we could.

Additionally, we're starting to experience transient failures due to timeouts from CloudFormation due to the high number of operations resulting from dozens of stacks being deployed at once, since each job can have more than one stack in it.

Before investing in any specific optimization of our test suite, a potentially low hanging fruit would be to expand the test suite to run into multiple AWS regions rather than on a single one.

Why is this needed?

This way, the operations would be split into more regions thus lowering the chance of transient issues derived from timeouts.

Which area does this relate to?

Automation

Solution

Pick a region randomly between a set of available target regions and run the test for a job there.

Acknowledgment

• This request meets Powertools for AWS Lambda (TypeScript) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[dreamorosi]

After some investigation and help from @leandrodamascena we concluded that the timeouts are likely being caused by network/CPU congestion of the shared GitHub hosted runners we are using across the repo.

I have been doing tests with a dedicated hosted runner and the timeouts seem to be gone, as well as having a slight performance improvement in terms of runtime.

I'm doing more tests, and I am also likely dropping the idea of splitting tests across regions.

[github-actions]

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.