Skip to content

[skip-changelog] Run integration tests concurrently #1953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

MatteoPologruto
Copy link
Contributor

@MatteoPologruto MatteoPologruto commented Oct 27, 2022

Please check if the PR fulfills these requirements

See how to contribute

  • The PR has no duplicates (please search among the Pull Requests
    before creating one)
  • The PR follows
    our contributing guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • UPGRADING.md has been updated with a migration guide (for breaking changes)

What kind of change does this PR introduce?

Infrastructure enhancement

What is the current behavior?

Integration tests are run consecutively and each test has to wait for the previous ones to end before starting, making the whole process slower any time a new test is added.
Unit tests and integration tests are executed by different jobs of the workflow.

What is the new behavior?

Using a matrix to run each package test concurrently reduces the waiting time. GO_TEST_PACKAGE is set to the test package name to identify the single package to run.

Does this PR introduce a breaking change, and is titled accordingly?

No

@MatteoPologruto MatteoPologruto added type: enhancement Proposed improvement topic: infrastructure Related to project infrastructure labels Oct 27, 2022
@MatteoPologruto MatteoPologruto self-assigned this Oct 27, 2022
@umbynos umbynos requested a review from per1234 October 27, 2022 13:52
@MatteoPologruto MatteoPologruto force-pushed the split-integrationtest branch 2 times, most recently from fee0b08 to b995d75 Compare November 2, 2022 15:54
@MatteoPologruto
Copy link
Contributor Author

MatteoPologruto commented Nov 2, 2022

I made some changes to avoid specifying a different go path. It is no longer needed to build the cli in a specific step of the workflow.
This line is used to have a single entry for each package, regardless of the number of files inside the directory. Merging all the files that share a package into one might be no longer necessary.

@MatteoPologruto MatteoPologruto force-pushed the split-integrationtest branch 2 times, most recently from c4d3230 to e178905 Compare November 3, 2022 16:19
Copy link
Member

@cmaglie cmaglie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of nitpicks but I think we are there. 👍🏼

This enhancement reduces the amount of time needed to execute integration tests, since each test package no longer has to wait for the previous ones to be completed. In order to do this, a regex is used
to match each test file. It is also specified the path to the directory that contains the file.
Having different test files that shared the same package was inefficient, because the whole package test was run one time for each file. This enhancement avoids repeating the same tests more than once.
@MatteoPologruto MatteoPologruto merged commit e53d2f5 into arduino:master Nov 4, 2022
@MatteoPologruto MatteoPologruto deleted the split-integrationtest branch November 4, 2022 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: infrastructure Related to project infrastructure type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants