Skip to content

Use versions of Task with Apple Silicon builds in tests #1030

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 2, 2024
Merged

Use versions of Task with Apple Silicon builds in tests #1030

merged 1 commit into from
Aug 2, 2024

Conversation

per1234
Copy link
Contributor

@per1234 per1234 commented Aug 2, 2024

The unit and integration tests install various versions of Task.

The GitHub Actions macos-latest runner was recently changed from using x86 machines to Apple Silicon/ARM machines:

https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

Task is now built for both architectures. However, this was not done for the older versions of Task previously used for the tests. The runner architecture switch caused the tests to start failing spuriously on the macos-latest GitHub Actions workflow run jobs:

https://github.com/arduino/setup-task/actions/runs/10205578962/job/28236648855#step:5:30

FAIL __tests__/main.test.ts
  installer tests
    ✕ Downloads version of Task if no matching version is installed (188 ms)
    Gets the latest release of Task
      ✕ Gets the latest version of Task 2.5 using 2.5 and no matching version is installed (174 ms)
      ✕ Gets latest version of Task using 2.x and no matching version is installed (68 ms)
      ✕ Gets preview version of Task using 3.x and no matching version is installed (167 ms)
      ✕ Skips version computing when a valid semver is provided (164 ms)

  ● installer tests › Downloads version of Task if no matching version is installed

    Failed to download version v2.6.0: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

  ● installer tests › Gets the latest release of Task › Gets the latest version of Task 2.5 using 2.5 and no matching version is installed

    Failed to download version v2.5.2: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

  ● installer tests › Gets the latest release of Task › Gets latest version of Task using 2.x and no matching version is installed

    Failed to download version v2.6.0: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

  ● installer tests › Gets the latest release of Task › Gets preview version of Task using 3.x and no matching version is installed

    Failed to download version v3.0.0-preview1: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

  ● installer tests › Gets the latest release of Task › Skips version computing when a valid semver is provided

    Failed to download version v3.0.0: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

https://github.com/arduino/setup-task/actions/runs/10205578965/job/28236651388#step:3:6

Error: Failed to download version v3.4.1: Error: Unexpected HTTP response: 404

The obvious solution would be to pin the runner to the last x86 runner: macos-13 in the test runner workflows. However, GitHub phases out runners over time so this would not be a future proof solution. The chosen solution is to continue to use the Apple Silicon macos-latest runner and adjust the tests to use versions of Task for which an Apple Silicon build is available.

This was not possible for the integration test of the action's major version pinning capability. The reason is that a previous major version (2.x) must be used in this test in order to allow a consistent assertion, since the installed version would change over time if the latest major version was used. Apple Silicon builds are not available in the 2.x Task version series. For this reason, the integration test runner workflow is configured to skip that test on the macos-latest runner job. The Linux and Windows integration test runner jobs, as well as unit test (which is not constrained in this way due to using artificial tags data) will provide sufficient coverage for this capability.

The unit and integration tests install various versions of Task.

The GitHub Actions macos-latest runner was recently changed from using x86 machines to Apple Silicon/ARM machines.

Task is now built for both architectures. However, this was not done for the older versions of Task previously used for the tests. The runner architecture switch caused the tests to start failing spuriously on the macos-latest GitHub Actions workflow run jobs:

```
FAIL __tests__/main.test.ts
  installer tests
    ✕ Downloads version of Task if no matching version is installed (188 ms)
    Gets the latest release of Task
      ✕ Gets the latest version of Task 2.5 using 2.5 and no matching version is installed (174 ms)
      ✕ Gets latest version of Task using 2.x and no matching version is installed (68 ms)
      ✕ Gets preview version of Task using 3.x and no matching version is installed (167 ms)
      ✕ Skips version computing when a valid semver is provided (164 ms)

  ● installer tests › Downloads version of Task if no matching version is installed

    Failed to download version v2.6.0: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

  ● installer tests › Gets the latest release of Task › Gets the latest version of Task 2.5 using 2.5 and no matching version is installed

    Failed to download version v2.5.2: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

  ● installer tests › Gets the latest release of Task › Gets latest version of Task using 2.x and no matching version is installed

    Failed to download version v2.6.0: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

  ● installer tests › Gets the latest release of Task › Gets preview version of Task using 3.x and no matching version is installed

    Failed to download version v3.0.0-preview1: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)

  ● installer tests › Gets the latest release of Task › Skips version computing when a valid semver is provided

    Failed to download version v3.0.0: Error: Unexpected HTTP response: 404

      165 |       core.debug(error.toString());
      166 |     }
    > 167 |     throw new Error(`Failed to download version ${version}: ${error}`);
          |           ^
      168 |   }
      169 |
      170 |   // Extract

      at src/installer.ts:167:11
          at Generator.throw (<anonymous>)
      at rejected (src/installer.ts:40:65)
```

```
Error: Failed to download version v3.4.1: Error: Unexpected HTTP response: 404
```

The obvious solution would be to pin the runner to the last x86 runner: macos-13 in the test runner workflows. However, GitHub phases out runners over time so this would not be a future proof solution. The chosen solution is to continue to use the Apple Silicon macos-latest runner and adjust the tests to use versions of Task for which an Apple Silicon build is available.

This was not possible for the integration test of the action's major version pinning capability. The reason is that a previous major version (2.x) must be used in this test in order to allow a consistent assertion, since the installed version would change over time if the latest major version was used. Apple Silicon builds are not available in the 2.x Task version series. For this reason, the integration test runner workflow is configured to skip that test on the macos-latest runner job. The Linux and Windows integration test runner jobs, as well as unit test (which is not constrained in this way due to using artificial tags data) will provide sufficient coverage for this capability.
@per1234 per1234 added type: imperfection Perceived defect in any part of project topic: infrastructure Related to project infrastructure os: macos Specific to macOS operating system labels Aug 2, 2024
@per1234 per1234 self-assigned this Aug 2, 2024
@per1234 per1234 merged commit 258789e into arduino:main Aug 2, 2024
18 checks passed
@per1234 per1234 deleted the fix-macos branch August 2, 2024 04:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
os: macos Specific to macOS operating system topic: infrastructure Related to project infrastructure type: imperfection Perceived defect in any part of project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant