Transit CI to GitHub Actions #1942

weihanglo · 2019-12-07T16:46:47Z

What

Transit the CI process to GitHub Actions.

Close #1903

Live GitHub Actions of this PR

Details

Create two independent jobs, Frontend and Backend, to run in parallel.
The pre-installed Rust in the environment of Github Actions is installed under root privileges. That makes tarball cache fails to restore. Instead, we install Rust via rustup script manually.
GitHub Actions has not yet support allow_failures option, so we set fail-fast to false to prevent failure on nightly to cancel the entire CI process.
Update all documents mentioned Travis-CI to GitHub Actions.

Need helps

I have no experience of handling Percy token. Need some help to check the correctness.
Not sure how to configure GitHub with bors correctly. I followed this and set the status to job names.

rust-highfive · 2019-12-07T16:46:50Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @smarnach (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

carols10cents

GitHub Actions has not yet support allow_failures option, so we set fail-fast to false to prevent failure on nightly to cancel the entire CI process.

That's unfortunate that they don't support allowing failures. Your handling of this sounds right!

I have no experience of handling Percy token. Need some help to check the correctness.

It looks like Percy has some documentation about setting up GitHub Actions? Does that help?

* Not sure how to configure GitHub with bors correctly. I followed [this](https://github.com/rust-osdev/x86_64/blob/master/bors.toml) [](/rust-osdev/x86_64/blob/HEAD@{2019-12-07T16:46:47Z}/bors.toml)

We're actually using the main rust-lang org's bors instance now, sorry about the confusion because of out-of-date documentation in this repo! I think when we're ready to switch to github actions, we'll need to make a change similar to this one in rust-central-station.

.github/workflows/ci.yml

.travis.yml

docs/ARCHITECTURE.md

.github/workflows/ci.yml

carols10cents · 2019-12-09T16:31:49Z

@jtgeibel I know you've worked on the CI config lately, so I'd love for you to review this PR too!

- Add `-D warnings` to `RUSTFLAGS`. Ref: f896a8e - Remove unnecessary `--force` flag of diesel installation

weihanglo · 2019-12-09T18:43:51Z

Percy builds in GitHub Actions has already integrated with the token. Seems no changes need. And this build is from the last commit b188188.

However it failed here and I cannot figure out why. It should be exact the same build from travis-ci. I've also test upgrade Chrome in CI environment but the build still got an error and differed

carols10cents · 2019-12-11T18:47:46Z

Hm what is the binary file named 3a8c2f23 that was added?

carols10cents · 2019-12-11T21:50:46Z

Percy builds in GitHub Actions has already integrated with the token. Seems no changes need. And this build is from the last commit b188188.

However it failed here and I cannot figure out why. It should be exact the same build from travis-ci. I've also test upgrade Chrome in CI environment but the build still got an error and differed

Strange! In the GitHub Actions logs you linked to where the percy API call failed, if I scroll back up a bit I see:

[percy] Finalizing build...
[percy] Visual diffs are now processing: https://percy.io/crates-io/crates.io/builds/3415214

When the frontend tests run in Travis, I also see logs like that in the middle of the test run, but I also see them again near the end, so twice total, which also happens to be the number of parallel splits we're running the tests with.

Percy's documentation mentions that the percy client has automatic support for parallel test runners on Travis CI.

To test this theory, could you try removing the --split=2 --parallel from the test command and see if GitHub actions is able to send all the screenshots to percy?

If that works, another thing to try would be to set the percy environment variables for the test job-- like PERCY_PARALLEL_NONCE=date +%s PERCY_PARALLEL_TOTAL=2 ember exam --split=2 --parallel or similar.

Could you please try those two things? If neither work, I'll do some more investigating. 😅 Let me know!

jtgeibel

Overall, this looks really good to me. As you mentioned, my biggest concern is that we may run into issues with the size of our caches.

In order to trigger a build, I've taken this PR and pushed it to a branch (after adding the branch name to the list). The build should now be visible on the Actions tab. I also forced a rebuild, so the logs should show successful cache hits.

.github/workflows/ci.yml

jtgeibel · 2019-12-12T00:48:08Z

.github/workflows/ci.yml

+        uses: actions/cache@v1
+        with:
+          path: ~/.rustup
+          key: ${{ runner.os }}-rustup-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock') }}


I'm not sure if it is necessary to include the -${{ hashFiles('**/Cargo.lock') }} portion for this directory and for the ~/.cargo/ paths below. (It definitely makes sense for the target/ directory.) My reasoning is that the contents of these directories shouldn't really be affected by the contents of the lock file.

By the same reasoning, -${{ matrix.rust }} may also not be necessary for many of these.

If you're unsure about removing it though, I definitely don't see this as a blocker. This is probably a micro-optimization given the expected size of these directories.

Thanks. We need figure out when to invalidate old cache. Maybe some trick like script//ci/cargo-clean-on-new-rustc-version.sh, but I am not sure how to implement that.

jtgeibel · 2019-12-12T00:58:38Z

For some reason, the beta branch (and only beta) has failed twice on the same test now.

 thread 'krate::new_krate' panicked at 'Could not run jobs: NoMessageReceived', src/libcore/result.rs:1165:5

The test isn't failing on Travis, so that's a bit strange. 😕

bors · 2019-12-12T09:41:08Z

☔ The latest upstream changes (presumably #1944) made this pull request unmergeable. Please resolve the merge conflicts.

pietroalbini

There is a binary file named 3a8c2f23.

Also, this PR is not enough to be able to gate bors on GitHub Actions: an unfortunate part of GHA is it doesn't provide a green or red check when the whole workflow passes or fails, but it just reports the individual checks. That means there is no single value we can put in bors's configuration file.

On Crater and (soon) on rustc the approach we've taken is to add a "success" and "failure" jobs with the same display name: both of them have all the other jobs as needs:, and one is if: success() while the other is if: !success(). That allows to gate on the bors build finished check.

https://github.com/rust-lang/crater/blob/a1930d2acd71382b66ce68d9e45ce57f350fae1a/.github/workflows/bors.yml#L125-L149

weihanglo · 2019-12-13T01:31:05Z

To test this theory, could you try removing the --split=2 --parallel from the test command and see if GitHub actions is able to send all the screenshots to percy?

If that works, another thing to try would be to set the percy environment variables for the test job-- like PERCY_PARALLEL_NONCE=date +%s PERCY_PARALLEL_TOTAL=2 ember exam --split=2 --parallel or similar.

Could you please try those two things? If neither work, I'll do some more investigating. 😅 Let me know!

@carols10cents

I've tried but still failed on parallel one.

Without any parallel tests, it passed breezily. Percy build
With a PERCY_PARALLEL_NONCE, it seems that CI uploaded correct snapshots but still failed to fetch the image on /crates/:crate. Percy build

Maybe there are network issues on GitHub Actions?

Here are environment variables:

Run npm test
  npm test
  shell: /bin/bash -e {0}
  env:
    JOBS: 1
    PERCY_PARALLEL_TOTAL: 2
    PERCY_TOKEN: 0d8707a02b19aebbec79bb0bf302b8d2fa95edb33169cfe41b084289596670b1
    PERCY_PROJECT: crates-io/crates.io
    PERCY_PARALLEL_NONCE: 1576200325

> cargo@0.0.0 test /home/runner/work/crates.io/crates.io
> ember exam --split=2 --parallel

weihanglo · 2019-12-13T01:52:22Z

@pietroalbini

There is a binary file named 3a8c2f23.

Removed. Thanks!

On Crater and (soon) on rustc the approach we've taken is to add a "success" and "failure" jobs with the same display name: both of them have all the other jobs as needs:, and one is if: success() while the other is if: !success(). That allows to gate on the bors build finished check.

https://github.com/rust-lang/crater/blob/a1930d2acd71382b66ce68d9e45ce57f350fae1a/.github/workflows/bors.yml#L125-L149

Current Github Actions does not support allow_failures config on job/matrix. Though we can let it fail silently but it is not the right way to handle it in my opinions.

pietroalbini

Current Github Actions does not support allow_failures config on job/matrix. Though we can let it fail silently but it is not the right way to handle it in my opinions.

Well, we still need to have a single job that needs: everything we gate on, otherwise we won't be able to integrate bors with GitHub Actions.

.github/workflows/ci.yml

carols10cents · 2019-12-13T14:24:38Z

* With a `PERCY_PARALLEL_NONCE`, it seems that CI uploaded correct snapshots but still failed to fetch the image on `/crates/:crate`. [Percy build](https://percy.io/crates-io/crates.io/builds/3462405)

Yeah, I'm seeing the download graph not load in Percy on a different PR, so I think that's unrelated to GitHub Actions. I'm going to investigate that a bit today.

Reference: https://github.com/rust-lang/crater/blob/a1930d2acd71382b66ce68d9e45ce57f350fae1a/.github/workflows/bors.yml#L125-L149

pietroalbini

Looks great to me! Let's see what the other team members say.

carols10cents

I'm good with this! I want to make sure @jtgeibel approves too, and I still need to take care of the crate download graph not loading in Percy... I'm trying to fix that in #1948 but I haven't gotten it working yet.

jtgeibel

LGTM!

jtgeibel · 2019-12-21T05:32:30Z

.github/workflows/ci.yml

+  push:
+    branches:
+      - auto
+      - try


On travis we need to include the master branch, because it's (well technically the repo HEAD) cache is used as the fallback for new branches and PRs. That doesn't seem to apply to the caching logic on GH Actions, so I think its okay to drop it here.

jtgeibel · 2019-12-21T05:44:54Z

@bors r+

bors · 2019-12-21T05:44:55Z

📌 Commit d201c50 has been approved by jtgeibel

bors · 2019-12-21T07:00:12Z

⌛ Testing commit d201c50 with merge 7e562c6...

Transit CI to GitHub Actions # What Transit the CI process to GitHub Actions. Close #1903 [Live GitHub Actions of this PR](https://github.com/weihanglo/crates.io/actions?query=workflow%3ACI+) # Details - Create two independent jobs, Frontend and Backend, to run in parallel. - The pre-installed Rust in the environment of Github Actions is installed under root privileges. That makes tarball cache fails to restore. Instead, we install Rust via rustup script manually. - GitHub Actions has not yet support `allow_failures` option, so we set` fail-fast` to false to prevent failure on nightly to cancel the entire CI process. - Update all documents mentioned Travis-CI to GitHub Actions. # Need helps - I have no experience of handling Percy token. Need some help to check the correctness. - Not sure how to configure GitHub with bors correctly. I followed [this](https://github.com/rust-osdev/x86_64/blob/master/bors.toml) and set the status to job names.

bors · 2019-12-21T07:06:21Z

☀️ Test successful - checks-travis
Approved by: jtgeibel
Pushing 7e562c6 to master...

weihanglo added 3 commits December 7, 2019 23:14

chore(prune-cache): exit when not build artifacts found

19bf60b

chore: github actions for frontend CI

0c74aeb

chore: github actions for backend CI

77c6a6a

rust-highfive assigned smarnach Dec 7, 2019

rust-highfive added the S-waiting-on-review label Dec 7, 2019

weihanglo mentioned this pull request Dec 8, 2019

Try out GitHub Actions #1903

Closed

carols10cents requested changes Dec 9, 2019

View reviewed changes

carols10cents assigned carols10cents and unassigned smarnach Dec 9, 2019

carols10cents requested a review from jtgeibel December 9, 2019 16:31

carols10cents added S-waiting-on-author and removed S-waiting-on-review labels Dec 9, 2019

weihanglo added 2 commits December 10, 2019 01:12

docs: travis -> github actions

76c0f39

chore: update RUSTFLAGS and remove unused flag

b188188

- Add `-D warnings` to `RUSTFLAGS`. Ref: f896a8e - Remove unnecessary `--force` flag of diesel installation

weihanglo requested a review from carols10cents December 10, 2019 04:32

jtgeibel mentioned this pull request Dec 12, 2019

Crates.io meeting agenda 2019-12-13 15:30 UTC (Discord, 30 min) rust-lang/crates-io-cargo-teams#65

Closed

jtgeibel requested changes Dec 12, 2019

View reviewed changes

pietroalbini requested changes Dec 12, 2019

View reviewed changes

weihanglo added 4 commits December 12, 2019 22:10

Merge branch 'master' into chore/github-actions

c8b89a6

remove accidentally added file

c2b00d2

chore: add try/auto branches to ci whitelist

c35075c

chore: add nonoe to identify pararllel percy tests

710ad1e

weihanglo requested review from pietroalbini and jtgeibel December 13, 2019 01:52

pietroalbini requested changes Dec 13, 2019

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

weihanglo added 2 commits December 14, 2019 18:23

chore(github-actions): remove master branch for bors to merge PRs

fb0c522

chore(github-actions): add bors workaround flag status

1b695ed

Reference: https://github.com/rust-lang/crater/blob/a1930d2acd71382b66ce68d9e45ce57f350fae1a/.github/workflows/bors.yml#L125-L149

weihanglo requested a review from pietroalbini December 14, 2019 10:27

chore(github-actions): remove all path-ignores

d201c50

pietroalbini approved these changes Dec 16, 2019

View reviewed changes

carols10cents approved these changes Dec 18, 2019

View reviewed changes

jtgeibel approved these changes Dec 21, 2019

View reviewed changes

bors merged commit d201c50 into rust-lang:master Dec 21, 2019

jtgeibel mentioned this pull request Dec 21, 2019

GitHub Actions displays frontend lints to every PR #2042

Closed

weihanglo deleted the chore/github-actions branch January 3, 2020 05:20

Transit CI to GitHub Actions #1942

Transit CI to GitHub Actions #1942

Uh oh!

Conversation

weihanglo commented Dec 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Details

Need helps

Uh oh!

rust-highfive commented Dec 7, 2019

Uh oh!

carols10cents left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carols10cents commented Dec 9, 2019

Uh oh!

weihanglo commented Dec 9, 2019

Uh oh!

carols10cents commented Dec 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

carols10cents commented Dec 11, 2019

Uh oh!

jtgeibel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jtgeibel Dec 12, 2019

Choose a reason for hiding this comment

Uh oh!

weihanglo Dec 13, 2019

Choose a reason for hiding this comment

Uh oh!

jtgeibel commented Dec 12, 2019

Uh oh!

bors commented Dec 12, 2019

Uh oh!

pietroalbini left a comment

Choose a reason for hiding this comment

Uh oh!

weihanglo commented Dec 13, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weihanglo commented Dec 13, 2019

Uh oh!

pietroalbini left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

carols10cents commented Dec 13, 2019

Uh oh!

pietroalbini left a comment

Choose a reason for hiding this comment

Uh oh!

carols10cents left a comment

Choose a reason for hiding this comment

Uh oh!

jtgeibel left a comment

Choose a reason for hiding this comment

Uh oh!

jtgeibel Dec 21, 2019

Choose a reason for hiding this comment

Uh oh!

jtgeibel commented Dec 21, 2019

Uh oh!

bors commented Dec 21, 2019

Uh oh!

bors commented Dec 21, 2019

Uh oh!

bors commented Dec 21, 2019

Uh oh!

Uh oh!

weihanglo commented Dec 7, 2019 •

edited

Loading

carols10cents commented Dec 11, 2019 •

edited

Loading

weihanglo commented Dec 13, 2019 •

edited

Loading