Skip to content

ci(e2e-report): support generating report site for version other than latest #2540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 12, 2024
Merged

ci(e2e-report): support generating report site for version other than latest #2540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1aacb81
ci(e2e-report): support version other than latest
serhalp Jul 9, 2024
462ef8c
Merge branch 'main' into ci/read-correct-e2e-results-file
pieh Jul 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions .github/workflows/e2e-report.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ on:
use-branch:
description: 'Enable if you want to test data from your selected branch instead of the scheduled test runs from Main'
type: boolean
version:
description: 'Version of Next.js (most recent test run must have included this version)'
type: choice
options:
- 'latest'
- 'canary'
- '13.5.1'
default: 'latest'

env:
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
Expand All @@ -27,12 +35,19 @@ jobs:
E2E_RUN_ID=$(gh run list -w test-e2e.yml -e schedule -s success --json databaseId --jq ".[0].databaseId" --repo $GITHUB_REPOSITORY)
fi
echo "runId=$E2E_RUN_ID" >> $GITHUB_OUTPUT
- name: Download latest e2e results
- name: Download e2e results
if: ${{ steps.get-run-id.outputs.runId }}
run: |
echo "Downloading latest test results from run https://github.com/netlify/next-runtime/actions/runs/${{ steps.get-run-id.outputs.runId }}"
rm e2e-report/data/test-results.json
gh run download ${{ steps.get-run-id.outputs.runId }} -n "latest-test-results.json" -D e2e-report/data/ --repo $GITHUB_REPOSITORY
version="${{ inputs.version }}"
version=${version:-latest}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a second layer of defaulting here because the scheduled run won't have the workflow_dispatch default populated

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, but we can have it manual only just so we can do first release, and then fix this.
I don't think we're ready to automatically deploy on a schedule yet, in terms of test stability. Would be great to have a deploy preview automatically though (so then it's much easier to evaluate before manual release)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify for anyone reading this: this job already runs on a schedule and only ever creates deploy previews. We publish deploys to production manually in the Netlify UI.

OUTPUT_DIR="e2e-report/data"
OUTPUT_FILENAME="test-results.json"
echo "Downloading ${version} test results from run https://github.com/netlify/next-runtime/actions/runs/${{ steps.get-run-id.outputs.runId }}"
rm "${OUTPUT_DIR}/${OUTPUT_FILENAME}"
artifact_name="${version}-test-results.json"
# NOTE: The artifact name is not necessarily the artifact *file* name. The file name here
# must be `test-results.json`, but this is defined at the artifact upload step.
gh run download ${{ steps.get-run-id.outputs.runId }} -n "${artifact_name}" -D "${OUTPUT_DIR}" --repo $GITHUB_REPOSITORY
- name: Install site dependencies
if: success()
run: |
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/test-e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,9 @@ jobs:
- name: Upload Test JSON
uses: actions/upload-artifact@v4
with:
# TODO(serhalp) Consider renaming this. It's misleading, since it's just an identifier,
# but it's formatted like a filename, and happens to be almost - but not quite - the
# actual filename.
name: ${{matrix.version_spec.selector}}-test-results.json
path: report/test-results.json

Expand Down
Loading