From 17954d95530182a2f99ca70b9d3eeddd824a7c87 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Wed, 7 Jun 2023 18:23:30 +0200 Subject: [PATCH 1/7] docs: add visual representation of release process --- MAINTAINERS.md | 146 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 105 insertions(+), 41 deletions(-) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 114ac52bf1e..88ca0f16cb1 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -13,6 +13,7 @@ - [Triage Bug Reports](#triage-bug-reports) - [Triage RFCs](#triage-rfcs) - [Releasing a new version](#releasing-a-new-version) + - [Release process visualized](#release-process-visualized) - [Drafting release notes](#drafting-release-notes) - [Run end to end tests](#run-end-to-end-tests) - [Releasing a documentation hotfix](#releasing-a-documentation-hotfix) @@ -69,42 +70,42 @@ Previous active maintainers who contributed to this project. These are the most common labels used by maintainers to triage issues, pull requests (PR), and for project management: -| Label | Usage | Notes | -| ---------------------- | ---------------------------------------------------------------- | --------------------------------------------------------------- | -| triage | New issues that require maintainers review | Issue template | -| bug | Unexpected, reproducible and unintended software behavior | PR/Release automation; Doc snippets are excluded; | -| not-a-bug | New and existing bug reports incorrectly submitted as bug | Analytics | -| documentation | Documentation improvements | PR/Release automation; Doc additions, fixes, etc.; | -| feature-request | New or enhancements to existing features | Issue template | -| typing | New or enhancements to static typing | Issue template | -| RFC | Technical design documents related to a feature request | Issue template | -| bug-upstream | Bug caused by upstream dependency | | -| help wanted | Tasks you want help from anyone to move forward | Bandwidth, complex topics, etc. | -| need-customer-feedback | Tasks that need more feedback before proceeding | 80/20% rule, uncertain, etc. | -| need-more-information | Missing information before making any calls | | -| need-documentation | PR is missing or has incomplete documentation | | -| need-issue | PR is missing a related issue for tracking change | PR automation | -| need-rfc | Feature request requires a RFC to improve discussion | | -| pending-release | Merged changes that will be available soon | Release automation auto-closes/notifies it | -| revisit-in-3-months | Blocked issues/PRs that need to be revisited | Often related to `need-customer-feedback`, prioritization, etc. | -| breaking-change | Changes that will cause customer impact and need careful triage | | -| do-not-merge | PRs that are blocked for varying reasons | Timeline is uncertain | -| size/XS | PRs between 0-9 LOC | PR automation | -| size/S | PRs between 10-29 LOC | PR automation | -| size/M | PRs between 30-99 LOC | PR automation | -| size/L | PRs between 100-499 LOC | PR automation | -| size/XL | PRs between 500-999 LOC, often PRs that grown with feedback | PR automation | -| size/XXL | PRs with 1K+ LOC, largely documentation related | PR automation | -| tests | PRs that add or change tests | PR automation | +| Label | Usage | Notes | +| ---------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------- | +| triage | New issues that require maintainers review | Issue template | +| bug | Unexpected, reproducible and unintended software behavior | PR/Release automation; Doc snippets are excluded; | +| not-a-bug | New and existing bug reports incorrectly submitted as bug | Analytics | +| documentation | Documentation improvements | PR/Release automation; Doc additions, fixes, etc.; | +| feature-request | New or enhancements to existing features | Issue template | +| typing | New or enhancements to static typing | Issue template | +| RFC | Technical design documents related to a feature request | Issue template | +| bug-upstream | Bug caused by upstream dependency | | +| help wanted | Tasks you want help from anyone to move forward | Bandwidth, complex topics, etc. | +| need-customer-feedback | Tasks that need more feedback before proceeding | 80/20% rule, uncertain, etc. | +| need-more-information | Missing information before making any calls | | +| need-documentation | PR is missing or has incomplete documentation | | +| need-issue | PR is missing a related issue for tracking change | PR automation | +| need-rfc | Feature request requires a RFC to improve discussion | | +| pending-release | Merged changes that will be available soon | Release automation auto-closes/notifies it | +| revisit-in-3-months | Blocked issues/PRs that need to be revisited | Often related to `need-customer-feedback`, prioritization, etc. | +| breaking-change | Changes that will cause customer impact and need careful triage | | +| do-not-merge | PRs that are blocked for varying reasons | Timeline is uncertain | +| size/XS | PRs between 0-9 LOC | PR automation | +| size/S | PRs between 10-29 LOC | PR automation | +| size/M | PRs between 30-99 LOC | PR automation | +| size/L | PRs between 100-499 LOC | PR automation | +| size/XL | PRs between 500-999 LOC, often PRs that grown with feedback | PR automation | +| size/XXL | PRs with 1K+ LOC, largely documentation related | PR automation | +| tests | PRs that add or change tests | PR automation | | `` | PRs related to a Powertools for AWS Lambda (Python) utility, e.g. `parameters`, `tracer` | PR automation | -| feature | New features or minor changes | PR/Release automation | -| dependencies | Changes that touch dependencies, e.g. Dependabot, etc. | PR/ automation | -| github-actions | Changes in GitHub workflows | PR automation | -| github-templates | Changes in GitHub issue/PR templates | PR automation | -| internal | Changes in governance and chores (linting setup, baseline, etc.) | PR automation | -| tech-debt | Changes in tech debt | | -| customer-reference | Authorization to use company name in our documentation | Public Relations | -| community-content | Suggested content to feature in our documentation | Public Relations | +| feature | New features or minor changes | PR/Release automation | +| dependencies | Changes that touch dependencies, e.g. Dependabot, etc. | PR/ automation | +| github-actions | Changes in GitHub workflows | PR automation | +| github-templates | Changes in GitHub issue/PR templates | PR automation | +| internal | Changes in governance and chores (linting setup, baseline, etc.) | PR automation | +| tech-debt | Changes in tech debt | | +| customer-reference | Authorization to use company name in our documentation | Public Relations | +| community-content | Suggested content to feature in our documentation | Public Relations | ## Maintainer Responsibilities @@ -185,13 +186,76 @@ Some examples using our initial and new RFC templates: #92, #94, #95, #991, #122 Firstly, make sure the commit history in the `develop` branch **(1)** it's up to date, **(2)** commit messages are semantic, and **(3)** commit messages have their respective area, for example `feat(logger): `, `chore(ci): ...`). -**Found typos or unclear commit messages?** +**Looks good, what's next?** -Reword through rebase and push with `--force-with-lease` once you're confident. This will ensure [CHANGELOG](./CHANGELOG.md) is always clear for customers looking to understand what changed in between releases - was that a bug? what new features and for which utility? +Kickoff the `Release` workflow with the intended version - this might take around 25m-30m to complete. -**Looks good, what's next?** +Once complete, you can start drafting the release notes to let customers know **what changed and what's in it for them (a.k.a why they should care)**. We have guidelines in the release notes section so you know what good looks like. + +> **NOTE**: Documentation might take a few minutes to reflect the latest version due to caching and CDN invalidations. + +#### Release process visualized + +Every release makes hundreds of checks, security scans, canaries and deployments - all of these are automated. + +This is a close visual representation of the main steps (GitHub Actions UI should be the source of truth). + +```mermaid +%% use 8s to graph steps faster than 8s like seal/checksum that actually takes 0s +gantt + +title Release process +dateFormat HH:mm +axisFormat %H:%M + +Release commit : milestone, m1, 10:00,2m + +section Seal + Bump release version : crit, 8s + Prevent source tampering : crit, 43s +section QA + Quality checks : crit, 2.2m +section Build + Checksum : crit, 8s + Build release artifact : crit, 39s + Seal : crit, 8s +section Release + Checksum : crit, 8s + PyPi temp credentials : crit, 8s + Publish PyPi : crit, pypi, 10:06, 29s + +PyPi release : milestone, m2, 10:06,1s -Kickoff the `Release` workflow with the intended version. Once complete, update the draft release notes within the `` section summarizing why customers should care about this release. +section Git release + Checksum : crit, after pypi, 8s + Git Tag : crit, 8s + Bump package version : crit, 8s + Create PR : crit, 8s + +section Layer release + Build (x86+ARM) : crit, layer_build, 10:08, 6m + Deploy Beta : crit, layer_beta, after layer_build, 6.3m + Deploy Prod : crit, layer_prod, after layer_beta, 6.3m + +Layer release : milestone, m3, 10:26,1s + +section SAR release + Deploy Beta : crit, sar_beta, after layer_build, 2.2m + Deploy Prod : crit, sar_prod, after sar_beta, 2.2m + +SAR release : milestone, m4, 10:18,1s + +section Docs + Create PR (Layer ARN) : crit, after layer_prod, 8s + Release versioned docs : crit, 2.2m + +Documentation release : milestone, m4, 10:28,1m + +section Post-release + Close pending issues : crit, 8s + +Release complete : milestone, m6, 10:31,2m +``` #### Drafting release notes @@ -205,6 +269,8 @@ You'll notice we group all changes based on their [labels](#labels) like `featur Edit the respective PR title and update their [labels](#labels). Then run the [Release Drafter workflow](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/release-drafter.yml) to update the Draft release. +> **NOTE**: This won't change the CHANGELOG as the merge commit is immutable. Don't worry about it. We'd only rewrite git history only if this can lead to confusion and we'd pair with another maintainer. + **All looking good, what's next?** The best part comes now. Replace the placeholder `[Human readable summary of changes]` with what you'd like to communicate to customers what this release is all about. Rule of thumb: always put yourself in the customers shoes. @@ -221,8 +287,6 @@ Once you're happy, hit `Publish release` 🎉🎉🎉. This will kick off the [Publishing workflow](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/release.yml) and within a few minutes you should see the latest version in PyPi, and all issues labeled as `pending-release` will be closed and notified. -> TODO: Include information to verify SAR and Lambda Layers deployment; we're still finalizing Lambda Layer automated deployment in GitHub Actions - ping @am29d when in doubt. - ### Run end to end tests E2E tests are run on every push to `develop` or manually via [run-e2e-tests workflow](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/run-e2e-tests.yml). From 9f116c5c61d77495553452f503ba559397ee5b8d Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Wed, 7 Jun 2023 18:27:13 +0200 Subject: [PATCH 2/7] chore: test without mermaid comment --- MAINTAINERS.md | 1 - 1 file changed, 1 deletion(-) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 88ca0f16cb1..1c70ec57f0e 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -201,7 +201,6 @@ Every release makes hundreds of checks, security scans, canaries and deployments This is a close visual representation of the main steps (GitHub Actions UI should be the source of truth). ```mermaid -%% use 8s to graph steps faster than 8s like seal/checksum that actually takes 0s gantt title Release process From 7570ea8bf796f9387604ba5687fb9f81ceb91999 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Wed, 7 Jun 2023 18:28:37 +0200 Subject: [PATCH 3/7] chore: test with active color --- MAINTAINERS.md | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 1c70ec57f0e..a2458e7a51f 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -210,48 +210,48 @@ axisFormat %H:%M Release commit : milestone, m1, 10:00,2m section Seal - Bump release version : crit, 8s - Prevent source tampering : crit, 43s + Bump release version : active, 8s + Prevent source tampering : active, 43s section QA - Quality checks : crit, 2.2m + Quality checks : active, 2.2m section Build - Checksum : crit, 8s - Build release artifact : crit, 39s - Seal : crit, 8s + Checksum : active, 8s + Build release artifact : active, 39s + Seal : active, 8s section Release - Checksum : crit, 8s - PyPi temp credentials : crit, 8s - Publish PyPi : crit, pypi, 10:06, 29s + Checksum : active, 8s + PyPi temp credentials : active, 8s + Publish PyPi : active, pypi, 10:06, 29s PyPi release : milestone, m2, 10:06,1s section Git release - Checksum : crit, after pypi, 8s - Git Tag : crit, 8s - Bump package version : crit, 8s - Create PR : crit, 8s + Checksum : active, after pypi, 8s + Git Tag : active, 8s + Bump package version : active, 8s + Create PR : active, 8s section Layer release - Build (x86+ARM) : crit, layer_build, 10:08, 6m - Deploy Beta : crit, layer_beta, after layer_build, 6.3m - Deploy Prod : crit, layer_prod, after layer_beta, 6.3m + Build (x86+ARM) : active, layer_build, 10:08, 6m + Deploy Beta : active, layer_beta, after layer_build, 6.3m + Deploy Prod : active, layer_prod, after layer_beta, 6.3m Layer release : milestone, m3, 10:26,1s section SAR release - Deploy Beta : crit, sar_beta, after layer_build, 2.2m - Deploy Prod : crit, sar_prod, after sar_beta, 2.2m + Deploy Beta : active, sar_beta, after layer_build, 2.2m + Deploy Prod : active, sar_prod, after sar_beta, 2.2m SAR release : milestone, m4, 10:18,1s section Docs - Create PR (Layer ARN) : crit, after layer_prod, 8s - Release versioned docs : crit, 2.2m + Create PR (Layer ARN) : active, after layer_prod, 8s + Release versioned docs : active, 2.2m Documentation release : milestone, m4, 10:28,1m section Post-release - Close pending issues : crit, 8s + Close pending issues : active, 8s Release complete : milestone, m6, 10:31,2m ``` From d4c9a5d0d0b099dab779efce0bc9fd5a644c6fe2 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Wed, 7 Jun 2023 18:30:36 +0200 Subject: [PATCH 4/7] chore: revert test with active color --- MAINTAINERS.md | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index a2458e7a51f..1c70ec57f0e 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -210,48 +210,48 @@ axisFormat %H:%M Release commit : milestone, m1, 10:00,2m section Seal - Bump release version : active, 8s - Prevent source tampering : active, 43s + Bump release version : crit, 8s + Prevent source tampering : crit, 43s section QA - Quality checks : active, 2.2m + Quality checks : crit, 2.2m section Build - Checksum : active, 8s - Build release artifact : active, 39s - Seal : active, 8s + Checksum : crit, 8s + Build release artifact : crit, 39s + Seal : crit, 8s section Release - Checksum : active, 8s - PyPi temp credentials : active, 8s - Publish PyPi : active, pypi, 10:06, 29s + Checksum : crit, 8s + PyPi temp credentials : crit, 8s + Publish PyPi : crit, pypi, 10:06, 29s PyPi release : milestone, m2, 10:06,1s section Git release - Checksum : active, after pypi, 8s - Git Tag : active, 8s - Bump package version : active, 8s - Create PR : active, 8s + Checksum : crit, after pypi, 8s + Git Tag : crit, 8s + Bump package version : crit, 8s + Create PR : crit, 8s section Layer release - Build (x86+ARM) : active, layer_build, 10:08, 6m - Deploy Beta : active, layer_beta, after layer_build, 6.3m - Deploy Prod : active, layer_prod, after layer_beta, 6.3m + Build (x86+ARM) : crit, layer_build, 10:08, 6m + Deploy Beta : crit, layer_beta, after layer_build, 6.3m + Deploy Prod : crit, layer_prod, after layer_beta, 6.3m Layer release : milestone, m3, 10:26,1s section SAR release - Deploy Beta : active, sar_beta, after layer_build, 2.2m - Deploy Prod : active, sar_prod, after sar_beta, 2.2m + Deploy Beta : crit, sar_beta, after layer_build, 2.2m + Deploy Prod : crit, sar_prod, after sar_beta, 2.2m SAR release : milestone, m4, 10:18,1s section Docs - Create PR (Layer ARN) : active, after layer_prod, 8s - Release versioned docs : active, 2.2m + Create PR (Layer ARN) : crit, after layer_prod, 8s + Release versioned docs : crit, 2.2m Documentation release : milestone, m4, 10:28,1m section Post-release - Close pending issues : active, 8s + Close pending issues : crit, 8s Release complete : milestone, m6, 10:31,2m ``` From 8c5d7b23af63fe1821c3b05866bf8d5efc9e2ed9 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Wed, 7 Jun 2023 18:33:14 +0200 Subject: [PATCH 5/7] chore: use active color and move mermaid comment as markdown comment --- MAINTAINERS.md | 44 +++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 1c70ec57f0e..811f4130248 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -200,6 +200,8 @@ Every release makes hundreds of checks, security scans, canaries and deployments This is a close visual representation of the main steps (GitHub Actions UI should be the source of truth). + + ```mermaid gantt @@ -210,48 +212,48 @@ axisFormat %H:%M Release commit : milestone, m1, 10:00,2m section Seal - Bump release version : crit, 8s - Prevent source tampering : crit, 43s + Bump release version : active, 8s + Prevent source tampering : active, 43s section QA - Quality checks : crit, 2.2m + Quality checks : active, 2.2m section Build - Checksum : crit, 8s - Build release artifact : crit, 39s - Seal : crit, 8s + Checksum : active, 8s + Build release artifact : active, 39s + Seal : active, 8s section Release - Checksum : crit, 8s - PyPi temp credentials : crit, 8s - Publish PyPi : crit, pypi, 10:06, 29s + Checksum : active, 8s + PyPi temp credentials : active, 8s + Publish PyPi : active, pypi, 10:06, 29s PyPi release : milestone, m2, 10:06,1s section Git release - Checksum : crit, after pypi, 8s - Git Tag : crit, 8s - Bump package version : crit, 8s - Create PR : crit, 8s + Checksum : active, after pypi, 8s + Git Tag : active, 8s + Bump package version : active, 8s + Create PR : active, 8s section Layer release - Build (x86+ARM) : crit, layer_build, 10:08, 6m - Deploy Beta : crit, layer_beta, after layer_build, 6.3m - Deploy Prod : crit, layer_prod, after layer_beta, 6.3m + Build (x86+ARM) : active, layer_build, 10:08, 6m + Deploy Beta : active, layer_beta, after layer_build, 6.3m + Deploy Prod : active, layer_prod, after layer_beta, 6.3m Layer release : milestone, m3, 10:26,1s section SAR release - Deploy Beta : crit, sar_beta, after layer_build, 2.2m - Deploy Prod : crit, sar_prod, after sar_beta, 2.2m + Deploy Beta : active, sar_beta, after layer_build, 2.2m + Deploy Prod : active, sar_prod, after sar_beta, 2.2m SAR release : milestone, m4, 10:18,1s section Docs - Create PR (Layer ARN) : crit, after layer_prod, 8s - Release versioned docs : crit, 2.2m + Create PR (Layer ARN) : active, after layer_prod, 8s + Release versioned docs : active, 2.2m Documentation release : milestone, m4, 10:28,1m section Post-release - Close pending issues : crit, 8s + Close pending issues : active, 8s Release complete : milestone, m6, 10:31,2m ``` From f6574a2d12358f4f7b68304f7d0b663518a967be Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Wed, 7 Jun 2023 18:56:01 +0200 Subject: [PATCH 6/7] fix: sar dependency on canaries --- MAINTAINERS.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 811f4130248..1fe8a3e4f0c 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -241,10 +241,10 @@ section Layer release Layer release : milestone, m3, 10:26,1s section SAR release - Deploy Beta : active, sar_beta, after layer_build, 2.2m + Deploy Beta : active, sar_beta, after layer_beta, 2.2m Deploy Prod : active, sar_prod, after sar_beta, 2.2m -SAR release : milestone, m4, 10:18,1s +SAR release : milestone, m4, 10:25,1s section Docs Create PR (Layer ARN) : active, after layer_prod, 8s From 9452638af42280fe3dd05914f4cb918c50861417 Mon Sep 17 00:00:00 2001 From: heitorlessa Date: Wed, 7 Jun 2023 19:00:05 +0200 Subject: [PATCH 7/7] chore: sar should depend on layer beta due to canaries out of abundance of cautious --- .github/workflows/publish_v2_layer.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/publish_v2_layer.yml b/.github/workflows/publish_v2_layer.yml index 28f767a3a3f..db677adcee6 100644 --- a/.github/workflows/publish_v2_layer.yml +++ b/.github/workflows/publish_v2_layer.yml @@ -143,7 +143,7 @@ jobs: latest_published_version: ${{ inputs.latest_published_version }} sar-beta: - needs: build-layer + needs: beta # canaries run on Layer Beta env permissions: # lower privilege propagated from parent workflow (release.yml) id-token: write @@ -159,7 +159,7 @@ jobs: package-version: ${{ inputs.latest_published_version }} sar-prod: - needs: [build-layer, beta, sar-beta] + needs: sar-beta permissions: # lower privilege propagated from parent workflow (release.yml) id-token: write