Skip to content

docs(maintainers): visual representation of release process #2399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 7, 2023
Merged

docs(maintainers): visual representation of release process #2399

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
17954d9
docs: add visual representation of release process
heitorlessa Jun 7, 2023
9f116c5
chore: test without mermaid comment
heitorlessa Jun 7, 2023
7570ea8
chore: test with active color
heitorlessa Jun 7, 2023
d4c9a5d
chore: revert test with active color
heitorlessa Jun 7, 2023
8c5d7b2
chore: use active color and move mermaid comment as markdown comment
heitorlessa Jun 7, 2023
f6574a2
fix: sar dependency on canaries
heitorlessa Jun 7, 2023
7a0c14a
Merge branch 'develop' into docs/release-process
heitorlessa Jun 7, 2023
9452638
chore: sar should depend on layer beta due to canaries out of abundan…
heitorlessa Jun 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/publish_v2_layer.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ jobs:
latest_published_version: ${{ inputs.latest_published_version }}

sar-beta:
needs: build-layer
needs: beta # canaries run on Layer Beta env
permissions:
# lower privilege propagated from parent workflow (release.yml)
id-token: write
Expand All @@ -159,7 +159,7 @@ jobs:
package-version: ${{ inputs.latest_published_version }}

sar-prod:
needs: [build-layer, beta, sar-beta]
needs: sar-beta
permissions:
# lower privilege propagated from parent workflow (release.yml)
id-token: write
Expand Down
147 changes: 106 additions & 41 deletions MAINTAINERS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
- [Triage Bug Reports](#triage-bug-reports)
- [Triage RFCs](#triage-rfcs)
- [Releasing a new version](#releasing-a-new-version)
- [Release process visualized](#release-process-visualized)
- [Drafting release notes](#drafting-release-notes)
- [Run end to end tests](#run-end-to-end-tests)
- [Releasing a documentation hotfix](#releasing-a-documentation-hotfix)
Expand Down Expand Up @@ -69,42 +70,42 @@ Previous active maintainers who contributed to this project.

These are the most common labels used by maintainers to triage issues, pull requests (PR), and for project management:

| Label | Usage | Notes |
| ---------------------- | ---------------------------------------------------------------- | --------------------------------------------------------------- |
| triage | New issues that require maintainers review | Issue template |
| bug | Unexpected, reproducible and unintended software behavior | PR/Release automation; Doc snippets are excluded; |
| not-a-bug | New and existing bug reports incorrectly submitted as bug | Analytics |
| documentation | Documentation improvements | PR/Release automation; Doc additions, fixes, etc.; |
| feature-request | New or enhancements to existing features | Issue template |
| typing | New or enhancements to static typing | Issue template |
| RFC | Technical design documents related to a feature request | Issue template |
| bug-upstream | Bug caused by upstream dependency | |
| help wanted | Tasks you want help from anyone to move forward | Bandwidth, complex topics, etc. |
| need-customer-feedback | Tasks that need more feedback before proceeding | 80/20% rule, uncertain, etc. |
| need-more-information | Missing information before making any calls | |
| need-documentation | PR is missing or has incomplete documentation | |
| need-issue | PR is missing a related issue for tracking change | PR automation |
| need-rfc | Feature request requires a RFC to improve discussion | |
| pending-release | Merged changes that will be available soon | Release automation auto-closes/notifies it |
| revisit-in-3-months | Blocked issues/PRs that need to be revisited | Often related to `need-customer-feedback`, prioritization, etc. |
| breaking-change | Changes that will cause customer impact and need careful triage | |
| do-not-merge | PRs that are blocked for varying reasons | Timeline is uncertain |
| size/XS | PRs between 0-9 LOC | PR automation |
| size/S | PRs between 10-29 LOC | PR automation |
| size/M | PRs between 30-99 LOC | PR automation |
| size/L | PRs between 100-499 LOC | PR automation |
| size/XL | PRs between 500-999 LOC, often PRs that grown with feedback | PR automation |
| size/XXL | PRs with 1K+ LOC, largely documentation related | PR automation |
| tests | PRs that add or change tests | PR automation |
| Label | Usage | Notes |
| ---------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
| triage | New issues that require maintainers review | Issue template |
| bug | Unexpected, reproducible and unintended software behavior | PR/Release automation; Doc snippets are excluded; |
| not-a-bug | New and existing bug reports incorrectly submitted as bug | Analytics |
| documentation | Documentation improvements | PR/Release automation; Doc additions, fixes, etc.; |
| feature-request | New or enhancements to existing features | Issue template |
| typing | New or enhancements to static typing | Issue template |
| RFC | Technical design documents related to a feature request | Issue template |
| bug-upstream | Bug caused by upstream dependency | |
| help wanted | Tasks you want help from anyone to move forward | Bandwidth, complex topics, etc. |
| need-customer-feedback | Tasks that need more feedback before proceeding | 80/20% rule, uncertain, etc. |
| need-more-information | Missing information before making any calls | |
| need-documentation | PR is missing or has incomplete documentation | |
| need-issue | PR is missing a related issue for tracking change | PR automation |
| need-rfc | Feature request requires a RFC to improve discussion | |
| pending-release | Merged changes that will be available soon | Release automation auto-closes/notifies it |
| revisit-in-3-months | Blocked issues/PRs that need to be revisited | Often related to `need-customer-feedback`, prioritization, etc. |
| breaking-change | Changes that will cause customer impact and need careful triage | |
| do-not-merge | PRs that are blocked for varying reasons | Timeline is uncertain |
| size/XS | PRs between 0-9 LOC | PR automation |
| size/S | PRs between 10-29 LOC | PR automation |
| size/M | PRs between 30-99 LOC | PR automation |
| size/L | PRs between 100-499 LOC | PR automation |
| size/XL | PRs between 500-999 LOC, often PRs that grown with feedback | PR automation |
| size/XXL | PRs with 1K+ LOC, largely documentation related | PR automation |
| tests | PRs that add or change tests | PR automation |
| `<utility>` | PRs related to a Powertools for AWS Lambda (Python) utility, e.g. `parameters`, `tracer` | PR automation |
| feature | New features or minor changes | PR/Release automation |
| dependencies | Changes that touch dependencies, e.g. Dependabot, etc. | PR/ automation |
| github-actions | Changes in GitHub workflows | PR automation |
| github-templates | Changes in GitHub issue/PR templates | PR automation |
| internal | Changes in governance and chores (linting setup, baseline, etc.) | PR automation |
| tech-debt | Changes in tech debt | |
| customer-reference | Authorization to use company name in our documentation | Public Relations |
| community-content | Suggested content to feature in our documentation | Public Relations |
| feature | New features or minor changes | PR/Release automation |
| dependencies | Changes that touch dependencies, e.g. Dependabot, etc. | PR/ automation |
| github-actions | Changes in GitHub workflows | PR automation |
| github-templates | Changes in GitHub issue/PR templates | PR automation |
| internal | Changes in governance and chores (linting setup, baseline, etc.) | PR automation |
| tech-debt | Changes in tech debt | |
| customer-reference | Authorization to use company name in our documentation | Public Relations |
| community-content | Suggested content to feature in our documentation | Public Relations |

## Maintainer Responsibilities

Expand Down Expand Up @@ -185,13 +186,77 @@ Some examples using our initial and new RFC templates: #92, #94, #95, #991, #122

Firstly, make sure the commit history in the `develop` branch **(1)** it's up to date, **(2)** commit messages are semantic, and **(3)** commit messages have their respective area, for example `feat(logger): <change>`, `chore(ci): ...`).

**Found typos or unclear commit messages?**
**Looks good, what's next?**

Reword through rebase and push with `--force-with-lease` once you're confident. This will ensure [CHANGELOG](./CHANGELOG.md) is always clear for customers looking to understand what changed in between releases - was that a bug? what new features and for which utility?
Kickoff the `Release` workflow with the intended version - this might take around 25m-30m to complete.

**Looks good, what's next?**
Once complete, you can start drafting the release notes to let customers know **what changed and what's in it for them (a.k.a why they should care)**. We have guidelines in the release notes section so you know what good looks like.

> **NOTE**: Documentation might take a few minutes to reflect the latest version due to caching and CDN invalidations.

#### Release process visualized

Every release makes hundreds of checks, security scans, canaries and deployments - all of these are automated.

This is a close visual representation of the main steps (GitHub Actions UI should be the source of truth).

<!-- use 8s to graph steps faster than 8s like seal/checksum that actually takes 0s -->

```mermaid
gantt

title Release process
dateFormat HH:mm
axisFormat %H:%M

Release commit : milestone, m1, 10:00,2m

section Seal
Bump release version : active, 8s
Prevent source tampering : active, 43s
section QA
Quality checks : active, 2.2m
section Build
Checksum : active, 8s
Build release artifact : active, 39s
Seal : active, 8s
section Release
Checksum : active, 8s
PyPi temp credentials : active, 8s
Publish PyPi : active, pypi, 10:06, 29s

Kickoff the `Release` workflow with the intended version. Once complete, update the draft release notes within the `<human readable>` section summarizing why customers should care about this release.
PyPi release : milestone, m2, 10:06,1s

section Git release
Checksum : active, after pypi, 8s
Git Tag : active, 8s
Bump package version : active, 8s
Create PR : active, 8s

section Layer release
Build (x86+ARM) : active, layer_build, 10:08, 6m
Deploy Beta : active, layer_beta, after layer_build, 6.3m
Deploy Prod : active, layer_prod, after layer_beta, 6.3m

Layer release : milestone, m3, 10:26,1s

section SAR release
Deploy Beta : active, sar_beta, after layer_beta, 2.2m
Deploy Prod : active, sar_prod, after sar_beta, 2.2m

SAR release : milestone, m4, 10:25,1s

section Docs
Create PR (Layer ARN) : active, after layer_prod, 8s
Release versioned docs : active, 2.2m

Documentation release : milestone, m4, 10:28,1m

section Post-release
Close pending issues : active, 8s

Release complete : milestone, m6, 10:31,2m
```

#### Drafting release notes

Expand All @@ -205,6 +270,8 @@ You'll notice we group all changes based on their [labels](#labels) like `featur

Edit the respective PR title and update their [labels](#labels). Then run the [Release Drafter workflow](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/release-drafter.yml) to update the Draft release.

> **NOTE**: This won't change the CHANGELOG as the merge commit is immutable. Don't worry about it. We'd only rewrite git history only if this can lead to confusion and we'd pair with another maintainer.

**All looking good, what's next?**

The best part comes now. Replace the placeholder `[Human readable summary of changes]` with what you'd like to communicate to customers what this release is all about. Rule of thumb: always put yourself in the customers shoes.
Expand All @@ -221,8 +288,6 @@ Once you're happy, hit `Publish release` 🎉🎉🎉.

This will kick off the [Publishing workflow](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/release.yml) and within a few minutes you should see the latest version in PyPi, and all issues labeled as `pending-release` will be closed and notified.

> TODO: Include information to verify SAR and Lambda Layers deployment; we're still finalizing Lambda Layer automated deployment in GitHub Actions - ping @am29d when in doubt.

### Run end to end tests

E2E tests are run on every push to `develop` or manually via [run-e2e-tests workflow](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/run-e2e-tests.yml).
Expand Down