Skip to content

Commit d30a412

Browse files
committed
docs(contributing): add toc and definition for tests
1 parent 6648366 commit d30a412

File tree

1 file changed

+53
-18
lines changed

1 file changed

+53
-18
lines changed

CONTRIBUTING.md

Lines changed: 53 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,20 @@
1-
[![Gitpod Ready-to-Code](https://img.shields.io/badge/Gitpod-Ready--to--Code-blue?logo=gitpod)](https://gitpod.io/from-referrer/)
1+
<!-- markdownlint-disable MD043 MD041 -->
2+
# Table of contents <!-- omit in toc -->
3+
4+
- [Contributing Guidelines](#contributing-guidelines)
5+
- [Reporting Bugs/Feature Requests](#reporting-bugsfeature-requests)
6+
- [Contributing via Pull Requests](#contributing-via-pull-requests)
7+
- [Dev setup](#dev-setup)
8+
- [Local documentation](#local-documentation)
9+
- [Conventions](#conventions)
10+
- [General terminology and practices](#general-terminology-and-practices)
11+
- [Testing definition](#testing-definition)
12+
- [Finding contributions to work on](#finding-contributions-to-work-on)
13+
- [Code of Conduct](#code-of-conduct)
14+
- [Security issue notifications](#security-issue-notifications)
15+
- [Troubleshooting](#troubleshooting)
16+
- [API reference documentation](#api-reference-documentation)
17+
- [Licensing](#licensing)
218

319
# Contributing Guidelines
420

@@ -25,9 +41,11 @@ Contributions via pull requests are much appreciated. Before sending us a pull r
2541

2642
### Dev setup
2743

44+
[![Gitpod Ready-to-Code](https://img.shields.io/badge/Gitpod-Ready--to--Code-blue?logo=gitpod)](https://gitpod.io/from-referrer/)
45+
2846
Firstly, [fork the repository](https://github.com/awslabs/aws-lambda-powertools-python/fork).
2947

30-
To setup your development environment, we recommend using our pre-configured Cloud environment: https://gitpod.io/#https://github.com/YOUR_USERNAME/aws-lambda-powertools-python. Replace YOUR_USERNAME with your GitHub username or organization so the Cloud environment can target your fork accordingly.
48+
To setup your development environment, we recommend using our pre-configured Cloud environment: <https://gitpod.io/#https://github.com/YOUR_USERNAME/aws-lambda-powertools-python>. Replace YOUR_USERNAME with your GitHub username or organization so the Cloud environment can target your fork accordingly.
3149

3250
Alternatively, you can use `make dev` within your local virtual environment.
3351

@@ -47,22 +65,38 @@ GitHub provides additional document on [forking a repository](https://help.githu
4765

4866
You might find useful to run both the documentation website and the API reference locally while contributing:
4967

50-
* **API reference**: `make docs-api-local`
51-
* **Docs website**: `make docs-local`
52-
- If you prefer using Docker: `make docs-local-docker`
68+
- **API reference**: `make docs-api-local`
69+
- **Docs website**: `make docs-local`
70+
- If you prefer using Docker: `make docs-local-docker`
71+
72+
## Conventions
73+
74+
### General terminology and practices
5375

54-
### Conventions
76+
| Category | Convention |
77+
| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
78+
| **Docstring** | We use a slight variation of Numpy convention with markdown to help generate more readable API references. |
79+
| **Style guide** | We use black as well as flake8 extensions to enforce beyond good practices [PEP8](https://pep8.org/). We use type annotations and enforce static type checking at CI (mypy). |
80+
| **Core utilities** | Core utilities use a Class, always accept `service` as a constructor parameter, can work in isolation, and are also available in other languages implementation. |
81+
| **Utilities** | Utilities are not as strict as core and focus on solving a developer experience problem while following the project [Tenets](https://awslabs.github.io/aws-lambda-powertools-python/#tenets). |
82+
| **Exceptions** | Specific exceptions live within utilities themselves and use `Error` suffix e.g. `MetricUnitError`. |
83+
| **Git commits** | We follow [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/). We do not enforce conventional commits on contributors to lower the entry bar. Instead, we enforce a conventional PR title so our label automation and changelog are generated correctly. |
84+
| **API documentation** | API reference docs are generated from docstrings which should have Examples section to allow developers to have what they need within their own IDE. Documentation website covers the wider usage, tips, and strive to be concise. |
85+
| **Documentation** | We treat it like a product. We sub-divide content aimed at getting started (80% of customers) vs advanced usage (20%). We also ensure customers know how to unit test their code when using our features. |
5586

56-
Category | Convention
57-
------------------------------------------------- | ---------------------------------------------------------------------------------
58-
**Docstring** | We use a slight variation of Numpy convention with markdown to help generate more readable API references.
59-
**Style guide** | We use black as well as flake8 extensions to enforce beyond good practices [PEP8](https://pep8.org/). We use type annotations and enforce static type checking at CI (mypy).
60-
**Core utilities** | Core utilities use a Class, always accept `service` as a constructor parameter, can work in isolation, and are also available in other languages implementation.
61-
**Utilities** | Utilities are not as strict as core and focus on solving a developer experience problem while following the project [Tenets](https://awslabs.github.io/aws-lambda-powertools-python/#tenets).
62-
**Exceptions** | Specific exceptions live within utilities themselves and use `Error` suffix e.g. `MetricUnitError`.
63-
**Git commits** | We follow [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/). We do not enforce conventional commits on contributors to lower the entry bar. Instead, we enforce a conventional PR title so our label automation and changelog are generated correctly.
64-
**API documentation** | API reference docs are generated from docstrings which should have Examples section to allow developers to have what they need within their own IDE. Documentation website covers the wider usage, tips, and strive to be concise.
65-
**Documentation** | We treat it like a product. We sub-divide content aimed at getting started (80% of customers) vs advanced usage (20%). We also ensure customers know how to unit test their code when using our features.
87+
### Testing definition
88+
89+
We group tests in different categories
90+
91+
| Test | When to write | Notes | Speed |
92+
| ----------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------- |
93+
| Unit tests | Verify the smallest possible unit works. | Networking access is prohibited. Prefer Functional tests given our complexity. | Lightning fast (nsec to ms) |
94+
| Functional tests | Guarantee functionality works as expected. It's a subset of integration test covering multiple units. | No external dependency. Prefer Fake implementations (in-memory) over Mocks and Stubs. | Fast (ms to few seconds at worst) |
95+
| Integration tests | Gain confidence that code works with one or more external dependencies. | No need for a Lambda function. Use our code base against an external dependency _e.g., fetch an existing SSM parameter_. | Moderate to slow (a few minutes) |
96+
| End-to-end tests | Gain confidence that a Lambda function with our code operates as expected. | It simulates how customers configure, deploy, and run their Lambda function - Event Source configuration, IAM permissions, etc. | Slow (minutes) |
97+
| Performance tests | Ensure critical operations won't increase latency and costs to customers. | CI arbitrary hardware can make it flaky. We'll resume writing perf test after our new Integ/End have significant coverage. | Fast to moderate (a few seconds to a few minutes) |
98+
99+
Functional tests is mandatory. Maintainers will help indicate whether additional tests are necessary and provide assistance as required.
66100

67101
## Finding contributions to work on
68102

@@ -75,6 +109,7 @@ For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of
75109
opensource-codeofconduct@amazon.com with any additional questions or comments.
76110

77111
## Security issue notifications
112+
78113
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
79114

80115
## Troubleshooting
@@ -85,9 +120,9 @@ When you are working on the codebase and you use the local API reference documen
85120

86121
This happens when:
87122

88-
* You did not install the local dev environment yet
123+
- You did not install the local dev environment yet
89124
- You can install dev deps with `make dev` command
90-
* The code in the repository is raising an exception while the `pdoc` is scanning the codebase
125+
- The code in the repository is raising an exception while the `pdoc` is scanning the codebase
91126
- Unfortunately, this exception is not shown to you, but if you run, `poetry run pdoc --pdf aws_lambda_powertools`, the exception is shown and you can prevent the exception from being raised
92127
- Once resolved the documentation should load correctly again
93128

0 commit comments

Comments
 (0)