Skip to content

improv: add example kitchen sink code #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 31, 2020
Merged

improv: add example kitchen sink code #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f1ca408
fix: delete Pipenv files
heitorlessa Mar 31, 2020
0f3cc01
feat: add example project
heitorlessa Mar 31, 2020
9e3d0e0
fix: clean up duplicate readme
heitorlessa Mar 31, 2020
316afda
chore: linting
heitorlessa Mar 31, 2020
141ab12
fix: CI build
heitorlessa Mar 31, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 3 additions & 186 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,194 +2,11 @@

![PackageStatus](https://img.shields.io/static/v1?label=status&message=beta&color=blueviolet?style=flat-square) ![PythonSupport](https://img.shields.io/static/v1?label=python&message=3.6%20|%203.7|%203.8&color=blue?style=flat-square&logo=python)

A suite of utilities for AWS Lambda Functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier - Currently available for Python only and compatible with Python >=3.6.
A suite of utilities for AWS Lambda Functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier.

**Status**: Beta
## Powertools available

## Features

**Tracing**

> It currently uses AWS X-Ray

* Decorators that capture cold start as annotation, and response and exceptions as metadata
* Run functions locally without code change to disable tracing
* Explicitly disable tracing via env var `POWERTOOLS_TRACE_DISABLED="true"`

**Logging**

* Decorators that capture key fields from Lambda context, cold start and structures logging output as JSON
* Optionally log Lambda request when instructed (disabled by default)
- Enable via `POWERTOOLS_LOGGER_LOG_EVENT="true"` or explicitly via decorator param
* Logs canonical custom metric line to logs that can be consumed asynchronously
* Log sampling enables DEBUG log level for a percentage of requests (disabled by default)
- Enable via `POWERTOOLS_LOGGER_SAMPLE_RATE=0.1`, ranges from 0 to 1, where 0.1 is 10% and 1 is 100%

**Environment variables** used across suite of utilities

Environment variable | Description | Default | Utility
------------------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | -------------------------------------------------
POWERTOOLS_SERVICE_NAME | Sets service name used for tracing namespace, metrics dimensions and structured logging | "service_undefined" | all
POWERTOOLS_TRACE_DISABLED | Disables tracing | "false" | tracing
POWERTOOLS_LOGGER_LOG_EVENT | Logs incoming event | "false" | logging
POWERTOOLS_LOGGER_SAMPLE_RATE | Debug log sampling | 0 | logging
LOG_LEVEL | Sets logging level | "INFO" | logging

## Usage

### Installation

With [pip](https://pip.pypa.io/en/latest/index.html) installed, run: ``pip install aws-lambda-powertools``

### Tracing

**Example SAM template using supported environment variables**

```yaml
Globals:
Function:
Tracing: Active # can also be enabled per function
Environment:
Variables:
POWERTOOLS_SERVICE_NAME: "payment"
POWERTOOLS_TRACE_DISABLED: "false"
```

**Pseudo Python Lambda code**

```python
from aws_lambda_powertools.tracing import Tracer
tracer = Tracer()
# tracer = Tracer(service="payment") # can also be explicitly defined

@tracer.capture_method
def collect_payment(charge_id):
# logic
ret = requests.post(PAYMENT_ENDPOINT)
# custom annotation
tracer.put_annotation("PAYMENT_STATUS", "SUCCESS")
return ret

@tracer.capture_lambda_handler
def handler(event, context)
charge_id = event.get('charge_id')
payment = collect_payment(charge_id)
...
```


### Logging

**Example SAM template using supported environment variables**

```yaml
Globals:
Function:
Environment:
Variables:
POWERTOOLS_SERVICE_NAME: "payment"
POWERTOOLS_LOGGER_SAMPLE_RATE: 0.1 # enable debug logging for 1% of requests, 0% by default
LOG_LEVEL: "INFO"
```

**Pseudo Python Lambda code**

```python
from aws_lambda_powertools.logging import logger_setup, logger_inject_lambda_context

logger = logger_setup()
# logger_setup(service="payment") # also accept explicit service name
# logger_setup(level="INFO") # also accept explicit log level

@logger_inject_lambda_context
def handler(event, context)
logger.info("Collecting payment")
...
# You can log entire objects too
logger.info({
"operation": "collect_payment",
"charge_id": event['charge_id']
})
...
```

**Exerpt output in CloudWatch Logs**

```json
{
"timestamp":"2019-08-22 18:17:33,774",
"level":"INFO",
"location":"collect.handler:1",
"service":"payment",
"lambda_function_name":"test",
"lambda_function_memory_size":"128",
"lambda_function_arn":"arn:aws:lambda:eu-west-1:12345678910:function:test",
"lambda_request_id":"52fdfc07-2182-154f-163f-5f0f9a621d72",
"cold_start": "true",
"message": "Collecting payment"
}

{
"timestamp":"2019-08-22 18:17:33,774",
"level":"INFO",
"location":"collect.handler:15",
"service":"payment",
"lambda_function_name":"test",
"lambda_function_memory_size":"128",
"lambda_function_arn":"arn:aws:lambda:eu-west-1:12345678910:function:test",
"lambda_request_id":"52fdfc07-2182-154f-163f-5f0f9a621d72",
"cold_start": "true",
"message":{
"operation":"collect_payment",
"charge_id": "ch_AZFlk2345C0"
}
}
```

#### Custom Metrics async

> **NOTE**: This will **likely change after Beta** in light of [new Amazon CloudWatch embedded metric format](https://aws.amazon.com/about-aws/whats-new/2019/11/amazon-cloudwatch-launches-embedded-metric-format/), meaning we won't need an additional stack and interface could change.

This feature requires [Custom Metrics SAR App](https://serverlessrepo.aws.amazon.com/applications/arn:aws:serverlessrepo:us-east-1:374852340823:applications~async-custom-metrics) in order to process canonical metric lines in CloudWatch Logs.

If you're starting from scratch, you may want to see a working example, tune to your needs and deploy within your account - [Serverless Airline Log Processing Stack](https://github.com/aws-samples/aws-serverless-airline-booking/blob/develop/src/backend/log-processing/template.yaml)

```python
from aws_lambda_powertools.logging import MetricUnit, log_metric

def handler(event, context)
log_metric(name="SuccessfulPayment", unit=MetricUnit.Count, value=10, namespace="MyApplication")

# Optional dimensions
log_metric(name="SuccessfulPayment", unit=MetricUnit.Count, value=10, namespace="MyApplication", customer_id="123-abc", charge_id="abc-123")

# Explicit service name
log_metric(service="paymentTest", name="SuccessfulPayment", namespace="MyApplication".....)
...
```

**Exerpt output in CloudWatch Logs**

```
MONITORING|10|Count|SuccessfulPayment|MyApplication|service="payment
MONITORING|10|Count|SuccessfulPayment|MyApplication|customer_id="123-abc",charge_id="abc-123",service="payment
MONITORING|10|Count|SuccessfulPayment|MyApplication|service="paymentTest
```


## Beta

This library may change its API/methods or environment variables as it receives feedback from customers. Currently looking for ideas in the following areas before making it stable:

* **Should Tracer patch all possible imported libraries by default or only AWS SDKs?**
- Patching all libraries may have a small performance penalty (~50ms) at cold start
- Alternatively, we could patch only AWS SDK if available and to provide a param to patch multiple `Tracer(modules=("boto3", "requests"))`
* **Create a Tracer provider to support additional tracing**
- Either duck typing or ABC to allow additional tracing providers

## TODO

* [ ] Add an example code using powertools
* [Python - Beta](./python/README.md)

## Credits

Expand Down
3 changes: 2 additions & 1 deletion python/.flake8
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[flake8]
exclude = docs, .eggs, setup.py
exclude = docs, .eggs, setup.py, example, .aws-sam
ignore = E203, E266, W503, BLK100, W291
max-line-length = 120
max-complexity = 18
Expand All @@ -10,3 +10,4 @@ include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
line_length = 120

30 changes: 0 additions & 30 deletions python/Pipfile
View file
Open in desktop

This file was deleted.

Loading