Skip to content

Bug: Custom CloudWatch metrics not always appearing for split route(s) #1805

Closed
@pawosty

Description

@pawosty

Expected Behaviour

When uploading a custom metric using the aws_lambda_powertools.metrics package, the metrics should appear in CloudWatch Metrics.

Current Behaviour

Metrics appear intermittently and appear to be tied to whether a lambda was cold started (and also had a cold start metric uploaded).

Code snippet

Split route handler:

metrics: Metrics = Metrics()

@router.delete("/v1/asset", cors=True)
@tracer.capture_method
def delete_asset() -> dict:
    metrics.add_metric(name="AssetDeleted", unit=MetricUnit.Count, value=1)
    return {"message": "Asset deletion successful"}

Lambda handler:

@metrics.log_metrics(capture_cold_start_metric=True)
@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST, log_event=True)
@tracer.capture_lambda_handler
@auth.enforce_allow_list
@auth.restrict_api_key
def lambda_handler(event: Dict, context: LambdaContext):
    return app.resolve(event, context)

Possible Solution

When examining the cloudwatch logs, the unsuccessful metrics appear in the same log line as the tracer log, for example:

{
    "resource": "/v1/asset",
    "path": "/v1/asset",
    "httpMethod": "DELETE",
...more data...
}{
    "_aws": {
        "Timestamp": 1672800187055,
        "CloudWatchMetrics": [
            {
                "Namespace": "MyNamespace",
                "Dimensions": [
                    [
                        "service"
                    ]
                ],
                "Metrics": [
                    {
                        "Name": "AssetDeleted",
                        "Unit": "Count"
                    }
                ]
            }
        ]
    },
    "service": "MyLambda",
    "AssetDeleted": [
        1
    ]
}

When successful metrics appear (such as when they are uploaded as part of a cold start), they appear in their own log line in CloudWatch and are presumably parsed.

Steps to Reproduce

  • Create a split route lambda handler, log cold starts
  • In the split route(s) attempt to create/upload custom metrics
  • Observe not all metrics appear in CloudWatch metrics as expected

AWS Lambda Powertools for Python version

latest

AWS Lambda function runtime

3.6

Packaging format used

PyPi

Debugging logs

No response

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions