Feature request: APIGatewayRestResolver exception_handler compress data

[royassis]

Expected Behaviour

exception_handler does not compress response

Current Behaviour

exception_handler compress response if decorator contained compress=True

Code snippet

from aws_lambda_powertools.event_handler import content_types

@app.exception_handler(Exception)
def handle_invalid_params(ex: Exception):
    response = Response(
            status_code=500,
            content_type=content_types.APPLICATION_JSON,
            body=json.dumps({"ex": str(ex)})
        )
    return response 

@app.get("/", compress=True)
def foo(datasetid: str):
    raise Exception

Possible Solution

No response

Steps to Reproduce

Run code snippet

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.7

Packaging format used

Serverless Application Repository (SAR) App

Debugging logs

No response

[leandrodamascena]

Looking at this now.

[leandrodamascena]

Hello, @royassis! Thank you so much for opening issues to report possible bugs, we really appreciate it because we can understand how customers are using Powertools, their needs, and potential improvements we can make based on that!

I was inspecting this report and I couldn't reproduce this. I would appreciate your help reviewing the steps I'm doing to see that I'm not missing anything you're using on your end.

1 - I created this application that raises a NameError exception (you can get another in your case)

import json

from aws_lambda_powertools import Logger
from aws_lambda_powertools.event_handler import (
    APIGatewayRestResolver,
    Response,
    content_types,
)
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.utilities.typing import LambdaContext

logger = Logger()
app = APIGatewayRestResolver()


@app.exception_handler(NameError)
def handle_invalid_params(ex: NameError):
    response = Response(
            status_code=500,
            content_type=content_types.APPLICATION_JSON,
            body=json.dumps({"ex": str(ex)})
        )
    return response 

@app.get("/hello", compress=True)
def hello():
    raise NameError("Error")


# You can continue to use other utilities just as before
@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
def lambda_handler(event: dict, context: LambdaContext) -> dict:
    return app.resolve(event, context)

2 - I sent a request to this URL with cURL

curl -v --location --request GET 'https://...amazonaws.../hello' --header 'Accept-Encoding: gzip, deflate, br'

and then I got the expected response

❯ curl -v --location --request GET 'https://...amazonaws.../hello' --header 'Accept-Encoding: gzip, deflate, br'
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 127.0.0.1:3000...
* Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
> GET /hello HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.82.0
> Accept: */*
> Accept-Encoding: gzip, deflate, br
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 500 INTERNAL SERVER ERROR
< Server: Werkzeug/2.2.3 Python/3.7.10
< Date: Thu, 22 Jun 2023 09:54:55 GMT
< Content-Type: application/json
< Content-Encoding: gzip
< Content-Length: 48
< Connection: close
< 
* Closing connection 0
H4sIAAAAAAACA6tWSq1QslJQci0qyi9SqgUAmcuV5A8AAAA=

                                                                                                                                                                                                                                                                        
❯ echo H4sIAAAAAAACA6tWSq1QslJQci0qyi9SqgUAmcuV5A8AAAA= |base64 -di | gunzip
{"ex": "Error"}                        

According to the AWS official documentation to enable compressed response, the client MUST send the Accept-Encoding header, otherwise, the response will be sent uncompressed.

Can you check please if you are sending this header in your request?

Thank you

[royassis]

Hey @leandrodamascena

I want the response for 2xx status to be compressed and encoded but, not encoded with status of 4xx 5xx

Before we talked I check I return value of app.resolve(event, context).
I noticed the Content-Encoding and removed it

if res.get("statusCode") != 200:
    del res["multiValueHeaders"]["Content-Encoding"]
    res["body"] = gzip.decompress(base64.b64decode(res["body"].encode())).decode()

After I removed it the body was not encoded (as was the desired result)

p.s I do not add this header myself. Probably added by the package.

[leandrodamascena]

Hi @royassis! Ohhh now I completely understand what you are waiting for. This is a very interesting question and potential addition to PowerTools. Following my considerations and what we can do in the next steps

1 - Use case
I understand you have a specific use case where you have one route (or many) and in any exception case - using exception_handler decorator - you don't want to compress the response in case of 4xx/5xx errors. So, in this way, you can send the response in plain and it works for routes with and without compression.

2 - Avoiding breaking changes
Unfortunately, we can't change the current default behavior, otherwise we may break the API contract with many customers who use Powertools and expect a compressed response (when using compress=True), even in the case of 4xx/5xx.

Just to share with you an overview of our strategies for handling breaking changes within this project: In the Powertools project, we prioritize customer preparation for breaking changes. We follow a strict process involving RFCs, clear communication, and ample advance notice. In general breaking changes are limited to major version updates to maintain stability. Following this process, we minimize disruptions and provide a seamless experience for all customers.

3 - Possible solution
Your use case may be the same for others and I think we can add in the Response constructor a new parameter to change the compression behavior, so it can override the compress defined in the route.

The code could look like this:

from aws_lambda_powertools.event_handler import content_types

@app.exception_handler(Exception)
def handle_invalid_params(ex: Exception):
    response = Response(
            status_code=500,
            content_type=content_types.APPLICATION_JSON,
            body=json.dumps({"ex": str(ex)}),
			compress=False
        )
    return response 

@app.get("/", compress=True)
def foo(datasetid: str):
    raise Exception

Do you think this works for you? Please give me feedback when you can.

Thank you so much.