Add support for serializing Enum values automatically in API Gateway Event Handler

[marcioemiranda]

If a route function returns a dict containing an Enum, the json encoder will fail.

Expected Behavior

Since the Response was not returned, API Gateway event handler will create one. It calls json.dumps to serialize the dict using its own encoder.
The expected json output would transform the Enum in its value.

Current Behavior

The current behavior is that the encoder throws a TypeError saying Enum is not serializable.

Possible Solution

My suggestion is to check if the object is an Enum and output its value.
The example bellow handle Enum and also a set

class CustomEncoder(JSONEncoder):
    def default(self, o):
        if isinstance(o, Enum):
            return o.value
        try:
            iterable = iter(o)
        except TypeError:
            pass
        else:
            return list(iterable)    
        # Let the base class default method raise the TypeError    
        return JSONEncoder.default(self, o)

For now the workaround is to always create a Response and call json.dumps with my own custom encoder

Steps to Reproduce (for bugs)

Try the following code

from enum import Enum

class Color(Enum): 
   RED =1
   BLUE = 2

@app.get("/colors")
def get_color() -> Dict:
   return {
      "color": Color.RED,
      "variations": {"light", "dark"} 
   }

Environment

Lambda Power Tools version 1.17.1
Running with PyTest

../../../../Library/Caches/pypoetry/virtualenvs/te-admin-api-y-cvF00h-py3.8/lib/python3.8/site-packages/aws_lambda_powertools/tracing/tracer.py:313: in decorate
    response = lambda_handler(event, context, **kwargs)
_admin_controller.py:273: in handler
    return app.resolve(event, context)
../../../../Library/Caches/pypoetry/virtualenvs/te-admin-api-y-cvF00h-py3.8/lib/python3.8/site-packages/aws_lambda_powertools/event_handler/api_gateway.py:418: in resolve
    return self._resolve().build(self.current_event, self._cors)
../../../../Library/Caches/pypoetry/virtualenvs/te-admin-api-y-cvF00h-py3.8/lib/python3.8/site-packages/aws_lambda_powertools/event_handler/api_gateway.py:450: in _resolve
    return self._call_route(route, match.groupdict())
../../../../Library/Caches/pypoetry/virtualenvs/te-admin-api-y-cvF00h-py3.8/lib/python3.8/site-packages/aws_lambda_powertools/event_handler/api_gateway.py:478: in _call_route
    return ResponseBuilder(self._to_response(route.func(**args)), route)
../../../../Library/Caches/pypoetry/virtualenvs/te-admin-api-y-cvF00h-py3.8/lib/python3.8/site-packages/aws_lambda_powertools/event_handler/api_gateway.py:497: in _to_response
    body=json.dumps(result, separators=(",", ":"), cls=Encoder),
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py:234: in dumps
    return cls(
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py:199: in encode
    chunks = self.iterencode(o, _one_shot=True)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py:257: in iterencode
    return _iterencode(o, 0)
../../../../Library/Caches/pypoetry/virtualenvs/te-admin-api-y-cvF00h-py3.8/lib/python3.8/site-packages/aws_lambda_powertools/shared/json_encoder.py:16: in default
    return super().default(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <aws_lambda_powertools.shared.json_encoder.Encoder object at 0x7f915fbaea90>
o = <Region.CENTRAL: 6>

    def default(self, o):
        """Implement this method in a subclass such that it returns
        a serializable object for ``o``, or calls the base implementation
        (to raise a ``TypeError``).
    
        For example, to support arbitrary iterators, you could
        implement default like this::
    
            def default(self, o):
                try:
                    iterable = iter(o)
                except TypeError:
                    pass
                else:
                    return list(iterable)
                # Let the base class default method raise the TypeError
                return JSONEncoder.default(self, o)
    
        """
>       raise TypeError(f'Object of type {o.__class__.__name__} '
                        f'is not JSON serializable')
E       TypeError: Object of type Region is not JSON serializable

/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py:179: TypeError

[heitorlessa]

Hey @marcioemiranda, thanks for raising the issue.

This is not a bug though but expected behaviour as Enum are not serialized - I'm changing it to a feature request instead and wait for other customers to chime in with 👍.

You'd have the same behaviour when using other web frameworks like Flask for example:

from flask import jsonify, Flask
from enum import Enum


class Color(Enum):
    RED = 1
    BLUE = 2


app = Flask(__name__)


@app.route("/colors")
def get_color():
    return jsonify({"color": Color.RED, "variations": {"light", "dark"}})

[2021-07-16 21:22:13,054] ERROR in app: Exception on /colors [GET]
Traceback (most recent call last):
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/lessa/DEV/pt-issue-520/app.py", line 15, in get_color
    return jsonify({"color": Color.RED, "variations": {"light", "dark"}})
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/json/__init__.py", line 370, in jsonify
    dumps(data, indent=indent, separators=separators) + "\n",
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/json/__init__.py", line 211, in dumps
    rv = _json.dumps(obj, **kwargs)
  File "/Users/lessa/.pyenv/versions/3.8.2/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/Users/lessa/.pyenv/versions/3.8.2/lib/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Users/lessa/.pyenv/versions/3.8.2/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/Users/lessa/.pyenv/versions/3.8.2/envs/work3.8/lib/python3.8/site-packages/flask/json/__init__.py", line 100, in default
    return _json.JSONEncoder.default(self, o)
  File "/Users/lessa/.pyenv/versions/3.8.2/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Color is not JSON serializable

[marcioemiranda]

@heitorlessa correct, it's a feature request. Meanwhile I am using a custom encoder. It would be nice though to have types such as Enum and set handled by LPT json encoder.

[ran-isenberg]

Enums are great but when you return a value for the handler to return back to the user, it must be a json-able value. Instead of Color.RED, write Color.RED.value or str(Color.red). Not sure what kind of feature is required here.

[marcioemiranda]

@risenberg-cyberark, thanks for the comment.

I was thinking something similar to this JSON encoder
https://pypi.org/project/orjson/

It handles types such as Dataclass, Enum, datetime, Decimal, UUID and others. Other cool features in the roadmap, such as filtering attributes of a Dataclass in the encoding process.

The ApiGatewayResolver can wrap the value returned by a handler in a Response object if it can handle the serialization, otherwise I have to use a custom encoder and create the Response myself. This is totally fine, but creates some boilerplate code. It would be nicer if ApiGatewayResolver json encoder could handle more types natively.

Other alternatives could be:

1. Let one specify an alternative json encoder in ApiGatewayResolver configuration
2. Let one specify a default function to handle the encode of non-supported types in ApiGatewayResolver configuration

[michaelbrewer]

@marcioemiranda should be easy to implement, but i would agree with @heitorlessa about see what the consensus should be on this. I can put up a draft PR to see what the impact would be

Let one specify an alternative json encoder in ApiGatewayResolver configuration

Otherwise alternative option 1, but be a good solution to allow for more flexibility.

[michaelbrewer]

@marcioemiranda you can also use IntEnum:

>>> from enum import IntEnum
>>> import json
>>>
>>>
>>> class Shape(IntEnum):
...     Circle = 1
...     Square = 2
...
...
>>> x = {"square": Shape.Square}
>>>
>>> print(json.dumps(x))
{"square": 2}

[ran-isenberg]

What we usually do it to define a schema for input validation and output validation in a lambda. We define it using Pydantic.
We return the main handler an output object, so all it needs to is to call the object's json() or dict() function which does the serialisation for it. This way I know that i'm returning a valid output back the api gw.

see https://pydantic-docs.helpmanual.io/usage/exporting_models/

[michaelbrewer]

@risenberg-cyberark haha, but this is not for Pydantic.

This works from for me:

from enum import IntEnum

class Color(IntEnum): 
   RED =1
   BLUE = 2

@app.get("/colors")
def get_color() -> Dict:
   return {
      "color": Color.RED,
      "variations": {"light", "dark"} 
   }