[FR] Dataclass features

[efimk-lu]

Hey,
We are working with dataclasses quite a lot, and there are a couple of features missing for us when moving to the power tools parser:

• Ability to make an object immutable, a.k.a freeze, for example

@dataclass(frozen=True)
class TwitterBackupJob:
    twitter_account: str

• Support for code-based field initialization

@dataclass(frozen=True)
class TwitterBackupJob:
    backup_date: datetime = field(default_factory=datetime.now)

• Fast to/from json/dict Do you have any performance numbers?

[heitorlessa]

Hey @efimk-lu, thanks for creating a discussion! This is better accomplished with Event Source Data Classes utility, not Parser (Pydantic thin wrapper).

Evet Source Data Class is a class that implements the Mapping interface to interop nicely with older middlewares prior to Powertools existence and Python 3.6 - it also uses @property for each available field known to a given Event Source, and for common day-to-day tasks to increase productivity.

Parser on the other hand is exclusively a thin-wrapper for Pydantic with an envelope feature for data extraction. Its secondary focus is on deep data validation provided by Pydantic but naming is hard. The same data extraction can be achieved with our JMESPath Custom Functions as used elsewhere (Validation, Idempotency), without the slower startup time in Pydantic due to heavy reflection.

I've recreated your example using APIGatewayProxyEvent using three common options:

• A. Create a TwitterBackupJob from the payload (str) deserialized to JSON via json_body property
• B. Same as A but using dacite, as it supports deeply nested structures in case there are JSON strings within JSON payload
• C. Shortcut to A by using the decorator event_source to mutate the event and save you keystrokes

As for your second question on performance, we chose a Class for compatibility (3.6) and performance, since dataclass is heavy on code generation and known to be significantly slower. I no longer have my own benchmark data - I did with pybenchmark + load test against an actual Lambda on 128M (can do again if necessary). Now that we dropped 3.6, we are actively following the official discussion on making dataclasses faster in upcoming Python versions to decide whether to port it in the future.

Let me know if you have any additional questions!

---

import json
from dataclasses import dataclass
from pathlib import Path

import dacite

from aws_lambda_powertools.utilities.data_classes import (
    APIGatewayProxyEvent,
    event_source,
)


@dataclass(frozen=True)
class TwitterBackupJob:
    twitter_account: str


def lambda_handler(event: dict, context):
    # Option A
    data = APIGatewayProxyEvent(event)
    tweet = TwitterBackupJob(**data.json_body)
    if tweet.twitter_account == "@heitor_lessa":
        return "success"

    # Option B
    tweet_dacite = dacite.from_dict(data_class=TwitterBackupJob, data=data.json_body)
    if tweet_dacite.twitter_account == "@heitor_lessa":
        return "success"


# Option C
@event_source(data_class=APIGatewayProxyEvent)
def lambda_handler_decorated(event: APIGatewayProxyEvent, context):
    tweet = TwitterBackupJob(**event.json_body)
    if tweet.twitter_account == "@heitor_lessa":
        return "success"


apigw_payload: str = json.dumps({"twitter_account": "@heitor_lessa"})
apigw_event: dict = json.loads(Path("tests/events/apiGatewayProxyEvent.json").read_text())
apigw_event["body"] = apigw_payload

lambda_handler(apigw_event, object())

---

apiGatewayProxyEvent.json

{
  "version": "1.0",
  "resource": "/my/path",
  "path": "/my/path",
  "httpMethod": "GET",
  "headers": {
    "Header1": "value1",
    "Header2": "value2"
  },
  "multiValueHeaders": {
    "Header1": [
      "value1"
    ],
    "Header2": [
      "value1",
      "value2"
    ]
  },
  "queryStringParameters": {
    "parameter1": "value1",
    "parameter2": "value"
  },
  "multiValueQueryStringParameters": {
    "parameter1": [
      "value1",
      "value2"
    ],
    "parameter2": [
      "value"
    ]
  },
  "requestContext": {
    "accountId": "123456789012",
    "apiId": "id",
    "authorizer": {
      "claims": null,
      "scopes": null
    },
    "domainName": "id.execute-api.us-east-1.amazonaws.com",
    "domainPrefix": "id",
    "extendedRequestId": "request-id",
    "httpMethod": "GET",
    "identity": {
      "accessKey": null,
      "accountId": null,
      "caller": null,
      "cognitoAuthenticationProvider": null,
      "cognitoAuthenticationType": null,
      "cognitoIdentityId": null,
      "cognitoIdentityPoolId": null,
      "principalOrgId": null,
      "sourceIp": "192.168.0.1/32",
      "user": null,
      "userAgent": "user-agent",
      "userArn": null,
      "clientCert": {
        "clientCertPem": "CERT_CONTENT",
        "subjectDN": "www.example.com",
        "issuerDN": "Example issuer",
        "serialNumber": "a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1",
        "validity": {
          "notBefore": "May 28 12:30:02 2019 GMT",
          "notAfter": "Aug  5 09:36:04 2021 GMT"
        }
      }
    },
    "path": "/my/path",
    "protocol": "HTTP/1.1",
    "requestId": "id=",
    "requestTime": "04/Mar/2020:19:15:17 +0000",
    "requestTimeEpoch": 1583349317135,
    "resourceId": null,
    "resourcePath": "/my/path",
    "stage": "$default"
  },
  "pathParameters": null,
  "stageVariables": null,
  "body": "Hello from Lambda!",
  "isBase64Encoded": false
}

[efimk-lu]

Tx @heitorlessa , looks good