Feature request: improve error handling

[dreamorosi]

Use case

Currently some of the errors in the Batch Processing utility are either generic or hard to parse/read. We could improve on them considering that we encourage JSON-structured logging.

Solution/User Experience

Currently the error thrown when the full batch fails processing looks like this:

{
    "errorType": "Error",
    "errorMessage": "\nAll records failed processing. 1 individual errors logged separately below.\n,Failed to process record, 3\n",
    "msg": "All records failed processing. 1 individual errors logged separately below.",
    "childErrors": [
        {
            "errorType": "Error",
            "errorMessage": "Failed to process record, 3",
            "stack": [
                "Error: Failed to process record, 3",
                "    at BatchProcessor2.recordHandler [as handler] (/var/task/index.js:416:11)",
                "    at BatchProcessor2.processRecord (/var/task/index.js:250:31)",
                "    at BatchProcessor2.process (/var/task/index.js:131:38)",
                "    at processPartialResponse2 (/var/task/index.js:299:18)",
                "    at Runtime.handler (/var/task/index.js:421:50)",
                "    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1083:29)"
            ]
        }
    ],
    "stack": [
        "Error: ",
        "All records failed processing. 1 individual errors logged separately below.",
        ",Failed to process record, 3",
        "",
        "    at BatchProcessor2.clean (/var/task/index.js:179:17)",
        "    at BatchProcessor2.process (/var/task/index.js:133:14)",
        "    at processPartialResponse2 (/var/task/index.js:299:18)",
        "    at Runtime.handler (/var/task/index.js:421:50)",
        "    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1083:29)"
    ]
}

The error message could use some improvement and be rendered instead like:

{
    "errorType": "FullBatchFailureError",
    "errorMessage": "All records failed processing. See individual errors below.",
    "name": "FullBatchFailureError",
    "recordErrors": [
        {
            "errorType": "Error",
            "errorMessage": "Failed to process record, 2",
            "stack": [
                "Error: Failed to process record, 2",
                "    at BatchProcessor2.recordHandler [as handler] (/var/task/index.js:421:11)",
                "    at BatchProcessor2.processRecord (/var/task/index.js:256:31)",
                "    at BatchProcessor2.process (/var/task/index.js:137:38)",
                "    at processPartialResponse2 (/var/task/index.js:304:18)",
                "    at Runtime.handler (/var/task/index.js:426:50)",
                "    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1083:29)"
            ]
        },
        {
            "errorType": "Error",
            "errorMessage": "Failed to process record, 3",
            "stack": [
                "Error: Failed to process record, 3",
                "    at BatchProcessor2.recordHandler [as handler] (/var/task/index.js:421:11)",
                "    at BatchProcessor2.processRecord (/var/task/index.js:256:31)",
                "    at BatchProcessor2.process (/var/task/index.js:137:38)",
                "    at processPartialResponse2 (/var/task/index.js:304:18)",
                "    at Runtime.handler (/var/task/index.js:426:50)",
                "    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1083:29)"
            ]
        },
        {
            "errorType": "Error",
            "errorMessage": "Failed to process record, 4",
            "stack": [
                "Error: Failed to process record, 4",
                "    at BatchProcessor2.recordHandler [as handler] (/var/task/index.js:421:11)",
                "    at BatchProcessor2.processRecord (/var/task/index.js:256:31)",
                "    at BatchProcessor2.process (/var/task/index.js:137:38)",
                "    at processPartialResponse2 (/var/task/index.js:304:18)",
                "    at Runtime.handler (/var/task/index.js:426:50)",
                "    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1083:29)"
            ]
        }
    ],
    "stack": [
        "FullBatchFailureError: All records failed processing. See individual errors below.",
        "    at BatchProcessor2.clean (/var/task/index.js:185:17)",
        "    at BatchProcessor2.process (/var/task/index.js:139:14)",
        "    at processPartialResponse2 (/var/task/index.js:304:18)",
        "    at Runtime.handler (/var/task/index.js:426:50)",
        "    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1083:29)"
    ]
}

Likewise, other errors thrown when the utility receives a wrong type, or short-circuits the processing of a FIFO SQS batch, can be improved either in their implementation or output.

Alternative solutions

No response

Acknowledgment

• This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[github-actions]

⚠️ COMMENT VISIBILITY WARNING ⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions]

This is now released under v1.13.0 version!