Bug: Kinesis Firehose SQS Record schema should fail parsing if data is invalid

[dreamorosi]

Expected Behavior

As per title, when working with a Kinesis Firehose event that contains a SQS event in the data field, if the field is not a valid SQS event the entire parsing should fail.

Current Behavior

Currently if this happens, the parsing is successful and the data is returned as-is.

Code snippet

N/A

Steps to Reproduce

N/A

Possible Solution

The schema should be modified from this:

const KinesisFirehoseSqsRecordSchema = KinesisFireHoseRecordBase.extend({
  data: z.string().transform((data) => {
    try {
      return SqsRecordSchema.parse(
        JSON.parse(Buffer.from(data, 'base64').toString('utf8'))
      );
    } catch (e) {
      return data;
    }
  }),
});

to something like this:

const KinesisFirehoseSqsRecordSchema = KinesisFireHoseRecordBase.extend({
  data: z.string().transform((data, ctx) => {
    try {
      return SqsRecordSchema.parse(
        JSON.parse(Buffer.from(data, 'base64').toString('utf8'))
      );
    } catch (e) {
      ctx.addIssue({
        code: z.ZodIssueCode.custom,
        message: 'Failed to parse SQS record',
        fatal: true,
      });

      return z.NEVER;
    }
  }),
});

Powertools for AWS Lambda (TypeScript) version

latest

AWS Lambda function runtime

20.x

Packaging format used

npm

Execution logs

No response

[dreamorosi]

While working on this, if picked up after #3248, remove change the corresponding unit test from it.fails to it in the packages/parser/tests/unit/schema/kinesis-firehose.test.ts file.

[svozza]

Happy to pick this up if you want. Seems like a very straightforward change.

[github-actions]

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.