Description
Expected Behaviour
When making a function idempotent via the @idempotent
decorator and enabling payload validation the Idempotency utility should raise a payload validation exception when the payload in subsequent requests differs from the one stored.
With this configuration:
persistence_layer = DynamoDBPersistenceLayer(table_name=os.environ["TABLE_NAME"])
config = IdempotencyConfig(
event_key_jmespath='["rcn", "subscriptionType"]',
payload_validation_jmespath="customerAccounts",
)
we are using two fields from the payload to generate the idempotency hash (rcn
and subscriptionType
) but we are using a different key (customerAccounts
) for payload validation.
If we send the following two requests:
{
"rcn": "RCN-000-000-000-000",
"customerAccounts": {
"customerAccount": [
{
"accountId": "12345",
"accountType": "someaccounttype"
}
]
},
"subscriptionType": "somesubscriptiontype"
}
and
{
"rcn": "RCN-000-000-000-000",
"customerAccounts": {
"customerAccount": [
{
"accountId": "54321",
"accountType": "someaccounttype"
}
]
},
"subscriptionType": "somesubscriptiontype"
}
These two requests should be considered idempotent because the compound value of rcn
and subscriptionType
is the same across requests.
At the same time, the values under customerAccounts
have changed between requests (note customerAccounts.customerAccount[0].accountId
being different). This means the payload validation should fail in the second request.
Current Behaviour
The function does not throw a validation error even though payload validation is enabled and parts of the payload differ from the stored one.
See screenshot below, where I send both requests and both are successful:
Code snippet
import os
from aws_lambda_powertools.utilities.idempotency import (
DynamoDBPersistenceLayer,
IdempotencyConfig,
idempotent,
)
from aws_lambda_powertools.utilities.typing import LambdaContext
persistence_layer = DynamoDBPersistenceLayer(table_name=os.environ["TABLE_NAME"])
config = IdempotencyConfig(
event_key_jmespath='["rcn", "subscriptionType"]',
payload_validation_jmespath="customerAccounts",
)
@idempotent(persistence_store=persistence_layer, config=config)
def handler(event: dict, context: LambdaContext):
return True
Possible Solution
As part of a feature added in #3446 we are using DynamoDB ReturnValuesOnConditionCheckFailure
whenever available. This feature allows the DynamoDB to return an item that caused a conditional operation to fail, which in turn allows the Idempotency utility to get the IdempotencyRecord
of a request without having to do an additional read.
In the base idempotency class (here), we have added an if/else
statement that uses the returned item when available, or otherwise it calls the _get_idemptoency_record()
method when one is not available - this last case can happen when customers are using older versions of boto3
.
With the current implementation, the logic that validates the payload is executed only in the else
clause, which causes payload validation to be skipped.
A potential fix would be to make the _validate_payload()
method in the BasePersistenceLayer
class and then call it on the exc.old_data_record
, similar to this:
record = exc.old_data_record
if exc.old_data_record is not None:
self.persistence_store.validate_payload(
self.data,
record
)
else:
record = self._get_idempotency_record()
Steps to Reproduce
See steps above.
Powertools for AWS Lambda (Python) version
latest
AWS Lambda function runtime
3.12
Packaging format used
PyPi
Debugging logs
No response
Metadata
Metadata
Assignees
Type
Projects
Status