Skip to content

Commit 57681a2

Browse files
committed
docs: reorder data validation; improve envelopes section
Signed-off-by: heitorlessa <lessa@amazon.co.uk>
1 parent de53605 commit 57681a2

File tree

1 file changed

+130
-109
lines changed

1 file changed

+130
-109
lines changed

docs/content/utilities/parser.mdx

Lines changed: 130 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ Use the decorator for fail fast scenarios where you want your Lambda function to
7070
```python:title=event_parser_decorator.py
7171
from aws_lambda_powertools.utilities.parser import event_parser, BaseModel, ValidationError
7272
from aws_lambda_powertools.utilities.typing import LambdaContext
73+
7374
import json
7475

7576
class OrderItem(BaseModel):
@@ -152,101 +153,6 @@ def my_function():
152153
}
153154
```
154155

155-
### Data model validation
156-
157-
<Note type="warning">
158-
This is radically different from the <strong>Validator utility</strong> which validates events against JSON Schema.
159-
</Note><br/>
160-
161-
You can use parser's validator for deep inspection of object values and complex relationships.
162-
163-
There are two types of class method decorators you can use:
164-
165-
* **`validator`** - Useful to quickly validate an individual field and its value
166-
* **`root_validator`** - Useful to validate the entire model's data
167-
168-
Keep the following in mind regardless of which decorator you end up using it:
169-
170-
* You must raise either `ValueError`, `TypeError`, or `AssertionError` when value is not compliant
171-
* You must return the value(s) itself if compliant
172-
173-
#### Validating fields
174-
175-
Quick validation to verify whether the field `message` has the value of `hello world`.
176-
177-
```python:title=deep_data_validation.py
178-
from aws_lambda_powertools.utilities.parser import parse, BaseModel, validator
179-
180-
class HelloWorldModel(BaseModel):
181-
message: str
182-
183-
@validator('message') # highlight-line
184-
def is_hello_world(cls, v):
185-
if v != "hello world":
186-
raise ValueError("Message must be hello world!")
187-
return v
188-
189-
parse(model=HelloWorldModel, event={"message": "hello universe"})
190-
```
191-
192-
If you run as-is, you should expect the following error with the message we provided in our exception:
193-
194-
```
195-
message
196-
Message must be hello world! (type=value_error)
197-
```
198-
199-
Alternatively, you can pass `'*'` as an argument for the decorator so that you can validate every value available.
200-
201-
```python:title=validate_all_field_values.py
202-
from aws_lambda_powertools.utilities.parser import parse, BaseModel, validator
203-
204-
class HelloWorldModel(BaseModel):
205-
message: str
206-
sender: str
207-
208-
@validator('*') # highlight-line
209-
def has_whitespace(cls, v):
210-
if ' ' not in v:
211-
raise ValueError("Must have whitespace...")
212-
213-
return v
214-
215-
parse(model=HelloWorldModel, event={"message": "hello universe", "sender": "universe"})
216-
```
217-
218-
#### Validating entire model
219-
220-
`root_validator` can help when you have a complex validation mechanism. For example finding whether data has been omitted, comparing field values, etc.
221-
222-
```python:title=validate_all_field_values.py
223-
from aws_lambda_powertools.utilities.parser import parse, BaseModel, validator
224-
225-
class UserModel(BaseModel):
226-
username: str
227-
password1: str
228-
password2: str
229-
230-
@root_validator
231-
def check_passwords_match(cls, values):
232-
pw1, pw2 = values.get('password1'), values.get('password2')
233-
if pw1 is not None and pw2 is not None and pw1 != pw2:
234-
raise ValueError('passwords do not match')
235-
return values
236-
237-
payload = {
238-
"username": "universe",
239-
"password1": "myp@ssword",
240-
"password2": "repeat password"
241-
}
242-
243-
parse(model=UserModel, event=payload)
244-
```
245-
246-
<Note type="info">
247-
You can read more about validating list items, reusing validators, validating raw inputs, and a lot more in <a href="https://pydantic-docs.helpmanual.io/usage/validators/">Pydantic's documentation</a>.
248-
</Note><br/>
249-
250156
## Extending built-in models
251157

252158
Parser comes with the following built-in models:
@@ -324,12 +230,22 @@ for order_item in ret.detail.items:
324230

325231
## Envelopes
326232

327-
Envelope parameter is useful when your actual payload is wrapped around a known structure, for example Lambda Event Sources like EventBridge.
233+
When trying to parse your payloads wrapped in a known structure, you might encounter the following situations:
234+
235+
* Your actual payload is wrapped around a known structure, for example Lambda Event Sources like EventBridge
236+
* You're only interested in a portion of the payload, for example parsing the `detail` of custom events in EventBridge, or `body` of SQS records
237+
238+
You can either solve these situations by creating a model of these known structures, parsing them, then extracting and parsing a key where your payload is.
328239

329-
Example of parsing a model found in an event coming from EventBridge, where all you want is what's inside the `detail` key.
240+
This can become difficult quite quickly. Parser makes this problem easier through a feature named `Envelope`.
241+
242+
Envelopes can be used via `envelope` parameter available in both `parse` function and `event_parser` decorator.
243+
244+
Here's an example of parsing a model found in an event coming from EventBridge, where all you want is what's inside the `detail` key.
330245

331246
```python:title=parse_eventbridge_payload.py
332-
from aws_lambda_powertools.utilities.parser import parse, BaseModel, envelopes
247+
from aws_lambda_powertools.utilities.parser import event_parser, parse, BaseModel, envelopes
248+
from aws_lambda_powertools.utilities.typing import LambdaContext
333249

334250
class UserModel(BaseModel):
335251
username: str
@@ -358,6 +274,11 @@ ret = parse(model=UserModel, envelope=envelopes.EventBridgeModel, event=payload)
358274

359275
# Parsed model only contains our actual model, not the entire EventBridge + Payload parsed
360276
assert ret.password1 == ret.password2
277+
278+
# Same behaviour but using our decorator
279+
@event_parser(model=UserModel, envelope=envelopes.EventBridgeModel) # highlight-line
280+
def handler(event: UserModel, context: LambdaContext):
281+
assert event.password1 == event.password2
361282
```
362283

363284
**What's going on here, you might ask**:
@@ -367,17 +288,18 @@ assert ret.password1 == ret.password2
367288
3. Parser parsed the original event against the EventBridge model
368289
4. Parser then parsed the `detail` key using `UserModel`
369290

291+
370292
### Built-in envelopes
371293

372-
Parser comes with the following built-in envelopes, where `BaseModel` in the return section is your given model.
294+
Parser comes with the following built-in envelopes, where `Model` in the return section is your given model.
373295

374296
Envelope name | Behaviour | Return
375297
------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------
376-
**DynamoDBStreamEnvelope** | 1. Parses data using `DynamoDBStreamModel`. <br/> 2. Parses records in `NewImage` and `OldImage` keys using your model. <br/> 3. Returns a list with a dictionary containing `NewImage` and `OldImage` keys | `List[Dict[Literal["NewImage", "OldImage"], BaseModel]]`
377-
**EventBridgeEnvelope** | 1. Parses data using `EventBridgeModel`. <br/> 2. Parses `detail` key using your model and returns it. | `BaseModel`
378-
**SqsEnvelope** | 1. Parses data using `SqsModel`. <br/> 2. Parses records in `body` key using your model and return them in a list. | `List[BaseModel]`
298+
**DynamoDBStreamEnvelope** | 1. Parses data using `DynamoDBStreamModel`. <br/> 2. Parses records in `NewImage` and `OldImage` keys using your model. <br/> 3. Returns a list with a dictionary containing `NewImage` and `OldImage` keys | `List[Dict[str, Optional[Model]]]`
299+
**EventBridgeEnvelope** | 1. Parses data using `EventBridgeModel`. <br/> 2. Parses `detail` key using your model and returns it. | `Model`
300+
**SqsEnvelope** | 1. Parses data using `SqsModel`. <br/> 2. Parses records in `body` key using your model and return them in a list. | `List[Model]`
379301

380-
### Bringing your own envelope model
302+
### Bringing your own envelope
381303

382304
You can create your own Envelope model and logic by inheriting from `BaseEnvelope`, and implementing the `parse` method.
383305

@@ -407,28 +329,31 @@ class EventBridgeModel(BaseModel):
407329
**EventBridge Envelope**
408330

409331
```python:title=eventbridge_envelope.py
410-
from aws_lambda_powertools.utilities.parser import BaseEnvelope, BaseModel
411-
from typing import Any, Dict
412-
from ..models import EventBridgeModel
332+
from aws_lambda_powertools.utilities.parser import BaseEnvelope, models
333+
from aws_lambda_powertools.utilities.parser.models import EventBridgeModel
334+
335+
from typing import Any, Dict, Optional, TypeVar
336+
337+
Model = TypeVar("Model", bound=BaseModel)
413338

414339
class EventBridgeEnvelope(BaseEnvelope): # highlight-line
415340

416-
def parse(self, data: Dict[str, Any], model: BaseModel) -> BaseModel: # highlight-line
341+
def parse(self, data: Optional[Union[Dict[str, Any], Any]], model: Model) -> Optional[Model]: # highlight-line
417342
"""Parses data found with model provided
418343
419344
Parameters
420345
----------
421346
data : Dict
422347
Lambda event to be parsed
423-
model : BaseModel
348+
model : Model
424349
Data model provided to parse after extracting data using envelope
425350
426351
Returns
427352
-------
428353
Any
429354
Parsed detail payload with model provided
430355
"""
431-
parsed_envelope = EventBridgeModel(**data) # highlight-line
356+
parsed_envelope = EventBridgeModel.parse_obj(data) # highlight-line
432357
return self._parse(data=parsed_envelope.detail, model=model) # highlight-line
433358
```
434359

@@ -439,6 +364,102 @@ class EventBridgeEnvelope(BaseEnvelope): # highlight-line
439364
3. Then, we parsed the incoming data with our envelope to confirm it matches EventBridge's structure defined in `EventBridgeModel`
440365
4. Lastly, we call `_parse` from `BaseEnvelope` to parse the data in our envelope (.detail) using the customer model
441366

367+
### Data model validation
368+
369+
<Note type="warning">
370+
This is radically different from the <strong>Validator utility</strong> which validates events against JSON Schema.
371+
</Note><br/>
372+
373+
You can use parser's validator for deep inspection of object values and complex relationships.
374+
375+
There are two types of class method decorators you can use:
376+
377+
* **`validator`** - Useful to quickly validate an individual field and its value
378+
* **`root_validator`** - Useful to validate the entire model's data
379+
380+
Keep the following in mind regardless of which decorator you end up using it:
381+
382+
* You must raise either `ValueError`, `TypeError`, or `AssertionError` when value is not compliant
383+
* You must return the value(s) itself if compliant
384+
385+
#### Validating fields
386+
387+
Quick validation to verify whether the field `message` has the value of `hello world`.
388+
389+
```python:title=deep_data_validation.py
390+
from aws_lambda_powertools.utilities.parser import parse, BaseModel, validator
391+
392+
class HelloWorldModel(BaseModel):
393+
message: str
394+
395+
@validator('message') # highlight-line
396+
def is_hello_world(cls, v):
397+
if v != "hello world":
398+
raise ValueError("Message must be hello world!")
399+
return v
400+
401+
parse(model=HelloWorldModel, event={"message": "hello universe"})
402+
```
403+
404+
If you run as-is, you should expect the following error with the message we provided in our exception:
405+
406+
```
407+
message
408+
Message must be hello world! (type=value_error)
409+
```
410+
411+
Alternatively, you can pass `'*'` as an argument for the decorator so that you can validate every value available.
412+
413+
```python:title=validate_all_field_values.py
414+
from aws_lambda_powertools.utilities.parser import parse, BaseModel, validator
415+
416+
class HelloWorldModel(BaseModel):
417+
message: str
418+
sender: str
419+
420+
@validator('*') # highlight-line
421+
def has_whitespace(cls, v):
422+
if ' ' not in v:
423+
raise ValueError("Must have whitespace...")
424+
425+
return v
426+
427+
parse(model=HelloWorldModel, event={"message": "hello universe", "sender": "universe"})
428+
```
429+
430+
#### Validating entire model
431+
432+
`root_validator` can help when you have a complex validation mechanism. For example finding whether data has been omitted, comparing field values, etc.
433+
434+
```python:title=validate_all_field_values.py
435+
from aws_lambda_powertools.utilities.parser import parse, BaseModel, validator
436+
437+
class UserModel(BaseModel):
438+
username: str
439+
password1: str
440+
password2: str
441+
442+
@root_validator
443+
def check_passwords_match(cls, values):
444+
pw1, pw2 = values.get('password1'), values.get('password2')
445+
if pw1 is not None and pw2 is not None and pw1 != pw2:
446+
raise ValueError('passwords do not match')
447+
return values
448+
449+
payload = {
450+
"username": "universe",
451+
"password1": "myp@ssword",
452+
"password2": "repeat password"
453+
}
454+
455+
parse(model=UserModel, event=payload)
456+
```
457+
458+
<Note type="info">
459+
You can read more about validating list items, reusing validators, validating raw inputs, and a lot more in <a href="https://pydantic-docs.helpmanual.io/usage/validators/">Pydantic's documentation</a>.
460+
</Note><br/>
461+
462+
442463
## FAQ
443464

444465
**When should I use parser vs data_classes utility?**

0 commit comments

Comments
 (0)