-
Notifications
You must be signed in to change notification settings - Fork 90
module validation #138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
module validation #138
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
72d8eac
module validation
jeromevdl 231293f
add licence headers
jeromevdl a2d7145
doc
jeromevdl b0821e5
improve exception messages
jeromevdl 81f4c7d
sample
jeromevdl 94010a8
review doc
jeromevdl 04dcd60
complete doc
jeromevdl File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,331 @@ | ||
--- | ||
title: Validation | ||
description: Utility | ||
--- | ||
|
||
import Note from "../../src/components/Note" | ||
|
||
This utility provides JSON Schema validation for payloads held within events and response used in AWS Lambda. | ||
|
||
**Key features** | ||
* Validate incoming events and responses | ||
* Built-in validation for most common events (API Gateway, SNS, SQS, ...) | ||
* JMESPath support validate only a sub part of the event | ||
|
||
## Install | ||
|
||
To install this utility, add the following dependency to your project. | ||
|
||
```xml | ||
<dependency> | ||
<groupId>software.amazon.lambda</groupId> | ||
<artifactId>powertools-validation</artifactId> | ||
<version>0.5.0-beta</version> | ||
</dependency> | ||
``` | ||
|
||
And configure the aspectj-maven-plugin to compile-time weave (CTW) the | ||
aws-lambda-powertools-java aspects into your project. You may already have this | ||
plugin in your pom. In that case add the dependency to the `aspectLibraries` | ||
section. | ||
|
||
```xml | ||
<build> | ||
<plugins> | ||
... | ||
<plugin> | ||
<groupId>org.codehaus.mojo</groupId> | ||
<artifactId>aspectj-maven-plugin</artifactId> | ||
<version>1.11</version> | ||
<configuration> | ||
<source>1.8</source> | ||
<target>1.8</target> | ||
<complianceLevel>1.8</complianceLevel> | ||
<aspectLibraries> | ||
<!-- highlight-start --> | ||
<aspectLibrary> | ||
<groupId>software.amazon.lambda</groupId> | ||
<artifactId>powertools-validation</artifactId> | ||
</aspectLibrary> | ||
<!-- highlight-end --> | ||
</aspectLibraries> | ||
</configuration> | ||
<executions> | ||
<execution> | ||
<goals> | ||
<goal>compile</goal> | ||
</goals> | ||
</execution> | ||
</executions> | ||
</plugin> | ||
... | ||
</plugins> | ||
</build> | ||
``` | ||
|
||
## Validating events | ||
|
||
You can validate inbound and outbound events using `@Validation` annotation. | ||
|
||
You can also use the `Validator#validate()` methods, if you want more control over the validation process such as handling a validation error. | ||
|
||
We support JSON schema version 4, 6, 7 and 201909 (from [jmespath-jackson library](https://github.com/burtcorp/jmespath-java)). | ||
|
||
### @Validation annotation | ||
|
||
`@Validation` annotation is used to validate either inbound events or functions' response. | ||
|
||
It will fail fast with `ValidationException` if an event or response doesn't conform with given JSON Schema. | ||
|
||
While it is easier to specify a json schema file in the classpath (using the notation `"classpath:/path/to/schema.json"`), you can also provide a JSON String containing the schema. | ||
|
||
```java | ||
public class MyFunctionHandler implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> { | ||
|
||
@Override | ||
@Validation(inboundSchema = "classpath:/schema_in.json", outboundSchema = "classpath:/schema_out.json") | ||
public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { | ||
// ... | ||
return something; | ||
} | ||
} | ||
``` | ||
|
||
**NOTE**: It's not a requirement to validate both inbound and outbound schemas - You can either use one, or both. | ||
msailes marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Validate function | ||
|
||
Validate standalone function is used within the Lambda handler, or any other methods that perform data validation. | ||
|
||
You can also gracefully handle schema validation errors by catching `ValidationException`. | ||
|
||
```java | ||
import static software.amazon.lambda.powertools.validation.Validator.*; | ||
|
||
public class MyFunctionHandler implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> { | ||
|
||
@Override | ||
public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { | ||
try { | ||
validate(input, "classpath:/schema.json"); | ||
} catch (ValidationException ex) { | ||
// do something before throwing it | ||
throw ex; | ||
} | ||
|
||
// ... | ||
return something; | ||
} | ||
} | ||
``` | ||
**NOTE**: Schemas are stored in memory for reuse, to avoid loading them from file each time. | ||
msailes marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Built-in events and responses | ||
|
||
For the following events and responses, the Validator will automatically perform validation on the content. | ||
|
||
** Events ** | ||
|
||
Type of event | Class | Path to content | | ||
------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- | ||
API Gateway REST | APIGatewayProxyRequestEvent | `body` | ||
API Gateway HTTP | APIGatewayV2HTTPEvent | `body` | ||
Application Load Balancer | ApplicationLoadBalancerRequestEvent | `body` | ||
Cloudformation Custom Resource | CloudFormationCustomResourceEvent | `resourceProperties` | ||
CloudWatch Logs | CloudWatchLogsEvent | `awslogs.powertools_base64_gzip(data)` | ||
EventBridge / Cloudwatch | ScheduledEvent | `detail` | ||
Kafka | KafkaEvent | `records[*][*].value` | ||
Kinesis | KinesisEvent | `Records[*].kinesis.powertools_base64(data)` | ||
Kinesis Firehose | KinesisFirehoseEvent | `Records[*].powertools_base64(data)` | ||
Kinesis Analytics from Firehose | KinesisAnalyticsFirehoseInputPreprocessingEvent | `Records[*].powertools_base64(data)` | ||
Kinesis Analytics from Streams | KinesisAnalyticsStreamsInputPreprocessingEvent | `Records[*].powertools_base64(data)` | ||
SNS | SNSEvent | `Records[*].Sns.Message` | ||
SQS | SQSEvent | `Records[*].body` | ||
|
||
** Responses ** | ||
|
||
Type of response | Class | Path to content (envelope) | ||
------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- | ||
API Gateway REST | APIGatewayProxyResponseEvent} | `body` | ||
API Gateway HTTP | APIGatewayV2HTTPResponse} | `body` | ||
API Gateway WebSocket | APIGatewayV2WebSocketResponse} | `body` | ||
Load Balancer | ApplicationLoadBalancerResponseEvent} | `body` | ||
Kinesis Analytics | KinesisAnalyticsInputPreprocessingResponse} | `Records[*].powertools_base64(data)`` | ||
|
||
## Custom events and responses | ||
|
||
You can also validate any Event or Response type, once you have the appropriate schema. | ||
|
||
Sometimes, you might want to validate only a portion of it - This is where the envelope parameter is for. | ||
|
||
Envelopes are [JMESPath expressions](https://jmespath.org/tutorial.html) to extract a portion of JSON you want before applying JSON Schema validation. | ||
|
||
Here is a custom event where we only want to validate each products: | ||
|
||
```json | ||
{ | ||
"basket": { | ||
"products" : [ | ||
{ | ||
"id": 43242, | ||
"name": "FooBar XY", | ||
"price": 258 | ||
}, | ||
{ | ||
"id": 765, | ||
"name": "BarBaz AB", | ||
"price": 43.99 | ||
} | ||
] | ||
} | ||
} | ||
``` | ||
|
||
Here is how you'd use the `envelope` parameter to extract the payload inside the products key before validating: | ||
|
||
```java | ||
public class MyCustomEventHandler implements RequestHandler<MyCustomEvent, String> { | ||
|
||
@Override | ||
@Validation(inboundSchema = "classpath:/my_custom_event_schema.json", | ||
envelope = "basket.products[*]") | ||
public String handleRequest(MyCustomEvent input, Context context) { | ||
return "OK"; | ||
} | ||
} | ||
``` | ||
|
||
This is quite powerful because you can use JMESPath Query language to extract records from | ||
[arrays, slice and dice](https://jmespath.org/tutorial.html#list-and-slice-projections), | ||
to [pipe expressions](https://jmespath.org/tutorial.html#pipe-expressions) | ||
and [function](https://jmespath.org/tutorial.html#functions) expressions, where you'd extract what you need before validating the actual payload. | ||
|
||
## JMESPath functions | ||
|
||
JMESPath functions ensure to make an operation on a specific part of the json.validate | ||
|
||
Powertools provides two built-in functions: | ||
|
||
### powertools_base64 function | ||
|
||
Use `powertools_base64` function to decode any base64 data. | ||
|
||
This sample will decode the base64 value within the data key, and decode the JSON string into a valid JSON before we can validate it: | ||
|
||
```json | ||
{ | ||
"data" : "ewogICJpZCI6IDQzMjQyLAogICJuYW1lIjogIkZvb0JhciBYWSIsCiAgInByaWNlIjogMjU4Cn0=" | ||
} | ||
``` | ||
|
||
```java | ||
public class MyEventHandler implements RequestHandler<MyEvent, String> { | ||
|
||
@Override | ||
public String handleRequest(MyEvent myEvent, Context context) { | ||
validate(myEvent, "classpath:/schema.json", "powertools_base64(data)"); | ||
return "OK"; | ||
} | ||
} | ||
``` | ||
|
||
### powertools_base64_gzip function | ||
|
||
Use `powertools_base64_gzip` function to decompress and decode base64 data. | ||
|
||
This sample will decompress and decode base64 data: | ||
|
||
```json | ||
{ | ||
"data" : "H4sIAAAAAAAA/6vmUlBQykxRslIwMTYyMdIBcfMSc1OBAkpu+flOiUUKEZFKYOGCosxkkLiRqQVXLQDnWo6bOAAAAA==" | ||
} | ||
``` | ||
|
||
```java | ||
public class MyEventHandler implements RequestHandler<MyEvent, String> { | ||
|
||
@Override | ||
public String handleRequest(MyEvent myEvent, Context context) { | ||
validate(myEvent, "classpath:/schema.json", "powertools_base64_gzip(data)"); | ||
return "OK"; | ||
} | ||
} | ||
``` | ||
|
||
**NOTE:** You don't need any function to transform a JSON String into a JSON object, powertools-validation will do it for you. | ||
In the 2 previous example, data contains JSON. Just provide the function to transform the base64 / gzipped / ... string into a clear JSON string. | ||
|
||
### Bring your own JMESPath function | ||
|
||
<Note type="warning"> | ||
This should only be used for advanced use cases where you have special formats not covered by the built-in functions. | ||
New functions will be added to the 2 built-in ones. | ||
</Note> | ||
|
||
|
||
Your function must extend `io.burt.jmespath.function.BaseFunction`, take a String as parameter and return a String. | ||
You can read the [doc](https://github.com/burtcorp/jmespath-java#adding-custom-functions) for more information. | ||
|
||
Here is an example that takes some xml and transform it into json: | ||
```java | ||
public class XMLFunction extends BaseFunction { | ||
public Base64Function() { | ||
super("powertools_xml", ArgumentConstraints.typeOf(JmesPathType.STRING)); | ||
} | ||
|
||
@Override | ||
protected <T> T callFunction(Adapter<T> runtime, List<FunctionArgument<T>> arguments) { | ||
T value = arguments.get(0).value(); | ||
String xmlString = runtime.toString(value); | ||
|
||
String jsonString = // ... transform xmlString to json | ||
|
||
return runtime.createString(jsonString); | ||
} | ||
} | ||
``` | ||
|
||
Once your function is created, you need to add it to powertools: | ||
|
||
```java | ||
ValidatorConfig.get().addFunction(new XMLFunction()); | ||
``` | ||
|
||
You can then use it to do your validation: | ||
```java | ||
public class MyXMLEventHandler implements RequestHandler<MyEventWithXML, String> { | ||
|
||
@Override | ||
public String handleRequest(MyEventWithXML myEvent, Context context) { | ||
validate(myEvent, "classpath:/schema.json", "powertools_xml(path.to.xml_data)"); | ||
return "OK"; | ||
} | ||
} | ||
``` | ||
or using annotation: | ||
```java | ||
public class MyXMLEventHandler implements RequestHandler<MyEventWithXML, String> { | ||
|
||
@Override | ||
@Validation(inboundSchema="classpath:/schema.json", envelope="powertools_xml(path.to.xml_data)") | ||
public String handleRequest(MyEventWithXML myEvent, Context context) { | ||
return "OK"; | ||
} | ||
} | ||
``` | ||
|
||
## Change the schema version | ||
By default, powertools-validation is configured with [V7](https://json-schema.org/draft-07/json-schema-release-notes.html). | ||
You can use the `ValidatorConfig` to change that behaviour: | ||
|
||
```java | ||
ValidatorConfig.get().setSchemaVersion(SpecVersion.VersionFlag.V4); | ||
``` | ||
|
||
## Advanced ObjectMapper settings | ||
If you need to configure the Jackson ObjectMapper, you can use the `ValidatorConfig`: | ||
|
||
```java | ||
ObjectMapper objectMapper= ValidatorConfig.get().getObjectMapper(); | ||
msailes marked this conversation as resolved.
Show resolved
Hide resolved
|
||
// update (de)serializationConfig or other properties | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.