Skip to content

Commit 3d5629e

Browse files
committed
remove auto-format which breaks doc and add mermaid diagram
1 parent d66e1ca commit 3d5629e

File tree

1 file changed

+77
-19
lines changed

1 file changed

+77
-19
lines changed

docs/utilities/large_messages.md

Lines changed: 77 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,10 @@ The large message utility handles SQS and SNS messages which have had their payl
77
offloaded to S3 if they are larger than the maximum allowed size (256 KB).
88

99
!!! Notice
10-
The large message utility (available in the `powertools-batch` module with v1.16.1 or lower) is now deprecated
11-
and replaced by the `powertools-large-messages` described in this page.
12-
You can still get the documentation [here](sqs_large_message_handling.md)
13-
and the migration guide [here](#migration-from-the-sqs-large-message-utility).
10+
The large message utility (available in the `powertools-batch` module with v1.16.1 or lower) is now deprecated
11+
and replaced by the `powertools-large-messages` described in this page.
12+
You can still get the documentation [here](sqs_large_message_handling.md)
13+
and the migration guide [here](#migration-from-the-sqs-large-message-utility).
1414

1515
## Features
1616

@@ -20,6 +20,64 @@ and the migration guide [here](#migration-from-the-sqs-large-message-utility).
2020

2121
## Background
2222

23+
```mermaid
24+
stateDiagram-v2
25+
direction LR
26+
Function : Lambda Function
27+
28+
state Application {
29+
direction TB
30+
sendMsg: sendMessage(QueueUrl, MessageBody)
31+
extendLib: extended-client-lib
32+
[*] --> sendMsg
33+
sendMsg --> extendLib
34+
state extendLib {
35+
state if_big <<choice>>
36+
bigMsg: MessageBody > 256KB ?
37+
putObject: putObject(S3Bucket, S3Key, Body)
38+
updateMsg: Update MessageBody<br>with a pointer to S3<br>and add a message attribute
39+
bigMsg --> if_big
40+
if_big --> [*]: size(body) <= 256kb
41+
if_big --> putObject: size(body) > 256kb
42+
putObject --> updateMsg
43+
updateMsg --> [*]
44+
}
45+
}
46+
47+
state Function {
48+
direction TB
49+
iterateMsgs: Iterate over messages
50+
ptLargeMsg: powertools-large-messages
51+
[*] --> Handler
52+
Handler --> iterateMsgs
53+
iterateMsgs --> ptLargeMsg
54+
state ptLargeMsg {
55+
state if_pointer <<choice>>
56+
pointer: Message attribute <br>for large message ?
57+
normalMsg: Small message,<br>body left unchanged
58+
getObject: getObject(S3Pointer)
59+
deleteObject: deleteObject(S3Pointer)
60+
updateBody: Update message body<br>with content from S3 object<br>and remove message attribute
61+
updateMD5: Update MD5 of the body<br>and attributes (SQS only)
62+
yourcode: <b>YOUR CODE HERE!</b>
63+
pointer --> if_pointer
64+
if_pointer --> normalMsg : False
65+
normalMsg --> [*]
66+
if_pointer --> getObject : True
67+
getObject --> updateBody
68+
updateBody --> updateMD5
69+
updateMD5 --> yourcode
70+
yourcode --> deleteObject
71+
deleteObject --> [*]
72+
}
73+
}
74+
75+
[*] --> Application
76+
Application --> Function : Lambda Invocation
77+
Function --> [*]
78+
79+
```
80+
2381
SQS and SNS message payload is limited to 256KB. If you wish to send larger message payload, you can leverage the
2482
[amazon-sqs-java-extended-client-lib](https://github.com/awslabs/amazon-sqs-java-extended-client-lib)
2583
or [amazon-sns-java-extended-client-lib](https://github.com/awslabs/amazon-sns-java-extended-client-lib) which
@@ -242,15 +300,15 @@ After your code is invoked and returns without error, the object is deleted from
242300
using the `deleteObject(bucket, key)` API. You can disable the deletion of S3 objects with the following configuration:
243301

244302
=== "Don't delete S3 Objects"
245-
```java
246-
@LargeMessage(deleteS3Object = false)
247-
private void processRawMessage(SQSEvent.SQSMessage sqsMessage) {
248-
// do something with the message
249-
}
250-
```
303+
```java
304+
@LargeMessage(deleteS3Object = false)
305+
private void processRawMessage(SQSEvent.SQSMessage sqsMessage) {
306+
// do something with the message
307+
}
308+
```
251309

252310
!!! tip
253-
This utility works perfectly together with the batch module (`powertools-batch`), especially for SQS:
311+
This utility works perfectly together with the batch module (`powertools-batch`), especially for SQS:
254312

255313
```java hl_lines="2 5-7 12 15 16" title="Combining batch and large message modules"
256314
public class SqsBatchHandler implements RequestHandler<SQSEvent, SQSBatchResponse> {
@@ -279,18 +337,18 @@ This utility works perfectly together with the batch module (`powertools-batch`)
279337
To interact with S3, the utility creates a default S3 Client :
280338

281339
=== "Default S3 Client"
282-
```java
283-
S3Client client = S3Client.builder()
284-
.httpClient(UrlConnectionHttpClient.builder().build())
285-
.region(Region.of(System.getenv(AWS_REGION_ENV)))
286-
.build();
287-
```
340+
```java
341+
S3Client client = S3Client.builder()
342+
.httpClient(UrlConnectionHttpClient.builder().build())
343+
.region(Region.of(System.getenv(AWS_REGION_ENV)))
344+
.build();
345+
```
288346

289347
If you need to customize this `S3Client`, you can leverage the `LargeMessageConfig` singleton:
290348

291349
=== "Custom S3 Client"
292-
```java hl_lines="6"
293-
import software.amazon.lambda.powertools.largemessages.LargeMessage;
350+
```java hl_lines="6"
351+
import software.amazon.lambda.powertools.largemessages.LargeMessage;
294352

295353
public class SnsRecordHandler implements RequestHandler<SNSEvent, String> {
296354

0 commit comments

Comments
 (0)