Skip to content

Commit 8dd973e

Browse files
committed
docs(streaming): combine testing your code to be more realistic
1 parent 92db724 commit 8dd973e

File tree

5 files changed

+42
-83
lines changed

5 files changed

+42
-83
lines changed

docs/utilities/streaming.md

Lines changed: 7 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -131,32 +131,18 @@ You can build your own custom data transformation by extending the `BaseTransfor
131131

132132
## Testing your code
133133

134-
### Testing that you transformation is applied
135-
136-
Test that your transformation pipeline is returning the correct object:
137-
138-
=== "Testing the data pipeline returned object"
139-
140-
```python hl_lines="14 17"
141-
--8<-- "examples/streaming/src/test_s3_pipeline_result.py"
142-
```
143-
144-
### Testing that your transformation is working in isolation
134+
### Asserting data transformations
145135

146136
Create an input payload using `io.BytesIO` and assert the response of the transformation:
147137

148-
=== "Testing transformation in isolation"
138+
=== "assert_transformation.py"
149139

150-
```python hl_lines="23-25"
151-
--8<-- "examples/streaming/src/test_s3_transform_isolated.py"
140+
```python hl_lines="3 13 15 23-28 31-32"
141+
--8<-- "examples/streaming/src/assert_transformation.py"
152142
```
153143

154-
### Testing that your transformation is working with S3 data
155-
156-
Use `botocore.stub` to stub the `get_object` response from S3:
157-
158-
=== "Testing transformation with mocked S3 data"
144+
=== "assert_transformation_module.py"
159145

160-
```python hl_lines="32-34 37"
161-
--8<-- "examples/streaming/src/test_s3_transform_mocked.py"
146+
```python hl_lines="16"
147+
--8<-- "examples/streaming/src/assert_transformation_module.py"
162148
```
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import io
2+
3+
import boto3
4+
from assert_transformation_module import UpperTransform
5+
from botocore import stub
6+
7+
from aws_lambda_powertools.utilities.streaming import S3Object
8+
from aws_lambda_powertools.utilities.streaming.compat import PowertoolsStreamingBody
9+
10+
11+
def test_upper_transform():
12+
# GIVEN
13+
data_stream = io.BytesIO(b"hello world")
14+
# WHEN
15+
data_stream = UpperTransform().transform(data_stream)
16+
# THEN
17+
assert data_stream.read() == b"HELLO WORLD"
18+
19+
20+
def test_s3_object_with_upper_transform():
21+
# GIVEN
22+
payload = b"hello world"
23+
s3_client = boto3.client("s3")
24+
s3_stub = stub.Stubber(s3_client)
25+
s3_stub.add_response(
26+
"get_object", {"Body": PowertoolsStreamingBody(raw_stream=io.BytesIO(payload), content_length=len(payload))}
27+
)
28+
s3_stub.activate()
29+
30+
# WHEN
31+
data_stream = S3Object(bucket="bucket", key="key", boto3_client=s3_client)
32+
data_stream.transform(UpperTransform(), in_place=True)
33+
34+
# THEN
35+
assert data_stream.read() == b"HELLO WORLD"

examples/streaming/src/test_s3_transform_isolated.py renamed to examples/streaming/src/assert_transformation_module.py

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,3 @@ def read(self, size: int = -1) -> Optional[bytes]:
1717
class UpperTransform(BaseTransform):
1818
def transform(self, input_stream: IO[bytes]) -> UpperIO:
1919
return UpperIO(input_stream=input_stream, encoding="utf-8")
20-
21-
22-
def test_s3_pipeline_result():
23-
stream = io.BytesIO(b"hello world")
24-
stream = UpperTransform().transform(stream)
25-
assert stream.read() == b"HELLO WORLD"

examples/streaming/src/test_s3_pipeline_result.py

Lines changed: 0 additions & 17 deletions
This file was deleted.

examples/streaming/src/test_s3_transform_mocked.py

Lines changed: 0 additions & 39 deletions
This file was deleted.

0 commit comments

Comments
 (0)