Skip to content

Feature request: implement BasePartialBatchProcessor class #1586

Closed
@dreamorosi

Description

@dreamorosi

Use Case

The Python version of Powertools for AWS features a classes called BasePartialBatchProcessor and BatchProcessor, and its respective interfaces. These class is used to provide a Batch Processor utility in Powertools.

Solution/User Experience

Refer to the implementation found here, and specifically to the BasePartialBatchProcessor class and BasePartialProcessor interface.

Refer to the implementation found here, and specifically to the BatchProcessor class.

Acceptance criteria

  • The class covers all the features found in the reference implementation
  • The class and its type are exported
  • Unit tests are implemented and have 100% coverage
  • Classes and methods are documented via docstrings, following project's conventions as much as possible

Notes

The reference implementation also implements an AsyncBatchProcessor and both classes have _process_record and _async_process_record methods.

I'd like to dedicate some time investigating whether we can merge these two classes & methods into a single one.

The ideal DX would have the customer simply importing BatchProcessor and passing it a function reference when calling process_partial_response. The utility should be smart enough to understand whether the function passed is synchronous or asynchronous and handle it accordingly (i.e. if it's async, then await the record handler).

If this is not possible, or too complex / unreliable, or the resulting types are confusing or overly broad, then let's stick with two separate entities.

Metadata

Metadata

Assignees

Labels

batchThis item relates to the Batch Processing UtilitycompletedThis item is complete and has been merged/shippedfeature-requestThis item refers to a feature request for an existing or new utility

Type

No type

Projects

Status

Shipped

Relationships

None yet

Development

No branches or pull requests

Issue actions