Description
Expected Behavior
there is docs
Current Behavior
there is no docs (on this page at least)
Context
I'm investigating DLT and retry flows with spring-kafka however, I stumbled on the difficulties of using it in tandem with a batch listener.
I can see historically things have changed around this area which has left me a little confused.
Originally I had just assumed that underneath the listener, when a batch processing exception is thrown that it would either produce the whole batch to the -retry topic or the -dlt or it would do something like binary search on the batch to find bad messages and pausing the consumer until complete.
I then discovered BatchListenerFailedException where you need to find the bad messages yourself during processing and throw this exception but I'm still not clear on what will happen here without delving into the code, is the bad message published to the dlt? do we just commit the offset up to that message and continually retry? how can this be integrated with @DltHandler?
Ultimately, if the answers to the questions I have is, "it's not supported" then that's fine although a little disappointing, but it would be good to update the documentation found here.
I also think the documentation could be better around the handling of deserialization exceptions vs processing exceptions e.g. if the error happens during processing you need to have a serializer set up in a KafkaTemplate that's able to serialize the message OR with batch I currently don't understand what the ErrorHandlingDeserializer will do with my batch mode consumer.
As for batch consuming as a concept, it's far more efficient in my scenarios where I'm consuming from a topic then performing database operations etc.
This issue describes my pain somewhat but there's not been much activity on it.