support Iterator for splitters [INT-651]

[spring-operator]

Andrew Skiba opened INT-651 and commented

Support methods returning Iterator for splitters:

@Splitter
Iterator<Item> split(Order order){..}

This should probably include

Iterator<Message> split(Message input)

for controlling headers, correlation ids, etc.

---

Reference URL: http://forum.springsource.org/showthread.php?p=239259#post239259

Attachments:

• SI-patch-int-651.txt.zip (6.00 kB)

Issue Links:

• Allow Aggregator complete groups based on timeout [INT-3420] #7390 Allow Aggregator complete groups based on timeout

Referenced from: pull request #1202

2 votes, 3 watchers

[spring-operator]

Iwein Fuld commented

Both Iterator and Iterable should be supported. This gives the most flexibility.

[spring-operator]

Alex Peters commented

i followed the discussion in the forum and i think the iterator would be very convenient but i guess will not work with the AbstractMessageSplitter. Andrew intends to lazy load the rows but behind the iterator any impl has to do the same as building a List<String> or List<MessageBuilder> to determining the full sequenceSize. A Future<Integer> would be nice but it is currently hard bounded to Integer in the MessageBuilder and other places.... any ideas?

[spring-operator]

Andrew Skiba commented

Alex, I think you are right. But since SEQUENCE_SIZE is put for AbstractMessageAggregator and CompletionStrategy the need to be redesigned to support iterators. Currently they rely on the idea that the message list will be hold in memory. So there will be no real advantage if Future<Integer> is used for cases when the number of messages is huge.

Or they can be left intact for the usual cases when splitter produces a small number of messages, and if user uses an iterator to split message to large number of messages, he will either implement his own AbstractMessageBarrierHandler or not aggregate messages at all. So I propose to put SEQUENCE_SIZE when a splitter returns a List and to omit this header when the returned value is an Iterator.

As a side note, AbstractMessageBarrierHandler .sendReplies should accept Iterable<Message> instead of Collection<Message>, because it only iterates on them.

Lots of changes :-(

[spring-operator]

Alex Peters commented

Andrew, good points. the sequenceSize can be embedded in the last message of a group so that any custom aggregator may use it. i prototyped a streaming splitter and will submit a patch.

[spring-operator]

Alex Peters commented

just uploaded a patch with an AbstractStreamingMessageSplitter class for Iterator+ support. this could replace the AbstractMessageSplitter class.

added minor refactoring and a litte fix in AbstractMessageHandlerFactoryBean

[spring-operator]

Alex Peters commented

I'm going to fix NPE scenarios in Resequencer and SequenceSizeCompletionStrategy

[spring-operator]

Iwein Fuld commented

Good points. I agree and I'll go further. Aggregation of all the split messages will make less sense if you need an Iterator (presumably because holding the full collection in memory is too expensive).

It could be that the split messages will be filtered (reduced in size) before aggregation but still requiring a custom CompletionStrategy for this sounds perfectly reasonable to me.

[spring-operator]

Gary Russell commented

Merged and backported.