Retain Order in DelayHandler [INT-3643]

[spring-operator]

Gary Russell opened INT-3643 and commented

Each delayed message is scheduled separately. Order can be lost.

It would be useful if messages arriving on the same thread could retain the same order (run on the same release thread) as arrival.

---

No further details from INT-3643

[spring-operator]

Artem Bilan commented

Gary, would you mind to share more context or thoughts?

I just some how feel like this request is fully unrelated to the delayer purpose.

1. We schedule message to the TaskScheduler - there is no guarantee that message with lower delay will really be emitted earlier than others. Because of thread pool. At least with short difference between delays.
2. That is exactly a goal of delayer to try to release messages with lower delay earlier independently of their origin.

In other words I definitely would like to receive quicker message earlier than long-delayed.
Even if we would have some queues to shift to release threads, that doesn't mean that message should reach the queue in the arrival order.

What am I missing?
Thanks

[spring-operator]

Gary Russell commented

It has nothing to do with messages with different delays, or messages arriving on different threads; the issue is with a fixed delay; millisecond is not enough resolution.

If two messages arrive in the same millisecond, on the same thread, they should be released in the same sequence.

[spring-operator]

Artem Bilan commented

I wonder if there would be enough to rely on the injected ConcurrentTaskScheduler capabilities.
Looks like we can supply ConcurrentTaskScheduler with any ScheduledExecutorService implementation including Reactor's ColocatedEventLoopGroup, which, in turn, may use Netty's DefaultEventLoop.

[spring-operator]

Gary Russell commented

That won't help after restart; the order will be lost again after message store retrieval; I think we'll need some kind of incrementing integer to apply to all messages, to qualify the timestamp.

[spring-operator]

Artem Bilan commented

Do you mean something similar what we have in the INT_CHANNEL_MESSAGE with the:

MESSAGE_SEQUENCE NUMBER(19,0) NOT NULL ,

but in this case at the INT_MESSAGE_GROUP table level?

[spring-operator]

Gary Russell commented

Yes; something like that - so that messages with the same timestamp are retrieved in order.

[spring-operator]

Artem Bilan commented

Good. Shound't we rename then INT_CHANNEL_MESSAGE to something like INT_SIMPLE_MESSAGE_GROUP and all the respective classes and use them, finally, in the DelayHandler?

5.2 theme then?

[spring-operator]

Gary Russell commented

Perhaps a new table indexed by RELEASE_DATE and SEQUENCE.

Yes, 5.2; I have renamed the backlog.