Throughput improvement to Stream::merge

[yoshuawuyts]

In #437 we fixed bugs to Stream::merge. There is another improvement we may want to apply here:

Randomization

If two items from Stream::merge are ready at the same time, we will always pick the item from the first stream first. We should probably apply some randomization here to make it so we don't exhaust one stream completely before starting the other. This should reduce overall latency in some cases, as it becomes more "fair".

We can use utils::random for this purpose.

[abhishekc-sharma]

If this were implemented what would be the correct way to handle the non-determinism for the purpose of tests ?

[Razican]

Hi I would like to work on this if nobody is working on it :)

[yoshuawuyts]

This has been merged as part of #503. Thanks @Razican!