Last message when streaming completion should not be empty

[abetlen]

When streaming completions the last message is sometimes empty to ensure that the last message contains a finish_reason. This is correct according to the OpenAI schema but is not expected by all clients and can cause bugs (noticed this in openplayground).

[BlueBlazin]

So what's the alternative?

[thiner]

I encountered this problem with the latest docker image for inferencing. When streeaming via /chat/completions API, the stream will be interrupted. The output obviously not complete yet. Call the API with the same input will not encounter this error.