[Messenger] Reconnect after connection was closed

[msubotovic]

Hi, i'm getting a lot of 'Library error: connection closed unexpectedly' errors when using RabbitMQ cluster with HAProxy in front.
The problem appears when handler is dispatching messages to another handler. The handler opens connection to dispatch a message, but if it stays iddle for
50 sec., this connection is automatically closed.
Is it possible to renew the connection every time the handler is dispatching a message?

[Nek-]

Hello,

First, you may configure your HA Proxy another way to make it work. But there's also some things that are not documented at all in the Symfony documentation: you can add options to your amqp connection. And here is a list of options:

https://github.com/php-amqp/php-amqp/blob/96cd5cb5eddd3db2faaa3643dad2fe4677d7c438/stubs/AMQPConnection.php#L27-L53

You may be interested in the heartbeat option :) .

This is related to : symfony/symfony-docs#11792

[jkobus]

Messenger's AmqpReceiver does not use blocking calls, therefore heartbeat option is useless.
See https://github.com/symfony/amqp-messenger/blob/5.x/Transport/AmqpReceiver.php#L52

[mleczakm]

I'm hitting exactly same problem, but during publishing messages. Probably ability to reset channel manually would allow to fix it, I'm gonna check it. In fact, whole https://github.com/symfony/amqp-messenger/blob/a7f681b022f9cfb88febc12face05b12a47232c1/Transport/Connection.php#L551 clearWhenDisconnected() method is useless, since checking isConnected() will always return true.

[carsonbot]

Thank you for this issue.
There has not been a lot of activity here for a while. Has this been resolved?

[carsonbot]

Could I get a reply or should I close this?

[jkobus]

@mleczakm were you able to solve this issue somehow?

@jkobus I've overridden transport IIRC and introduce custom callback to be called on every message handled and reset the connection. It is definitely not the fastest way to handle it. It probably should be handled otherwise (on message loop events, time-based like worker soft-kill) and still clearWhenDisconnected() does nothing :(

[jkobus]

IMHO the solution would be to catch this exception (and other, connection related errors like this one) inside the \Symfony\Component\Messenger\Transport\AmqpExt\Connection and use internal flag (for example, (bool) reconnect) and not to rely only on AmqpChannel::isConnected.

One could then catch this exception and decide whether to retry the operation. During retry, clearWhenDisconnected() would check whether reconnect is true aside from what AMQPChannel::isConnected says and if it is, it would create a new connection.

The one thing that worries me is that some connection problems are thrown as \AMQPException rather than \AMQPConnectionException.

Guys, let me know whether this would make sense?

[carsonbot]

Thank you for this issue.
There has not been a lot of activity here for a while. Has this been resolved?