Skip to content

Commit e73e923

Browse files
committed
[Messenger] More on unique Redis consumer names
1 parent 80cd5b8 commit e73e923

File tree

1 file changed

+19
-4
lines changed

1 file changed

+19
-4
lines changed

messenger.rst

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -619,7 +619,18 @@ times:
619619
process_name=%(program_name)s_%(process_num)02d
620620
621621
Change the ``async`` argument to use the name of your transport (or transports)
622-
and ``user`` to the Unix user on your server. Next, tell Supervisor to read your
622+
and ``user`` to the Unix user on your server.
623+
624+
If you use the Redis Transport, note that each worker needs a unique consumer name to
625+
avoid the same message being handled by multiple workers. One way to achieve this is
626+
to set an environment variable in the Supervisor configuration file, which you can
627+
then refer to in `messenger.yaml` (see Redis section above):
628+
629+
.. code-block:: ini
630+
631+
environment=MESSENGER_CONSUMER_NAME=%(program_name)s_%(process_num)02d
632+
633+
Next, tell Supervisor to read your
623634
config and start your workers:
624635

625636
.. code-block:: terminal
@@ -1209,9 +1220,13 @@ claim_interval Interval on which pending/abandoned ``60000`` (1 Minute)
12091220
.. caution::
12101221

12111222
There should never be more than one ``messenger:consume`` command running with the same
1212-
config (stream, group and consumer name) to avoid having a message handled more than once.
1213-
Using the ``HOSTNAME`` as the consumer might often be a good idea. In case you are using
1214-
Kubernetes to orchestrate your containers, consider using a ``StatefulSet``.
1223+
combination of ``stream``, ``group`` and ``consumer``, or messages could end up being
1224+
handled more than once. If you run multiple queue workers, ``consumer` can be set to an
1225+
environment variable (like ``%env(MESSENGER_CONSUMER_NAME)%`)` set by Supervisor
1226+
(example below) or whatever service used to manage the worker processes.
1227+
In a container environment, the ``HOSTNAME`` can be used as the consumer name, since
1228+
there is only one worker per container/host. If using Kubernetes to orchestrate the
1229+
containers, consider using a ``StatefulSet`` to have stable names.
12151230

12161231
.. tip::
12171232

0 commit comments

Comments
 (0)