Skip to content

Commit 981e076

Browse files
committed
minor #22129 [Messenger] Add some missing content (javiereguiluz)
This PR was squashed before being merged into the 6.4 branch. Discussion ---------- [Messenger] Add some missing content Our friend `@mo2l` published a very nice blog post about Symfony Messenger: [Symfony Messenger: What the Documentation Does Not Cover](https://marcelmoll.dev/symfony-messenger-what-the-documentation-does-not-cover/) So, let's improve the Messenger docs with some of those missing contents. Commits ------- 0d9ae93 [Messenger] Add some missing content
2 parents 61502dc + 0d9ae93 commit 981e076

File tree

1 file changed

+261
-32
lines changed

1 file changed

+261
-32
lines changed

messenger.rst

Lines changed: 261 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -404,6 +404,69 @@ Then, in your handler, you can query for a fresh object::
404404

405405
This guarantees the entity contains fresh data.
406406

407+
.. _messenger-message-versioning:
408+
409+
Versioning Message Classes
410+
~~~~~~~~~~~~~~~~~~~~~~~~~~
411+
412+
A message class defines the **contract** between the code that dispatches the
413+
message and the worker that handles it. Because Messenger processes messages
414+
asynchronously, some messages may still be pending in the queue when you deploy
415+
a new version of your application. If you change a message class, those older
416+
messages may no longer deserialize correctly, which can cause failures when
417+
workers try to process them.
418+
419+
For **minor changes**, keep backward compatibility by making new constructor
420+
arguments optional and providing a sensible default value::
421+
422+
final class SendInvoice
423+
{
424+
public function __construct(
425+
public readonly int $orderId,
426+
public readonly ?string $locale = null, // added later
427+
) {
428+
}
429+
}
430+
431+
With this approach, older messages that do not include the new ``$locale``
432+
argument can still be deserialized correctly.
433+
434+
Another breaking change is **removing properties** from a message class. Messages
435+
already stored in the queue may still contain those properties when they are
436+
deserialized by newer workers. Since PHP 8.2, this can trigger deprecation
437+
warnings because dynamic properties are deprecated.
438+
439+
If you must remove a property, consider one of the following approaches:
440+
441+
* Keep the property temporarily (for example as ``public ?type $property =
442+
null``) until all old messages have been processed.
443+
* Add the ``#[\AllowDynamicProperties]`` attribute to the class to allow older
444+
serialized messages to set properties that no longer exist.
445+
* Implement custom serialization logic to control how the message is
446+
serialized and deserialized.
447+
448+
If the change alters the meaning of the message instead of simply extending it,
449+
create a **new version of the message class** instead of modifying the existing one::
450+
451+
// Keep SendInvoice until all queued messages using it are processed
452+
final class SendInvoiceV2
453+
{
454+
public function __construct(
455+
public readonly int $orderId,
456+
public readonly string $locale,
457+
public readonly string $templateId,
458+
) {
459+
}
460+
}
461+
462+
During deployment, both versions may need to coexist temporarily. First
463+
deploy the new ``SendInvoiceV2`` message class and its handler while keeping
464+
the old ones. After the queue has been fully drained, remove the old class and
465+
handler in a later deployment.
466+
467+
Removing the old class too early can cause workers to fail when they attempt
468+
to deserialize messages that were dispatched before the deployment.
469+
407470
.. _messenger-handling-messages-synchronously:
408471

409472
Handling Messages Synchronously
@@ -512,45 +575,75 @@ Deploying to Production
512575
On production, there are a few important things to think about:
513576

514577
**Use a Process Manager like Supervisor or systemd to keep your worker(s) running**
515-
You'll want one or more "workers" running at all times. To do that, use a
516-
process control system like :ref:`Supervisor <messenger-supervisor>`
517-
or :ref:`systemd <messenger-systemd>`.
578+
579+
You'll want one or more "workers" running at all times. To do that, use a
580+
process control system like :ref:`Supervisor <messenger-supervisor>`
581+
or :ref:`systemd <messenger-systemd>`.
518582

519583
**Don't Let Workers Run Forever**
520-
Some services (like Doctrine's ``EntityManager``) will consume more memory
521-
over time. So, instead of allowing your worker to run forever, use a flag
522-
like ``messenger:consume --limit=10`` to tell your worker to only handle 10
523-
messages before exiting (then the process manager will create a new process). There
524-
are also other options like ``--memory-limit=128M`` and ``--time-limit=3600``.
584+
585+
Some services (like Doctrine's ``EntityManager``) will consume more memory over
586+
time. So, instead of allowing your worker to run forever, use a flag like
587+
``messenger:consume --limit=10`` to tell your worker to only handle 10 messages
588+
before exiting (then the process manager will create a new process). There are
589+
also other options like ``--memory-limit=128M`` and ``--time-limit=3600``.
525590

526591
**Stopping Workers That Encounter Errors**
527-
If a worker dependency like your database server is down, or timeout is reached,
528-
you can try to add :ref:`reconnect logic <middleware-doctrine>`, or just quit
529-
the worker if it receives too many errors with the ``--failure-limit`` option of
530-
the ``messenger:consume`` command.
592+
593+
If a worker dependency like your database server is down, or timeout is reached,
594+
you can try to add :ref:`reconnect logic <middleware-doctrine>`, or just quit
595+
the worker if it receives too many errors with the ``--failure-limit`` option
596+
of the ``messenger:consume`` command.
531597

532598
**Restart Workers on Deploy**
533-
Each time you deploy, you'll need to restart all your worker processes so
534-
that they see the newly deployed code. To do this, run ``messenger:stop-workers``
535-
on deployment. This will signal to each worker that it should finish the message
536-
it's currently handling and should shut down gracefully. Then, the process manager
537-
will create new worker processes. The command uses the :ref:`app <cache-configuration-with-frameworkbundle>`
538-
cache internally - so make sure this is configured to use an adapter you like.
599+
600+
Each time you deploy, restart all worker processes so they pick up the new code.
601+
To do this, run ``messenger:stop-workers`` after the new code is on disk and
602+
the cache is warmed up, but before traffic shifts. This sets a cache flag that
603+
tells each worker to finish its current message and exit cleanly. The process
604+
manager then restarts them against the new codebase:
605+
606+
.. code-block:: terminal
607+
608+
# in your deployment script, after code deploy and cache warmup: $ php
609+
php bin/console messenger:stop-workers
610+
611+
The command uses the :ref:`app cache <cache-configuration-with-frameworkbundle>`
612+
internally. If your application runs on multiple hosts, configure the app cache
613+
to use a shared adapter (for example Redis) so that all web and worker processes
614+
use the same cache.
615+
616+
.. note::
617+
618+
In a Kubernetes environment, a rolling restart of the worker ``Deployment``
619+
achieves the same result, but only if ``terminationGracePeriodSeconds`` is
620+
long enough for the longest-running handler to complete before the pod is
621+
replaced. ``SIGKILL`` does not give workers a chance to finish the current
622+
message, which can leave a handler mid-execution.
539623

540624
**Use the Same Cache Between Deploys**
541-
If your deploy strategy involves the creation of new target directories, you
542-
should set a value for the :ref:`cache.prefix_seed <reference-cache-prefix-seed>`
543-
configuration option in order to use the same cache namespace between deployments.
544-
Otherwise, the ``cache.app`` pool will use the value of the ``kernel.project_dir``
545-
parameter as base for the namespace, which will lead to different namespaces
546-
each time a new deployment is made.
625+
626+
If your deploy strategy involves the creation of new target directories, you
627+
should set a value for the :ref:`cache.prefix_seed <reference-cache-prefix-seed>`
628+
configuration option in order to use the same cache namespace between deployments.
629+
Otherwise, the ``cache.app`` pool will use the value of the ``kernel.project_dir``
630+
parameter as base for the namespace, which will lead to different namespaces
631+
each time a new deployment is made.
547632

548633
Prioritized Transports
549634
~~~~~~~~~~~~~~~~~~~~~~
550635

551-
Sometimes certain types of messages should have a higher priority and be handled
552-
before others. To make this possible, you can create multiple transports and route
553-
different messages to them. For example:
636+
Use separate transports for message types with different latency requirements,
637+
failure modes, or retry windows. When multiple message types share the same
638+
transport, a slow or failing handler for one type can delay all others in the
639+
same queue.
640+
641+
For example, if catalog sync messages and payment confirmations are routed to
642+
the same transport, a slow product feed handler could delay payment confirmations
643+
for customers going through checkout.
644+
645+
Consider assigning each transport its own worker so that failures or slowdowns
646+
in one message flow do not affect others:
554647

555648
.. configuration-block::
556649

@@ -1054,6 +1147,13 @@ this is configurable for each transport:
10541147
;
10551148
};
10561149
1150+
.. tip::
1151+
1152+
If failures in your application involve external services, including
1153+
rate-limiting issues, the default retry window may be too short. Consider
1154+
increasing ``max_retries``, ``delay``, ``multiplier``, and ``max_delay`` to
1155+
better match your use case.
1156+
10571157
.. tip::
10581158

10591159
Symfony triggers a :class:`Symfony\\Component\\Messenger\\Event\\WorkerMessageRetriedEvent`
@@ -1197,6 +1297,66 @@ If the message fails again, it will be re-sent back to the failure transport
11971297
due to the normal :ref:`retry rules <messenger-retries-failures>`. Once the max
11981298
retry has been hit, the message will be discarded permanently.
11991299

1300+
.. _messenger-handler-idempotency:
1301+
1302+
Writing Idempotent Handlers
1303+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
1304+
1305+
A message can be **delivered more than once** under normal operating conditions.
1306+
For example, a worker may process a message successfully but crash before
1307+
acknowledging it to the transport. In that case, the transport will redeliver
1308+
the message.
1309+
1310+
If running a handler twice produces different outcomes (for example charging
1311+
a customer twice, sending duplicate emails, or reducing inventory twice),
1312+
retry mechanisms cannot prevent those errors.
1313+
1314+
Whenever possible, design handlers to be naturally **idempotent**. This means
1315+
that running them multiple times should produce the same result as running them
1316+
once. For example, setting an absolute value (such as the current stock level
1317+
from an external feed) is safe to repeat, whereas decrementing a counter is not.
1318+
1319+
For operations that are inherently non-idempotent, such as payment processing,
1320+
include a **stable idempotency key** in the message. The key should be derived
1321+
from the business event, not generated at dispatch time, so that any redelivery
1322+
of the same logical event uses the same key::
1323+
1324+
final class ProcessPayment
1325+
{
1326+
public function __construct(
1327+
public readonly int $orderId,
1328+
// the idempotencyKey is derived from the order and not generated at dispatch time
1329+
// for example: "payment-{orderId}-{orderVersion}"
1330+
public readonly string $idempotencyKey,
1331+
) {
1332+
}
1333+
}
1334+
1335+
In the handler, check the key before performing the operation and enforce a
1336+
database-level uniqueness constraint. The check helps avoid unnecessary work,
1337+
while the constraint protects against concurrent redeliveries::
1338+
1339+
#[AsMessageHandler]
1340+
final class ProcessPaymentHandler
1341+
{
1342+
public function __invoke(ProcessPayment $message): void
1343+
{
1344+
if ($this->paymentRepository->existsByIdempotencyKey($message->idempotencyKey)) {
1345+
return;
1346+
}
1347+
1348+
// ... process payment, persist record with the idempotency key
1349+
}
1350+
}
1351+
1352+
.. note::
1353+
1354+
A UUID generated at dispatch time is not suitable as an idempotency key.
1355+
If the same business event is dispatched twice (for example because of a
1356+
double form submission), each dispatch generates a different UUID and
1357+
both executions will proceed. The key must remain stable across all
1358+
dispatches of the same logical event.
1359+
12001360
Multiple Failed Transports
12011361
~~~~~~~~~~~~~~~~~~~~~~~~~~
12021362

@@ -1582,8 +1742,11 @@ DSN by using the ``table_name`` option:
15821742
# .env
15831743
MESSENGER_TRANSPORT_DSN=doctrine://default?table_name=your_custom_table_name
15841744
1585-
Or, to create the table yourself, set the ``auto_setup`` option to ``false`` and
1586-
:ref:`generate a migration <doctrine-creating-the-database-tables-schema>`.
1745+
By default, the Doctrine table is created automatically when the first message
1746+
is dispatched. This is convenient in development, but in production you may
1747+
prefer to set ``auto_setup`` to ``false`` in ``MESSENGER_TRANSPORT_DSN``. Then
1748+
:ref:`generate a migration <doctrine-creating-the-database-tables-schema>` and
1749+
create the table explicitly as part of your deployment process.
15871750

15881751
The transport has a number of options:
15891752

@@ -1791,9 +1954,12 @@ under the transport in ``messenger.yaml``:
17911954
handled more than once. If you run multiple queue workers, ``consumer`` can be set to an
17921955
environment variable, like ``%env(MESSENGER_CONSUMER_NAME)%``, set by Supervisor
17931956
(example below) or any other service used to manage the worker processes.
1794-
In a container environment, the ``HOSTNAME`` can be used as the consumer name, since
1795-
there is only one worker per container/host. If using Kubernetes to orchestrate the
1796-
containers, consider using a ``StatefulSet`` to have stable names.
1957+
1958+
.. tip::
1959+
1960+
In a container environment, the ``HOSTNAME`` can be used as the consumer name,
1961+
since there is only one worker per container/host. If using Kubernetes to
1962+
orchestrate the containers, consider using a ``StatefulSet`` to have stable names.
17971963

17981964
.. tip::
17991965

@@ -1895,6 +2061,40 @@ The transport has a number of options:
18952061
:class:`Symfony\\Bundle\\FrameworkBundle\\Test\\KernelTestCase`
18962062
or :class:`Symfony\\Bundle\\FrameworkBundle\\Test\\WebTestCase`.
18972063

2064+
You can optionally use two complementary testing strategies for message
2065+
handling. First, test handlers as plain PHP classes by injecting mocks and
2066+
calling ``__invoke()`` directly, without any Messenger infrastructure. This
2067+
allows you to verify business logic, exception classification, and edge cases::
2068+
2069+
// tests/MessageHandler/SendInvoiceHandlerTest.php
2070+
final class SendInvoiceHandlerTest extends TestCase
2071+
{
2072+
public function testThrowsUnrecoverableForUnknownOrder(): void
2073+
{
2074+
$repository = $this->createMock(OrderRepository::class);
2075+
$repository->method('find')->willReturn(null);
2076+
2077+
$handler = new SendInvoiceHandler($repository);
2078+
2079+
$this->expectException(UnrecoverableMessageHandlingException::class);
2080+
$handler(new SendInvoice(orderId: 99));
2081+
}
2082+
}
2083+
2084+
Then, use the ``in-memory://`` transport in functional tests to verify that the
2085+
correct message is dispatched to the expected transport. Map each named transport
2086+
to ``in-memory://`` in your test configuration and assert on the specific transport::
2087+
2088+
// tests/Controller/CheckoutControllerTest.php
2089+
/** @var InMemoryTransport $transport */
2090+
$transport = self::getContainer()->get('messenger.transport.orders_high');
2091+
2092+
self::assertCount(1, $transport->getSent());
2093+
self::assertInstanceOf(
2094+
SendOrderConfirmation::class,
2095+
$transport->getSent()[0]->getMessage()
2096+
);
2097+
18982098
Amazon SQS
18992099
~~~~~~~~~~
19002100

@@ -2802,6 +3002,24 @@ are a variety of different stamps for different purposes and they're used intern
28023002
to track information about a message - like the message bus that's handling it
28033003
or if it's being retried after failure.
28043004

3005+
.. tip::
3006+
3007+
Symfony doesn't inject the :class:`Symfony\\Component\\Messenger\\Envelope`
3008+
automatically when you add it as an argument of the ``__invoke()`` method
3009+
in your handler. To do so, you can create the following custom :ref:`middleware <messenger_middleware>`
3010+
to stamp the envelope before ``HandleMessageMiddleware`` runs::
3011+
3012+
final class InjectEnvelopeMiddleware implements MiddlewareInterface
3013+
{
3014+
public function handle(Envelope $envelope, StackInterface $stack): Envelope
3015+
{
3016+
return $stack->next()->handle(
3017+
$envelope->with(new HandlerArgumentsStamp([$envelope])),
3018+
$stack
3019+
);
3020+
}
3021+
}
3022+
28053023
.. _messenger_middleware:
28063024

28073025
Middleware
@@ -3249,6 +3467,17 @@ You can configure these buses and their rules by using middleware.
32493467
It might also be a good idea to separate actions from reactions by introducing
32503468
an **event bus**. The event bus could have zero or more subscribers.
32513469

3470+
.. tip::
3471+
3472+
A single bus is a **good default**. Add another bus only when you need a
3473+
different middleware stack, not because an architecture pattern suggests it.
3474+
For example, you might add ``doctrine_transaction`` middleware to a command
3475+
bus but not to a query bus, since read operations do not need the overhead
3476+
of a transaction.
3477+
3478+
If you cannot identify a concrete behavioral difference between the buses,
3479+
you likely do not need more than one yet.
3480+
32523481
.. configuration-block::
32533482

32543483
.. code-block:: yaml

0 commit comments

Comments
 (0)