7.7x Smaller and Encrypted at Rest: Compressing Symfony Messenger Payloads Without Touching a Handler

Symfony Messenger serializes a message, hands the bytes to a transport and forgets about them. Two things routinely go wrong with those bytes:

1. They are too big. A bulk export, a sync job with a few thousand rows, a report with embedded data, and you're past SQS's 256 KB limit or your Redis memory is going somewhere it shouldn't.
2. They are readable. Names, emails, IBANs sit in plain text in messenger_messages, in a Redis RDB dump, on a RabbitMQ disk and in the dead-letter queue nobody has looked at since March.

mattleads/messenger-compress-bundle fixes both at the transport-serializer layer. You opt in per message with a stamp or enforce a policy per transport, and your handlers receive the same plain PHP objects they always did. We'll walk through the whole thing on a small Doctrine (SQLite) transport, because then the queue is a table you can SELECT from and see what changed.

Prerequisites

• PHP 8.2 or newer. ext-zlib and ext-sodium ship with PHP, so there is nothing to install.
• A Symfony 6.4, 7.x or 8.x application with symfony/messenger. The outputs below come from Symfony 8.1.
• mattleads/messenger-compress-bundle v0.1.1.
• To follow along exactly, clone the demo. It uses a Doctrine transport on SQLite.

1. Install

composer require mattleads/messenger-compress-bundle

Symfony Flex registers the bundle. Without Flex, add it yourself:

// config/bundles.php
return [
    // ...
    MattLeads\MessengerCompressBundle\MattLeadsMessengerCompressBundle::class => ['all' => true],
];

Nothing happens yet. The bundle registers a serializer service, mattleads_messenger_compress.serializer, that wraps Messenger's default serializer. Until a transport uses it, your messages are untouched.

2. Wire it to a transport

You have two options. Start with one transport:

# config/packages/messenger.yaml
framework:
    messenger:
        failure_transport: failed
        transports:
            async:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                serializer: mattleads_messenger_compress.serializer
            failed: 'doctrine://default?queue_name=failed'
        routing:
            'App\Message\GenerateInvoiceBatch': async
            'App\Message\ExportCustomerData': async

Or wrap the default serializer so every transport gets it:

# config/packages/mattleads_messenger_compress.yaml
mattleads_messenger_compress:
    decorate_default_serializer: true

One tip for the walkthrough only: Messenger's default serializer is PHP serialize(). If you switch the default to the JSON one, plain bodies become human-readable in the database, which makes the "before" picture much clearer:

framework:
    messenger:
        serializer:
            default_serializer: messenger.transport.symfony_serializer

The bundle doesn't care which one you wrap. It compresses and encrypts bytes.

3. See the problem first

Here is a message you would not want sitting in a queue table:

final readonly class ExportCustomerData
{
    public function __construct(
        public string $customerId,
        public string $fullName,
        public string $email,
        public string $iban,
    ) {}
}

Dispatch it as usual and look at the row. In the demo, app:queue:inspect --grep <string> does this. By hand it's just SQL:

SELECT id, length(body), substr(body, 1, 60), headers FROM messenger_messages;

 id   message              X-Compression   X-Encryption   key   encoding   body bytes   body preview
 1    ExportCustomerData   -               -              -     -          119          {"customerId":"cust-4711","fullName":"Nino Beri…
 [ERROR] "GE29NB0000000101904917" is readable in 1 stored message(s).

The IBAN is right there. Every backup of that database has it too.

4. Encrypt it

Generate a key

bin/console messenger-compress:generate-key -v

k3Jm…(44 characters of base64)…=

Add to .env.local (never commit it):
  MESSENGER_ENCRYPTION_KEY_V1=k3Jm…
and to config/packages/mattleads_messenger_compress.yaml:
  encryption: { enabled: true, keys: { v1: '%env(MESSENGER_ENCRYPTION_KEY_V1)%' } }

That's 32 random bytes, base64-encoded. Put it in .env.local for development and in your secrets store for production. Symfony's own vault works fine:

bin/console secrets:set MESSENGER_ENCRYPTION_KEY_V1

Configure

# config/packages/mattleads_messenger_compress.yaml
mattleads_messenger_compress:
    encryption:
        enabled: true
        keys:
            v1: '%env(MESSENGER_ENCRYPTION_KEY_V1)%'

Dispatch with a stamp

use MattLeads\MessengerCompressBundle\Stamp\EncryptStamp;

$bus->dispatch(new ExportCustomerData(...), [new EncryptStamp()]);

Same row, after:

 2    ExportCustomerData   -               xchacha20poly1305   v1    base64     212          jUa0PNnAtVBUT5+uTPOOxsbUrOTsIg9cYKnD1WAKsnRQEyN…

The body is XChaCha20-Poly1305 ciphertext (libsodium, fresh 192-bit nonce per message, authenticated). The headers are still readable: the message type, the scheme, the key id. That's deliberate: the consumer needs them to know what to do and Messenger needs the type to route the message. Which brings up the one rule worth memorizing, call it the header rule: stamps travel as headers and are not encrypted. Put secrets in the message body, never in a stamp.

Notice the size went up: 119 bytes became 212. A 24-byte nonce, a 16-byte tag and base64's 33 % overhead cost more than a tiny body saves. That's fine. Encryption is a decision about sensitivity, not about size.

Or encrypt everything on this transport

If a transport only ever carries sensitive work, stop thinking about stamps:

mattleads_messenger_compress:
    encryption:
        enabled: true
        always: true
        keys: { v1: '%env(MESSENGER_ENCRYPTION_KEY_V1)%' }

A single message can opt out with PlainStamp. An explicit EncryptStamp always wins over policy.

5. Compress the big ones

Now the other problem. GenerateInvoiceBatch with 5,000 order lines serializes to 759 KB of JSON. Three ways to shrink it:

Explicitly, per message:

use MattLeads\MessengerCompressBundle\Stamp\CompressStamp;

$bus->dispatch($batch, [new CompressStamp()]);            // configured default: deflate
$bus->dispatch($batch, [new CompressStamp('gzip')]);
$bus->dispatch($batch, [new CompressStamp('zstd', 3)]);   // algorithm + level; zstd needs ext-zstd

Automatically above a size. This is the default to pick, because nobody remembers to add a stamp to the one message that grows:

mattleads_messenger_compress:
    compression:
        default_algorithm: deflate     # deflate | gzip | zstd
        level: null                    # null = the algorithm's default
        auto_threshold: 4096           # any body >= 4 KiB is compressed, no stamp needed

Always:

mattleads_messenger_compress:
    compression:
        always: true

Two details of the automatic policies: they only keep the compressed body when it is actually smaller than the original (random blobs and already-compressed data pass through untouched) and PlainStamp opts a message out. An explicit CompressStamp is always honored even if the result is larger.

What it looks like with auto_threshold: 4096:

The third row is the common production combination: compress and encrypt. The bundle always compresses first and encrypts second regardless of the order of your stamps, because ciphertext does not compress. The 76 bytes between rows two and three are the gzip header plus the nonce and tag.

One caveat from the CRIME/BREACH family: compressed-then-encrypted length can leak information about the plaintext. If a single message mixes attacker-controlled input with a secret and the length is observable to that attacker, do not compress it. For almost every queue job this isn't a concern, but it's worth knowing.

6. What goes over the wire

The bundle adds up to four headers to the encoded envelope. Everything else (the type header, the stamp headers) is whatever the inner serializer produced.

Base64 is the default because the Doctrine, Redis and SQS transports all want text bodies. On AMQP you can drop it and save the 33 %:

mattleads_messenger_compress:
    encoding: raw

Because everything is in headers and the body is self-describing, a consumer with the bundle installed decodes any mix of plain, compressed, encrypted and compressed-and-encrypted messages from the same queue. You can roll the bundle out to consumers first and start stamping messages later.

7. Consume, and get metrics for free

Nothing changes in the handler:

#[AsMessageHandler]
final readonly class ExportCustomerDataHandler
{
    public function __invoke(ExportCustomerData $message): void
    {
        // $message is the plain object. Decryption happened in the transport serializer.
    }
}

But the envelope your handler is wrapped in now carries a TransformedStamp describing what the serializer undid: compression, encryption, keyId, wireSize, rawSize and a ratio() helper. It implements NonSendableStampInterface, so it is never sent back out if the message is re-dispatched. A tiny listener turns it into logs or metrics:

use MattLeads\MessengerCompressBundle\Stamp\TransformedStamp;
use Symfony\Component\EventDispatcher\Attribute\AsEventListener;
use Symfony\Component\Messenger\Event\WorkerMessageReceivedEvent;

final readonly class TransformedStampLogger
{
    public function __construct(private LoggerInterface $logger) {}

    #[AsEventListener]
    public function __invoke(WorkerMessageReceivedEvent $event): void
    {
        $stamp = $event->getEnvelope()->last(TransformedStamp::class);
        if (null === $stamp) {
            return; // message was neither compressed nor encrypted
        }

        $this->logger->info('compression={c} encryption={e} key={k} wire={w} raw={r} ratio={ratio}', [
            'c' => $stamp->compression ?? '-',
            'e' => $stamp->encryption ?? '-',
            'k' => $stamp->keyId ?? '-',
            'w' => $stamp->wireSize,
            'r' => $stamp->rawSize,
            'ratio' => sprintf('%.1f%%', 100 * $stamp->ratio()),
        ]);
    }
}

$ bin/console messenger:consume async --limit=5
Received ExportCustomerData: compression=- encryption=xchacha20poly1305 key=v1 wire=212 B raw=119 B ratio=178.2%
Handled export for cust-4711 (Nino Beridze, iban ends 4917)
Received GenerateInvoiceBatch: compression=deflate encryption=- key=- wire=98,060 B raw=758,571 B ratio=12.9%
Handled batch-5000: 5000 lines, total 15,537,656.65 EUR

Push ratio() and wireSize into your metrics backend tagged by message class and you've got a dashboard of what each job actually costs your broker.

8. Rotate keys without draining the queue

Keys leak, people leave, policies say "every 90 days". Rotation is the part most home-grown encryption layers get wrong, because there's always a message in flight encrypted with the old key.

The bundle's answer is a key ring. Every encrypted message carries its key id in X-Encryption-Key-Id; consumers decode with whichever key the message names and active_key says which key new messages use.

mattleads_messenger_compress:
    encryption:
        enabled: true
        active_key: '%env(MESSENGER_ACTIVE_KEY)%'
        keys:
            v1: '%env(MESSENGER_ENCRYPTION_KEY_V1)%'
            v2: '%env(MESSENGER_ENCRYPTION_KEY_V2)%'

The procedure, in order:

1. Generate v2 (messenger-compress:generate-key --id v2 -v) and deploy it to every consumer alongside v1. Nothing changes yet; active_key is still v1.
2. Flip active_key to v2 on producers. New messages are v2; consumers still decode the v1 backlog.
3. Once the queues (including the failure transport) have cycled, remove v1.

You can watch it happen in the demo by running the producer with one key and the consumer with the other:

php bin/console app:dispatch --export --encrypt                              # active_key=v1
MESSENGER_ACTIVE_KEY=v2 php bin/console app:dispatch --export --encrypt
MESSENGER_ACTIVE_KEY=v2 php bin/console messenger:consume async --limit=2

Received ExportCustomerData: … encryption=xchacha20poly1305 key=v1 …
Received ExportCustomerData: … encryption=xchacha20poly1305 key=v2 …

If active_key is omitted, the last key in the map is used.

9. When it goes wrong: wrong key, tampered body

This is the part to understand before production. Decryption is authenticated, so a message encrypted with a key the consumer doesn't have, or modified in the broker, is refused rather than handed to your handler as garbage. Let's corrupt a stored body by hand:

php bin/console app:dispatch --export --encrypt
sqlite3 var/data.db "UPDATE messenger_messages SET body = substr(body,1,40) || 'X' || substr(body,42) WHERE delivered_at IS NULL;"
php bin/console messenger:consume async --limit=3
php bin/console messenger:failed:show

Error thrown while handling message MessageDecodingFailedException. Sending for retry #1 using 93 ms delay. Error: "Decryption failed: authentication tag mismatch (wrong key or tampered payload)."
Error thrown while handling message MessageDecodingFailedException. Sending for retry #2 using 110 ms delay. …
Error thrown while handling message MessageDecodingFailedException. Removing from transport after 2 retries. …

There is 1 message pending in the failure transport.
  Id   Class                                                                   Failed at             Error
  4    Symfony\Component\Messenger\Exception\MessageDecodingFailedException   2026-11-01 15:12:53   Decryption failed: authentication tag mismatch (wrong key or tampered payload).

So: the worker doesn't crash, the handler never runs, the retry strategy you configured applies and the message is parked on the failure transport with the original bytes intact. That last point matters for the "wrong key" case. Suppose a consumer was deployed without v2 and some v2 messages failed. Add the key and replay:

php bin/console messenger:failed:retry --force

Handled export for cust-4711 (Nino Beridze, iban ends 4917)

The failed message is re-sent through the serializer untouched (it isn't compressed or encrypted a second time), decoded with the now-present key and handled normally.

A note on Symfony 8.1. Messenger 8.1 changed how a transport serializer must report a message it cannot decode: it has to return an envelope wrapping MessageDecodingFailedException instead of throwing, so the stamps that count retries survive. Bundle v0.1.0 threw and on 8.1 that meant "retry #1" forever with an empty failure transport. The scenario above is how I found it; v0.1.1 returns the wrapped failure. The official custom-serializer example in the Symfony docs now shows the same pattern (symfony-docs#23459) and the throwing path itself is tracked in symfony/symfony#67791, with a one-file reproducer.

The errors you can see and what they mean:

10. Testing

Your handler tests do not change; they never saw the serializer. For an integration test that exercises the real round trip, the in-memory transport can run the serializer like a broker would:

# config/packages/test/messenger.yaml
framework:
    messenger:
        transports:
            async:
                dsn: 'in-memory://?serialize=true'
                serializer: mattleads_messenger_compress.serializer

Dispatch with stamps, consume, assert on the TransformedStamp of the received envelope. This is how the bundle's own functional tests work.

11. Production checklist

• Keys come from a secrets store, never from the repository. secrets:set, Vault, SSM, whatever you already use.
• Consumers before producers. Install the bundle on consumers first; add keys to consumers first; only then stamp or switch active_key.
• Prefer a threshold over stamps. auto_threshold: 4096 catches the message that grows six months from now. Reserve explicit stamps for zstd/level tuning.
• Encrypt by transport if you can. A dedicated sensitive transport with encryption.always: true is easier to audit than stamps scattered through services.
• Headers are plain text. Message class, stamp contents, key id are all visible. Keep secrets in the body.
• Keep the failure transport in mind during rotation. Messages parked there may still be on the old key; do not remove it until the failure transport is empty or replayed.
• Watch ratio(). If a message class is consistently above 100 %, it's small and doesn't need compression; if it is consistently at 100 %, the data is already compressed (images, archives) and PlainStamp saves CPU.

Where to go from here

The bundle is on Packagist and GitHub (MIT, Symfony 6.4–8.x, PHP 8.2+). The runnable walkthrough with the inspect command, the listener and the tamper scenario is at github.com/mattleads/messenger-compress-demo.

Custom compressors (Brotli, LZ4) plug in by implementing CompressorInterface and are picked up by the mattleads_messenger_compress.compressor tag. Custom encryptors implement EncryptorInterface. If you run this against SQS, Redis or AMQP at volume, open an issue with the ratios you see and anything that surprised you. That's what the issue tracker is for.

Source Code: You can find the full implementation and follow the project's progress on GitHub: https://github.com/mattleads/messenger-compress-bundle

Let's Connect!

If you found this helpful or have questions about the implementation, I'd love to hear from you. Let's stay in touch and keep the conversation going across these platforms:

• LinkedIn: https://www.linkedin.com/in/matthew-mochalkin/
• X (Twitter): https://x.com/MattLeads
• Telegram: https://t.me/MattLeads
• GitHub: https://github.com/mattleads