Dev.to Security ๐Ÿ” Cybersecurity ๐Ÿ‘ 0 ๐Ÿ“– 9 min read

Hybrid post-quantum E2EE in a messenger: what is hybrid, and how to check it yourself

Hybrid X25519 + ML-KEM-1024 and ML-DSA-65, layer by layer, with tests you can run. Umbera is an end-to-end encrypted messenger. This post walks through every place where it uses post-quantum cryptography, points to the

Hybrid post-quantum E2EE in a messenger: what is hybrid, and how to check it yourself

Hybrid X25519 + ML-KEM-1024 and ML-DSA-65, layer by layer, with tests you can run.

Umbera is an end-to-end encrypted messenger. This post walks through every place where it uses post-quantum cryptography, points to the file and the test behind each one, and ends with an equally explicit list of what you can't verify yet. All code excerpts come from the public repository and are Apache-2.0.

Where the hybrid sits: mechanism, classical half, post-quantum half

A 90-second look at the app before the details:

What is open, and what isn't

Open, in UmberaAPP/UMBERA:

  • core/: the Rust crypto core (ML-KEM-1024, ML-DSA-65, X25519, Ed25519, secp256k1, AES-GCM, key derivation, MLS groups with X-Wing, call frame keys, QR device linking, message franking), plus tests, vectors and fuzz targets. The same code is published on crates.io as umbera-core 1.1.0.
  • kotlin/: the protocol layer of the apps (envelopes, Double Ratchet, mailbox addresses, group signatures, safety number, the second call layer, meetings by link).
  • communities/: Communities cryptography, with an independent Go reference implementation.
  • docs/: PROTOCOL.md, PROOF.md, and PQ_STATUS.md, which lists every post-quantum mechanism with its file, line and test.

Not open: the server, the app UI, the network and storage layers. The full code will go to trusted independent auditors when the project is audited, and we'll publish the report afterwards. Builds aren't reproducible yet.

Why hybrid, and why now

Harvest now, decrypt later. Anyone can record ciphertext today, whether from the wire, the server or a database backup. If the session keys were agreed with elliptic curves only, a large enough quantum computer could later recover them with Shor's algorithm and read the archive. For messages that should stay private for years, the timeline of that computer barely matters, because the recording already exists.

Why not pure post-quantum? ML-KEM (FIPS 203) and ML-DSA (FIPS 204) were standardised in 2024. The math has been studied for a long time, but the implementations are young, and timing leaks have already turned up in Kyber implementations (KyberSlash). X25519 is the opposite: very well understood, and broken by a quantum adversary. A hybrid feeds both secrets into one key derivation, so an attacker has to break both. Signal's PQXDH, Apple's PQ3 and the X25519MLKEM768 key exchange in browsers' TLS make the same choice.

From docs/PROTOCOL.md, section 4:

ss_classical = X25519(ephemeral_private, recipient_x25519_public)
(ct, ss_pq)  = ML-KEM-1024.Encapsulate(recipient_kyber_public)
okm = HKDF-SHA256(ikm = ss_classical || ss_pq, salt = HYBRID_LABEL, info = "key-derivation", L = 64)

What exactly is hybrid

1:1 messages: envelope v3 and sealed sender

A message is encrypted by the Double Ratchet, packed together with the sender's identity, and sealed to the recipient. Because the sender is inside the ciphertext, the server never learns who sent it (sealed sender). In 2.1 the seal is envelope v3: an ephemeral X25519 key plus an ML-KEM-1024 encapsulation, HKDF-SHA256 salted with a hash of the transcript (ephemeral key, KEM ciphertext, both recipient public keys), then AES-256-GCM and padding to a size class. Each transcript part is length-prefixed:

// core/src/sealv3.rs, lines 8โ€“20 ยท Apache-2.0
fn part(h: &mut Sha256, x: &[u8]) {
    h.update((x.len() as u32).to_be_bytes());
    h.update(x);
}

fn th(label: &str, parts: &[&[u8]]) -> Vec<u8> {
    let mut h = Sha256::new();
    part(&mut h, label.as_bytes());
    for p in parts {
        part(&mut h, p);
    }
    h.finalize().to_vec()
}

An ML-KEM-1024 ciphertext is 1,568 bytes, so not every message carries one. A full v3 envelope has a fresh capsule. Up to 49 short v3s envelopes (24 hours at most) then use a fresh ephemeral X25519 key plus a one-way chain from the hybrid pair secret of the last confirmed full envelope. Measured sizes: 21,682 bytes for the previous v2 format, 5,038 for full v3, 3,078 for short v3s, and 3,117 on average over 50.

The ratchet itself runs on X25519, and a fresh ML-KEM-1024 secret is mixed into the root key at session start and then every 20 new chains or every 24 hours. File keys are wrapped with X25519 + ML-KEM-1024 and bound to the transcript (file wrap v2).

Mailboxes: where an envelope goes

The server has no "from" or "to" fields. An envelope goes to a mailbox address that only the two people in the conversation can compute, with one address per direction. Between two 2.1 apps, the address mixes the X25519 pair secret with a direction secret derived from the hybrid v3 exchange, so someone who later breaks X25519 still can't compute it. The address also rotates weekly, at the same second for everyone:

// kotlin/src/commonMain/kotlin/app/umbera/core/crypto/RendezvousV2.kt, lines 39โ€“40 ยท Apache-2.0
    fun ridEpoch(ssX25519: ByteArray, dir: ByteArray, fromId: String, toId: String, epoch: Long): String =
        hex(RustCore.hkdfSha256(ssX25519 + dir, SALT, "rdv-e|$fromId|$toId|$epoch".encodeToByteArray(), 32u))

If either side still runs 2.0.x, the pair keeps its previous, fixed address.

Groups: MLS with X-Wing, plus ML-DSA-65 on top

Groups use MLS (RFC 9420) through OpenMLS 0.9.2. New groups are created on MLS_256_XWING_CHACHA20POLY1305_SHA256_Ed25519 (0x004D), where the KEM is X-Wing (X25519 + ML-KEM-768). An existing group moves to X-Wing by itself once every member supports it, and keeps its members and history. A group with a 2.0.x member stays on suite 0x0001.

MLS member signatures are still Ed25519, as the standard defines today. On top of them, every Commit, Proposal and Welcome carries an ML-DSA-65 signature. Ordinary messages form a per-sender hash chain that is closed by an ML-DSA-65 checkpoint every 20 messages or after 30 seconds of silence. Once a member has announced these signatures, an unsigned change in their name is refused. The cost is +4,429 bytes per signed Commit and about +275 bytes per message on average.

Calls: a second post-quantum layer over DTLS-SRTP

DTLS-SRTP agrees its key classically, so a call recorded today could be decrypted later. That's why every audio, video and screen-sharing frame is encrypted a second time, with keys from a fresh X25519 + ML-KEM-1024 exchange per call. The exchange is carried inside the encrypted, caller-signed call setup, the keys are bound to a hash of the whole exchange, and each direction gets its own key:

// core/src/callkey.rs, lines 74โ€“86 (excerpt) ยท Apache-2.0
    let th = transcript(call_id, public, answer);
    let ikm = Zeroizing::new([ss_x, ss_k].concat());
    let prk = Zeroizing::new(hkdf::Hkdf::<Sha256>::extract(Some(&th), &ikm).0.to_vec());
    let exp = |info: &str, n: u32| p::hkdf_sha256_expand(prk.to_vec(), info.as_bytes().to_vec(), n);
    let i2r = exp("umbera-call-v1/frames/initiator->responder", 32)?;
    let r2i = exp("umbera-call-v1/frames/responder->initiator", 32)?;

Frames use AES-128-GCM in WebRTC's frame-encryptor format. The key is PBKDF2-HMAC-SHA256 (100,000 rounds) of the 256-bit hybrid output. If AES-128 here bothers you, that's a fair thing to push on. In group calls, every pair of participants runs its own exchange, and no server mixes media. In meetings by link, a 256-bit room key lives only in the URL fragment after #, so the server sees only a random room number.

Stopping key substitution

  • Safety number v2 hashes all four identity keys (secp256k1, X25519, ML-DSA-65, ML-KEM-1024), each one length-prefixed. v1 covered only the first two, which meant a server could have swapped the post-quantum keys without changing the number.
// kotlin/src/commonMain/kotlin/app/umbera/core/crypto/SafetyNumber.kt, lines 51โ€“55 ยท Apache-2.0
    fun identityV2(secp256k1: ByteArray, x25519: ByteArray, mlDsaPk: ByteArray, mlKemPk: ByteArray): String {
        var buf = V2_LABEL
        for (k in listOf(secp256k1, x25519, mlDsaPk, mlKemPk)) buf += lenBE(k.size) + k
        return HashUtils.sha256(buf).toHexLower()
    }
  • The key transparency log is an append-only Merkle tree. Since 16 November 2026, the server signs each tree head with Ed25519 and ML-DSA-65; the ML-DSA message includes the Ed25519 key. The client side is open (KeyTransparencyVerifier.sthAuthentic): once the log's ML-DSA-65 key is pinned, a tree head without a valid PQ co-signature is rejected. Pinning happens on first use, so the very first contact with the log is the weak point.
  • No silent downgrade. A new format is sent only after the contact has announced support inside an end-to-end encrypted message. If that announcement later disappears, the app shows "key changed" instead of quietly falling back.

Linking a computer by QR follows the same pattern: X25519 + ML-KEM-1024, transcript-salted HKDF, key confirmation in both directions, a 6-digit code on both screens, and a device certificate signed with both secp256k1 and ML-DSA-65.

Try it yourself

You need Rust 1.91 or newer.

git clone https://github.com/UmberaAPP/UMBERA
cd UMBERA/core
cargo test --release
# 8 test binaries, 114 passed, 0 failed

cargo test --release --test proof          # 1:1 seal: 7 tests
cargo test --release --test proof_groups   # MLS groups: 6 tests
cargo test --release --test compat_v06     # new core vs. the previous core (OpenMLS 0.6)
cargo test --release --lib sealv3::        # envelope v3 vectors, shared with Kotlin
cargo test --release --lib callkey::       # call frame keys
cargo test --release --lib xwing_hundred_leaves_and_sizes -- --nocapture

cd ../communities/pqc && go test -v ./...  # independent Go check of the Communities vectors

tests/proof.rs models the hybrid seal and checks each property directly. Here's the one that matters for post-quantum:

// core/tests/proof.rs, lines 87โ€“94 ยท Apache-2.0
fn breaking_only_x25519_is_not_enough() {
    // An attacker who somehow learned Bob's X25519 key (for example with a future
    // quantum computer) still cannot open the message without his Kyber key.
    let bob = new_recipient();
    let eve = new_recipient();
    let sealed = seal(&bob, MESSAGE);
    assert_eq!(open(&bob.x25519_private, &eve.kyber_private, &sealed), None);
}

The mirror test, breaking_only_kyber_is_not_enough, is right below it. tests/proof_groups.rs checks that every member can read a message, that an outsider can't even with a group under the same ID, that a removed member can't read new epochs, and that a new member can't read old ones. One caveat: these tests prove properties of the scheme on the core's primitives. They don't prove what a store build runs.

Known-answer vectors pin each formula byte for byte. core/tests/vectors/seal-v3-vectors.txt covers the v3/v3s envelope, the file key, mailbox addresses and their weekly epochs. It's computed in Rust with sha2 + hkdf directly and checked by the Kotlin tests too. primitives-golden.json pins SHA-256, HKDF, AES-GCM, X25519, Ed25519, ML-KEM-1024, ML-DSA-65 and more. The strongest review is to implement a formula from PROTOCOL.md yourself and diff it against these files.

Capture your own traffic. On a network you control, use Wireshark with the filter dtls || rtp || srtp during a call. You should see a DTLS handshake followed only by SRTP, with no plain RTP. Send a one-letter message and then a long paragraph, and the upload sizes fall into the same few fixed classes. Turn on Tor and repeat: nothing goes to the server directly any more.

What the server sees

Since the server code is closed, that table describes what reaches the server according to the open client code. In short, the server sees mailbox addresses and when they're used, public keys, the number of linked computers and your IP address (unless Tor is on). It doesn't see content, the sender, the recipient account, group membership, exact sizes or meeting keys.

What we don't claim

What we don't claim

  • No independent audit yet.
  • No reproducible builds yet. Until they exist, "the app runs this code" rests on our word and on what you can observe.
  • The server and the apps aren't open.
  • Without Tor, the server sees your IP address. Tor is off by default, and Tor itself has no post-quantum handshake yet.
  • Older apps (2.0.x) use older schemes: no second call layer, the previous MLS suite, safety number v1, and a fixed mailbox address. Everything is still end-to-end encrypted.
  • A key change warns, it doesn't block, and a missing transparency log entry doesn't raise a warning yet.
  • Mailbox reads are authorised with Ed25519, so that part is classical today. PQ_STATUS.md covers end-to-end encryption, not the transport to the server.
  • A compromised device is out of scope.

Break it

If the code does less than PQ_STATUS.md says, that's a security report. Email the address in SECURITY.md with the subject SECURITY (acknowledgement within 72 hours), and use GitHub Discussions for questions. There's no paid bounty yet, but confirmed findings are credited. Good places to start: the v3s inheritance window, trust-on-first-use pinning of the log's ML-DSA key, and the call frame format.

Links: repository ยท PQ_STATUS.md ยท PROTOCOL.md ยท PROOF.md ยท crates.io/umbera-core ยท umbera.app/verify

๐Ÿ“ฐ Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes โ€” full credit and traffic to the original publisher.