Skip to content

feat: DTLS 1.3 with X25519MLKEM768 post-quantum key exchange (draft, needs unreleased BouncyCastle) - #2451

Draft
JonathanLennox wants to merge 2 commits into
jitsi:masterfrom
JonathanLennox:dtls13
Draft

JonathanLennox wants to merge 2 commits into
jitsi:masterfrom
JonathanLennox:dtls13

Conversation

@JonathanLennox

@JonathanLennox JonathanLennox commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Negotiate DTLS 1.3 (RFC 9147) when the remote endpoint supports it, falling back to DTLS 1.2 otherwise, and use the X25519MLKEM768 post-quantum hybrid key exchange when the peer offers it. Also picks up #2260 (RFC 9925 unsigned local certificates) using BouncyCastle's own NoSignatureContentSigner.

Draft because it depends on BouncyCastle changes that are not released: the DTLS 1.3 series by @mondain (bcgit/bc-java#2439, #2440, #2441, #2442, stacked) plus a fix for the RFC 9147 §5.9 dtls13 HKDF label prefix without which no non-BC peer can decrypt anything (mondain/bc-java#1, reported on bcgit/bc-java#2441), and three further conformance fixes not needed for interop (mondain/bc-java#2). The root pom points at a locally built 1.86.13-SNAPSHOT, so CI will not resolve it until those land in a BC release.

What changes

  • jmt.dtls.enable-dtls13 (default true): client offers and server accepts DTLS 1.3 down to 1.2.
  • jmt.dtls.enable-post-quantum-key-exchange (default true, DTLS 1.3 only): X25519MLKEM768 offered first with an X25519 key share alongside, so a 1.3 server without the hybrid needs no HelloRetryRequest; as server we prefer our own group order so the hybrid is used whenever the client lists it.
  • jmt.dtls.use-alg-unsigned (default true): local certificate left unsigned per RFC 9925.
  • jmt.dtls.mtu (default 1200, was a hardcoded 1500): sizes outgoing DTLS datagrams. A DTLS 1.3 ServerHello with a post-quantum key share is over 1200 bytes, and on a path with a 1280-byte MTU the 1500-byte assumption silently lost every copy of it (DF is set, and the kernel's EMSGSIZE on retransmission is swallowed by the send path). 1200 is what browsers use.
  • Default cipher-suite list gains TLS_AES_128_GCM_SHA256 and TLS_CHACHA20_POLY1305_SHA256.
  • Server-side DTLS 1.3 support in TlsServerImpl: credential selection without a key-exchange algorithm, the 1.3-form CertificateRequest, and a Certificate message carrying a certificate_request_context (CertificateInfo.certificateFor).
  • DtlsStack exposes negotiatedProtocolVersion and negotiatedGroup; both are logged at handshake completion and shown in the debug state.

Tests

  • DtlsTest: DTLS 1.3 + X25519MLKEM768 by default, DTLS 1.2 with 1.3 disabled, DTLS 1.3 + X25519 with PQ disabled.
  • New DtlsVersionNegotiationTest: a real DtlsStack in each role against a plain BouncyCastle peer that is either DTLS 1.2-only or DTLS 1.3 without the hybrid, asserting version, group, and byte-identical SRTP keying material.
  • Verified on a staging bridge against both Chrome (BoringSSL) and Firefox (NSS): Negotiated DTLS version DTLS 1.3, key exchange group X25519MLKEM768, media flowing both ways. The Firefox client was behind a 1280-byte path MTU, which is what found the MTU issue above.

DtlsTest carries a local withConfig helper until jicoco with jitsi/jicoco#244 (config reset in finally) is picked up.

…t, falling back to DTLS 1.2.

Offer/accept DTLS 1.3 (RFC 9147) in addition to DTLS 1.2, prefer the X25519MLKEM768 post-quantum
hybrid key exchange when available, and leave the local certificate unsigned (RFC 9925).
Requires the BouncyCastle DTLS 1.3 series (bcgit/bc-java#2439-jitsi#2442) plus the RFC 9147 5.9
"dtls13" HKDF label fix; the pom points at a locally built 1.86.13-SNAPSHOT until those are released.
jitsi-ci Bot pushed a commit to jitsi/jitsi-pr-tests-pages that referenced this pull request Sep 14, 2026
…d 1500).

Outgoing DTLS datagrams larger than the path MTU are dropped (DF is set), which silently breaks the
handshake. A DTLS 1.3 ServerHello carrying an X25519MLKEM768 key share is over 1200 bytes, so with a
1500-byte assumption it failed to reach a Firefox client on a path with a 1280-byte MTU. Browsers use
1200 for DTLS, and it stays under the IPv6 minimum of 1280.
jitsi-ci Bot pushed a commit to jitsi/jitsi-pr-tests-pages that referenced this pull request Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants