Counter intuitive TLS 1.3 handshake latency: MLKEM768 vs X25519 vs X25519MLKEM768

[embetrix]

Hi,

I’ve been experimenting with OpenSSL 3 + OQS provider on embedded Linux/raspberrypi5 (ARM64) and benchmarking TLS 1.3 handshake latency:

I’m observing something counter-intuitive in the results:

X25519 KEX:

root@raspberrypi5-c8-bf-64:~# for i in $(seq 1 1000); do curl -o /dev/null -s -w "%{time_appconnect}\n" https://localhost:443; done | awk '{sum += $1 * 1000} END {printf "Average TLS handshake: %.3f ms\n", sum/NR}'
Average TLS handshake: 63.388 ms

X25519MLKEM768 KEX:

root@raspberrypi5-c8-bf-64:~# for i in $(seq 1 1000); do curl -o /dev/null -s -w "%{time_appconnect}\n" https://localhost:444; done | awk '{sum += $1 * 1000} END {printf "Average TLS handshake: %.3f ms\n", sum/NR}'
Average TLS handshake: 51.195 ms

MLKEM768 KEX:

root@raspberrypi5-c8-bf-64:~# for i in $(seq 1 1000); do curl -o /dev/null -s -w "%{time_appconnect}\n" https://localhost:445; done | awk '{sum += $1 * 1000} END {printf "Average TLS handshake: %.3f ms\n", sum/NR}'
Average TLS handshake: 38.889 ms

I'm a bit puzzeld by the results since I have expected:

X25519 < Hybrid < Pure PQC

Has anyone observed similar behavior?

My Nginx configs are:

X2551 (Port 443):

server {

    listen 443 ssl;

    ssl_certificate     /opt/oqs-demos/certs/rsa-device-server-cert.pem;
    ssl_certificate_key /opt/oqs-demos/certs/rsa-device-server-key.pem;

    ssl_protocols TLSv1.3;
    # use classical curve for key exchange
    ssl_ecdh_curve X25519;
    
    location / {
        root   /var/www/localhost/html;
        index  index.html index.htm;
    }
}

X25519MLKEM768 (Port 444):

server {

    listen 444 ssl;

    ssl_certificate     /opt/oqs-demos/certs/rsa-device-server-cert.pem;
    ssl_certificate_key /opt/oqs-demos/certs/rsa-device-server-key.pem;

    ssl_protocols TLSv1.3;
    # use hybrid curve for key exchange
    ssl_ecdh_curve X25519MLKEM768;

    location / {
        root   /var/www/localhost/html;
        index  index.html index.htm;
    }
}

MLKEM768 (Port 445):

server {

    listen 445 ssl;

    ssl_certificate     /opt/oqs-demos/certs/rsa-device-server-cert.pem;
    ssl_certificate_key /opt/oqs-demos/certs/rsa-device-server-key.pem;

    ssl_protocols TLSv1.3;
    # use PQC only curve for key exchange (Note that this can be also be used with classical keys/certs)
    ssl_ecdh_curve mlkem768;

    location / {
        root   /var/www/localhost/html;
        index  index.html index.htm;
    }
}

[pratikrath126]

Hey! Those results are actually plausible: MLKEM768 can be faster than X25519 on some hardware due to highly optimized AVX2/NEON implementations and smaller ciphertext sizes. The hybrid (X25519MLKEM768) combines both, so it may sit between them. Also, TLS 1.3 handshake latency is dominated by round trips and crypto operations; if the PQC code is better optimized or fits in cache, it can beat classical ECDH. Ensure you're testing with cold caches, same CPU frequency, and that the OpenSSL OQS provider is built with optimal flags. Interesting data — thanks for sharing!

[embetrix]

yes I built liboqs with arm64 optimizations flags:

-DOQS_USE_ARM_NEON_INSTRUCTIONS=ON
-DOQS_USE_ARM_AES_INSTRUCTIONS=ON 
-DOQS_USE_ARM_SHA2_INSTRUCTIONS=ON 

But I suppose that OpenSSL build also benefits from optimization while using X25519 KEX

Before I ran the benchmark I set cpu(s) governor to performance to have similar frequency for all 3 tests, the curl calls in my bechmark are not cached since it's a new process at each round.

[embetrix]

I tried the benchmark on an stm32mp157 (ARM32 Bits) with 800MHz with liboqs having no optimzations and built using -DOQS_PERMIT_UNSUPPORTED_ARCHITECTURE=ON flag, I get follwoing:

Key Exchange: x25519
Running 100 TLS handshakes for https://localhost:443 ...
Average TLS handshake: 612.19 ms (100 rounds)

Key Exchange: X25519MLKEM768
Running 100 TLS handshakes for https://localhost:444 ...
Average TLS handshake: 453.77 ms (100 rounds)

Key Exchange: mlkem768
Running 100 TLS handshakes for https://localhost:445 ...
Average TLS handshake: 625.43 ms (100 rounds)

still that hybrid KEX is faster than x25519 !

[baentsch]

Can I ask @embetrix which exact openssl and liboqs/oqsprovider versions are being used in these tests and how these tests turn out on different platforms?

Your very own embetrix/meta-oqs@e46e227 can easily explain why OQS MLKEM768 may be faster than OpenSSL X25519...

However,

hybrid KEX is faster than x25519

indeed is a statement that also surprises me if indeed the oqsprovider code base were running: It calls on the very same X25519 logic in the hybrid case and hence should not be able to operate an X25519 hybrid faster than pure X25519 (as the MLKEM768 code does not run in parallel, but in sequence).

One last comment: Handshakes are a "pretty heavy" operation with many "moving parts" that may contribute to these results. At the time OQS had benchmarking support for TLS handshakes (a few years back) those results were among the least reliable, certainly when compared to pure openssl speed (algorithm-only) measurements: I'd thus recommend checking also those results for the same algorithm set on the same platform.

[embetrix]

@baentsch thanks for your suggestion:

I made OpenSSL speed tests:

1. Classical

root@raspberrypi5-c8-bf-64:~# openssl speed ecdhx25519
Doing 253 bits  ecdh ops for 10s: 60366 253-bits ECDH ops in 10.00s
version: 3.2.6
built on: Tue Sep 30 13:05:05 2025 UTC
options: bn(64,64)
compiler: aarch64-poky-linux-gcc  -mcpu=cortex-a76+crypto -mbranch-protection=standard -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=recipe-sysroot -O2 -pipe -g -feliminate-unused-debug-types -fcanon-prefix-map  -fmacro-prefix-map=  -fdebug-prefix-map=  -fmacro-prefix-map=  -fdebug-prefix-map=  -fdebug-prefix-map=  -fmacro-prefix-map=  -fdebug-prefix-map=  -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG
CPUINFO: OPENSSL_armcap=0xbd
                              op      op/s
 253 bits ecdh (X25519)   0.0002s   6036.6

2. Hybrid

root@raspberrypi5-c8-bf-64:~# openssl speed X25519MLKEM768
Doing X25519MLKEM768 keygen ops for 10s: 98080 X25519MLKEM768 KEM keygen ops in 9.95s
Doing X25519MLKEM768 encaps ops for 10s: 37174 X25519MLKEM768 KEM encaps ops in 9.97s
Doing X25519MLKEM768 decaps ops for 10s: 36587 X25519MLKEM768 KEM decaps ops in 10.00s
version: 3.2.6
built on: Tue Sep 30 13:05:05 2025 UTC
options: bn(64,64)
compiler: aarch64-poky-linux-gcc  -mcpu=cortex-a76+crypto -mbranch-protection=standard -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=recipe-sysroot -O2 -pipe -g -feliminate-unused-debug-types -fcanon-prefix-map  -fmacro-prefix-map=  -fdebug-prefix-map=  -fmacro-prefix-map=  -fdebug-prefix-map=  -fdebug-prefix-map=  -fmacro-prefix-map=  -fdebug-prefix-map=  -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG
CPUINFO: OPENSSL_armcap=0xbd
                               keygen    encaps    decaps keygens/s  encaps/s  decaps/s
             X25519MLKEM768 0.000101s 0.000268s 0.000273s    9857.3    3728.6    3658.7

3. PQC

root@raspberrypi5-c8-bf-64:~# openssl speed mlkem768
Doing mlkem768 keygen ops for 10s: 287919 mlkem768 KEM keygen ops in 9.94s
Doing mlkem768 encaps ops for 10s: 296703 mlkem768 KEM encaps ops in 9.94s
Doing mlkem768 decaps ops for 10s: 253984 mlkem768 KEM decaps ops in 10.00s
version: 3.2.6
built on: Tue Sep 30 13:05:05 2025 UTC
options: bn(64,64)
compiler: aarch64-poky-linux-gcc  -mcpu=cortex-a76+crypto -mbranch-protection=standard -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=recipe-sysroot -O2 -pipe -g -feliminate-unused-debug-types -fcanon-prefix-map  -fmacro-prefix-map=  -fdebug-prefix-map=  -fmacro-prefix-map=  -fdebug-prefix-map=  -fdebug-prefix-map=  -fmacro-prefix-map=  -fdebug-prefix-map=  -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG
CPUINFO: OPENSSL_armcap=0xbd
                               keygen    encaps    decaps keygens/s  encaps/s  decaps/s
                   mlkem768 0.000035s 0.000034s 0.000039s   28965.7   29849.4   25398.4

As expected Hybrid > Classical but PQC smoke classical kex :-)

[baentsch]

As expected Hybrid > Classical but PQC smoke classical kex :-)

I think there's consensus that MLKEM works pretty well by design, arguably a prime reason for its selection as a standard. But then again, on these specific code bases, please also consider whether this may be an "apples-and-oranges" comparison: This squares off code designed for as-reliable-as-possible operation in all kinds of --even possibly hostile-- environments (openssl crypto) with highly optimized assembly code primarily meant to show what may be possible/push boundaries (liboqs crypto).

[embetrix]

I think that oqs-provider it's somehow causing a bottleneck on classical operations:

if I remove .include /etc/ssl/openssl-oqs.cnf from my /etc/ssl/openssl.cnf

root@raspberrypi5-c8-bf-64:~# openssl list -providers
Providers:
  default
    name: OpenSSL Default Provider
    version: 3.2.6
    status: active

In this case:

root@raspberrypi5-c8-bf-64:~# for i in $(seq 1 1000); do curl -o /dev/null -s -w "%{time_appconnect}\n" https://localhost:443; done | awk '{sum += $1 * 1000} END {printf "Average TLS handshake: %.3f ms\n", sum/NR}'
Average TLS handshake: 31.698 ms

Enable back oqs-provider

root@raspberrypi5-c8-bf-64:~# printf "\n# Enable oqs provider config\n.include /etc/ssl/openssl-oqs.cnf\n"  >> /etc/ssl/openssl.cnf
root@raspberrypi5-c8-bf-64:~# 
root@raspberrypi5-c8-bf-64:~# openssl list -providers
Providers:
  default
    name: OpenSSL Default Provider
    version: 3.2.6
    status: active
  oqsprovider
    name: OpenSSL OQS Provider
    version: 0.11.0
    status: active

In this case:

root@raspberrypi5-c8-bf-64:~# for i in $(seq 1 1000); do curl -o /dev/null -s -w "%{time_appconnect}\n" https://localhost:443; done | awk '{sum += $1 * 1000} END {printf "Average TLS handshake: %.3f ms\n", sum/NR}'
Average TLS handshake: 63.238 ms

from another side openssl speed ecdhx25519 results are similar if oqs-provider is enabled or not

[baentsch]

oqs-provider it's somehow causing a bottleneck on classical operations:

If everything works as designed, i.e., the localhost server at port 443 indeed only accepts/uses X25519 (have you validated that also via negative testing as well as by using openssl s_time instead of the rather heavy-weight curl?) there's also the possibility that the mere (possibly oft-repeated) activation of another provider is a reason for this slow-down (have you tested with another external provider instead?) or the openssl version (tried a more recent one?).

[embetrix]

you are right, specifying the exchange method as curl option give more logical results:

x25519

root@raspberrypi5-c8-bf-64:~# for i in $(seq 1 1000); do curl --curves x25519 -o /dev/null -s -w "%{time_appconnect}\n" https://localhost:443; done | awk '{sum += $1 * 1000} END {printf "Average TLS handshake: %.3f ms\n", sum/NR}'
Average TLS handshake: 46.976 ms

X25519MLKEM768

root@raspberrypi5-c8-bf-64:~# for i in $(seq 1 1000); do curl --curves X25519MLKEM768 -o /dev/null -s -w "%{time_appconnect}\n" https://localhost:444; done | awk '{sum += $1 * 1000} END {printf "Average TLS handshake: %.3f ms\n", sum/NR}'
Average TLS handshake: 49.030 ms

mlkem768

root@raspberrypi5-c8-bf-64:~# for i in $(seq 1 1000); do curl --curves mlkem768 -o /dev/null -s -w "%{time_appconnect}\n" https://localhost:445; done | awk '{sum += $1 * 1000} END {printf "Average TLS handshake: %.3f ms\n", sum/NR}'
Average TLS handshake: 47.700 ms