MBEDTLS_ERR_SSL_FEATURE_UNAVAILABLE occurs specifically when server accepts an SDP Answer from Firefox with local testing

[silbinarywolf]

Update (2026-02-27):
This behaviour with Firefox seems to be new-ish as of Nov/Dec 2025 and its caused the lack of DTLS fragmented packets in mbedtls to be more common.

The related mbedtls issue here: Mbed-TLS/mbedtls#7549
There's a fix backported to 3.6 here: Mbed-TLS/mbedtls#10623

What is the issue?

I've breakpointed the issue to mbedtls v3.6.5 not supporting fragmented ClientHello in TLS 1.2 when accepting an SDP answer from a Firefox client after trying to negotiate two tracks.

mbedtls/library/ssl_tls12_server.c

# From debugger panel when testing with Firefox
fragment_offset = 0
fragment_length = 1383
length = 1461

# From debugger panel when testing with Chrome
fragment_offset = 0
fragment_length = 191
length = 191

if (fragment_offset != 0 || length != fragment_length) {
    MBEDTLS_SSL_DEBUG_MSG(1, ("ClientHello fragmentation not supported"));
    return MBEDTLS_ERR_SSL_FEATURE_UNAVAILABLE;
}

library/ssl_tls.c

#if defined(MBEDTLS_SSL_SRV_C)
    if (ssl->conf->endpoint == MBEDTLS_SSL_IS_SERVER) {
#if defined(MBEDTLS_SSL_PROTO_TLS1_2) && defined(MBEDTLS_SSL_PROTO_TLS1_3)
        if (ssl->tls_version == MBEDTLS_SSL_VERSION_TLS1_3) {
            ret = mbedtls_ssl_tls13_handshake_server_step(ssl);
        } else {
>>>>        ret = mbedtls_ssl_handshake_server_step(ssl); // <- In my breakpointing this is the code path taken
        }
#elif defined(MBEDTLS_SSL_PROTO_TLS1_2)
        ret = mbedtls_ssl_handshake_server_step(ssl);
#else
        ret = mbedtls_ssl_tls13_handshake_server_step(ssl);
#endif
    }
#endif /* MBEDTLS_SSL_SRV_C */

libdatachannel/src/impl/dtlstransport.cpp

while (true) {
	int ret;
	{
		std::lock_guard lock(mSslMutex);
		ret = mbedtls_ssl_handshake(&mSsl);
	}

Other notes

• So far I can't reproduce this issue on Chromium.

Possibly related issues

• Force TLS 1.2 to resolve similar error code but for TLS 1.3
  • TLS handshale failure with Mbed TLS 3.6.0 #1147 (comment)

Firefox SDP answer (1451 bytes)

180.420.69.50 = my external IP address at the time, modified it to anonymize my IP

Click to expand/collapse

v=0
o=mozilla...THIS_IS_SDPARTA-99.0 2964469810652852766 0 IN IP4 0.0.0.0
s=-
t=0 0
a=sendrecv
a=fingerprint:sha-256 2B:35:1E:D9:3D:64:15:57:EA:C6:91:9B:9C:AF:79:EA:F7:35:03:3F:9A:BB:21:46:50:E0:5F:AC:4A:D8:8A:74
a=group:BUNDLE mic-1 video-1
a=ice-options:trickle
a=msid-semantic:WMS *
m=audio 56419 UDP/TLS/RTP/SAVPF 111
c=IN IP4 180.420.69.50
a=candidate:0 1 UDP 2122252543 192.168.0.165 56419 typ host
a=candidate:2 1 UDP 2122187007 192.168.68.1 33239 typ host
a=candidate:4 1 TCP 2105524479 192.168.0.165 9 typ host tcptype active
a=candidate:5 1 TCP 2105458943 192.168.68.1 9 typ host tcptype active
a=candidate:1 1 UDP 1686052863 180.420.69.50 56419 typ srflx raddr 192.168.0.165 rport 56419
a=candidate:3 1 UDP 1685987327 180.420.69.50 33239 typ srflx raddr 192.168.68.1 rport 33239
a=recvonly
a=end-of-candidates
a=fmtp:111 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=ice-pwd:300fae07c4503ffc1088553a7cb74435
a=ice-ufrag:9562017c
a=mid:mic-1
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=setup:active
a=ssrc:2940946743 cname:{254ece63-f409-4a8b-90d6-4c0e48e73958}
m=video 56419 UDP/TLS/RTP/SAVPF 96
c=IN IP4 180.420.69.50
a=recvonly
a=fmtp:96 profile-level-id=42000a;level-asymmetry-allowed=1
a=ice-pwd:300fae07c4503ffc1088553a7cb74435
a=ice-ufrag:9562017c
a=mid:video-1
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=rtcp-mux
a=rtpmap:96 H264/90000
a=setup:active
a=ssrc:2113180387 cname:{254ece63-f409-4a8b-90d6-4c0e48e73958}

Chrome SDP answer (1305 bytes)

Click to expand/collapse

v=0
o=- 5527426653086694778 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE mic-1 video-1
a=msid-semantic: WMS
m=audio 36318 UDP/TLS/RTP/SAVPF 111
c=IN IP4 180.420.69.50
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:1430034131 1 udp 2122260223 192.168.0.165 36318 typ host generation 0 network-id 1
a=candidate:4202067949 1 udp 1686052607 180.420.69.50 36318 typ srflx raddr 192.168.0.165 rport 36318 generation 0 network-id 1
a=candidate:737375307 1 tcp 1518280447 192.168.0.165 9 typ host tcptype active generation 0 network-id 1
a=ice-ufrag:54QM
a=ice-pwd:3jW5WpNg9EysaYRmY/r8ezAM
a=ice-options:trickle
a=fingerprint:sha-256 0D:92:C1:02:AD:0F:D3:D8:7A:1C:92:DE:23:74:2B:85:B1:85:B6:80:26:4F:A3:9C:6E:DA:8B:43:08:6D:B8:9D
a=setup:active
a=mid:mic-1
a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10;useinbandfec=1
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:54QM
a=ice-pwd:3jW5WpNg9EysaYRmY/r8ezAM
a=ice-options:trickle
a=fingerprint:sha-256 0D:92:C1:02:AD:0F:D3:D8:7A:1C:92:DE:23:74:2B:85:B1:85:B6:80:26:4F:A3:9C:6E:DA:8B:43:08:6D:B8:9D
a=setup:active
a=mid:video-1
a=recvonly
a=rtcp-mux
a=rtpmap:96 H264/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f

[paullouisageneau]

Thank you for investigating and reporting! However, it looks like an mbedTLS issue, the only way to work around it in libdatachannel would be to reconstruct the ClientHello before forwarding it to mbedTLS. Is my understanding correct?

[silbinarywolf]

Yeah that sounds correct to me.

The reason I initially raised it here was because I didn't understand it was an mbedTLS issue, so if you want to close this, I personally do not mind. For now I'm just running the aforementioned fork that handles fragmented ClientHello messages.