-
Notifications
You must be signed in to change notification settings - Fork 584
Description
Description:
Key Facts:
- We recently attempted to upgrade to 1.5.0 from 1.3.0.
- We run in merged gateways mode
- We utilise Proxy Protocol with AWS NLB
Listener filters are the basis upon which low-level transport stream inspection and mutation occur.
As such, their ordering is highly significant and incorrect ordering can lead to unintended behaviour.
Proxy Protocol V2 (PPv2) involves a fronting intermediary (usually a load balancer) injecting a simple binary TLV sequence into the TCP bytestream to identify the originating address in an application-protocol agnostic way. The receiver is meant to consume this data and restore the bytestream before handing off to higher-layer libraries/frameworks/filters. Envoy implements PPv2 support in a listener filter.
The TLS inspector filter is also a listener filter. It is critical in scenarios where you want to do SNI matching against one or more filter chains. It inspects the early TLS ClientHello to extract the SNI extension and populates the stream context with this information.
When using these features together, it is imperative that the PPv2 filter is executed before the TLS inspector filter. Unfortunately, the latest version of Envoy Gateway seems prioritise the TLS inspector over the PPv2 listener. This is a regression from previous behaviour.
Repro steps:
- Setup a Envoy Gateway deployment using:
- Merged Gateways mode
- Multiple Gateways on a single port with TLS termination configured and appropriate HTTPRoutes
- ClientTrafficPolicy configured to expect Proxy Protocol
- Send PPv2-marked traffic to an Envoy
- This can be done with cURL locally
This should trigger afilter_chain_not_foundresult.
- This can be done with cURL locally
Environment:
[NLB with Proxy Protocol enabled] -> [Envoy Gateway with merged gateways and Proxy Protocol configured on CTP]
Working Version: 1.3.0
Regression Version: 1.5.0
Logs:
Example log under the failure scenario:
{":authority":null,"bytes_received":0,"bytes_sent":0,"connection_termination_details":null,"downstream_local_address":"10.25.27.5:443","downstream_remote_address":"12.34.56.78:18486","duration":0,"method":null,"protocol":null,"requested_server_name":null,"response_code":0,"response_code_details":"filter_chain_not_found","response_flags":"NR","route_name":null,"start_time":"2025-08-29T13:04:54.554Z","upstream_cluster":null,"upstream_host":null,"upstream_local_address":null,"upstream_transport_failure_reason":null,"user-agent":null,"x-envoy-origin-path":null,"x-envoy-upstream-service-time":null,"x-forwarded-for":null,"x-request-id":null}
With the TLS Inspector listener filter ordered before the Proxy Protocol listener, the TLS Inspector gets garbled data when inspecting the stream, and hence fails to extract the SNI. As such, requested_server_name is empty.
Config Dumps:
Working configuration from 1.3.0