Add support for backend tls config for Gateways #3900

salonichf5 · 2025-09-12T15:25:22Z

Proposed changes

Write a clear and concise description that helps reviewers understand the purpose and impact of your changes. Use the
following format:

Problem: Users want to be able to specify their gateway's identity when communicating to the backend pods.

Solution: Add a field to provide a secret name that stores that gateways cert and key to be used when doing TLS handshake with backend pods

NOTE: I refactored this test because it was not easy to debug in such large tests as part of this PR.

Ran unit tests multiple times to avoid data race situations.

Some fields conform to the Gateway API 1.3. They need to be updated once Gateway API release 1.4 -- depending how far that is i'll decide if i should wait for merging or merge now , update it later

Testing: Unit tests added as needed , No IPv6 testing done for this (not related with ports or in container network)

Manual tests

Tested with Securing backend traffic by additionally requiring client certificates from backend and ensuring they are signed by the right CN to verify identity

my secure-app config

    server {
        listen 8443 ssl;
        listen [::]:8443 ssl;

        server_name secure-app.example.com;

        default_type text/plain;

        ssl_certificate /etc/nginx/ssl/secret/tls.crt;
        ssl_certificate_key /etc/nginx/ssl/secret/tls.key;

        ssl_client_certificate /etc/nginx/ssl/ca-cert/ca.crt;
        ssl_verify_client on;


        # Enable access logging
        access_log /var/log/nginx/access.log ssl_log;


        location / {
            return 200 "hello from pod secure-app\n";
        }
    }

NGF config

# Gateway Certificate
proxy_ssl_certificate /etc/nginx/secrets/ssl_keypair_default_gateway-presents-this-cert-for-validation.pem;
proxy_ssl_certificate_key /etc/nginx/secrets/ssl_keypair_default_gateway-presents-this-cert-for-validation.pem;


js_preload_object matches from /etc/nginx/conf.d/matches.json;
server {
    listen 80 default_server;
    listen [::]:80 default_server;
    default_type text/html;
    return 404;
}

server {
    listen 80;
    listen [::]:80;

    server_name secure-app.example.com;


    location / {






        proxy_http_version 1.1;
        proxy_set_header Host "$gw_api_compliant_host";
        proxy_set_header X-Forwarded-For "$proxy_add_x_forwarded_for";
        proxy_set_header X-Real-IP "$remote_addr";
        proxy_set_header X-Forwarded-Proto "$scheme";
        proxy_set_header X-Forwarded-Host "$host";
        proxy_set_header X-Forwarded-Port "$server_port";
        proxy_set_header Upgrade "$http_upgrade";
        proxy_set_header Connection "$connection_upgrade";
        proxy_pass https://default_secure-app_8443$request_uri;



        proxy_ssl_server_name on;
        proxy_ssl_verify on;
        proxy_ssl_name secure-app.example.com;
        proxy_ssl_trusted_certificate /etc/nginx/secrets/cert_bundle_default_backend-cert.crt;
    }
}

Curl and logs to verify client -- I signed CA with gateway CN so that's what we should see get logged in the backend

curl -v --resolve secure-app.example.com:$GW_PORT:$GW_IP http://secure-app.example.com:$GW_PORT/
hello from pod secure-app

k logs secure-app-69c558d9d9-lwcjl
10.244.0.117 ssl_client_verify=SUCCESS ssl_client_subject=CN=gateway
10.244.0.117 ssl_client_verify=SUCCESS ssl_client_subject=CN=gateway

Please focus on (optional): If you any specific areas where you would like reviewers to focus their attention or provide
specific feedback, add them here.

Closes #3153

Checklist

Before creating a PR, run through this checklist and mark each as complete.

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works
I have checked that all unit tests pass after adding my changes
I have updated necessary documentation
I have rebased my branch onto main
I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

Added support for configuring backend TLS on Gateways to enable secure communication between the gateway and pods.

codecov · 2025-09-12T15:28:11Z

Codecov Report

❌ Patch coverage is 77.77778% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.80%. Comparing base (635b3fc) to head (8168ac4).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/controller/state/conditions/conditions.go	0.00%	14 Missing ⚠️
internal/controller/manager.go	0.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3900      +/-   ##
==========================================
- Coverage   86.85%   86.80%   -0.06%     
==========================================
  Files         128      128              
  Lines       16503    16569      +66     
  Branches       62       62              
==========================================
+ Hits        14334    14382      +48     
- Misses       1992     2009      +17     
- Partials      177      178       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull Request Overview

This PR adds support for backend TLS configuration on Gateways to enable secure communication between the gateway and backend pods using client certificates. The implementation includes validation logic, configuration building, and nginx template updates to support TLS handshake with client certificate authentication.

Adds experimental backend TLS support via Gateway.Spec.BackendTLS.ClientCertificateRef
Implements validation and reference grant checking for cross-namespace secret references
Updates NGINX configuration to include gateway client certificates in the proxy configuration

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
internal/controller/state/graph/gateway.go	Core gateway validation logic for backend TLS configuration
internal/controller/state/graph/gateway_test.go	Comprehensive test coverage for backend TLS validation scenarios
internal/controller/state/dataplane/configuration.go	SSL key pair building and base HTTP config updates
internal/controller/nginx/config/base_http_config_template.go	NGINX template for gateway client certificates
internal/controller/state/conditions/conditions.go	New condition types for secret reference validation
internal/controller/state/graph/graph.go	Graph building updates to pass experimental features flag

Comments suppressed due to low confidence (1)

internal/controller/state/dataplane/configuration.go:1

Inconsistent formatting with extra comma and closing parenthesis on separate lines. Should follow the existing pattern of having the closing parenthesis on the same line as the last parameter.

package dataplane

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

internal/controller/state/graph/gateway_test.go

internal/controller/state/dataplane/configuration.go

Copilot

Pull Request Overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

internal/controller/state/graph/gateway_test.go

internal/controller/state/dataplane/configuration_test.go

salonichf5 · 2025-09-15T21:59:17Z

internal/controller/state/conditions/conditions.go

+func NewGatewaySecretRefNotPermitted(msg string) Condition {
+	return Condition{
+		Type:    string(GatewayReasonResolvedRefs),
+		Status:  metav1.ConditionFalse,
+		Reason:  string(GatewayReasonSecretRefNotPermitted),
+		Message: msg,
+	}
+}
+
+// NewGatewaySecretRefInvalid returns Condition that indicates that the Gateway references a TLS secret that is invalid.
+func NewGatewaySecretRefInvalid(msg string) Condition {
+	return Condition{
+		Type:    string(GatewayReasonResolvedRefs),
+		Status:  metav1.ConditionFalse,
+		Reason:  string(GatewayReasonSecretRefInvalid),
+		Message: msg,
+	}
+}
+


currently working on the bug related to marking HTTPRoute invalid leading to issues and I am wondering if this is still valid? Should we mark the gateway invalid? its not syntactically wrong for it to be marked false. We just won't reference the secret in the our configuration so I would like to change this to Accepted: true but i'd like to get second opinions.

The Gateway API community may be able to clarify for sure, but I'm fairly confident this should be Accepted: false. If a user wants a secure connection, we definitely shouldn't ignore that if the Secret is invalid. That would defeat the purpose of a secure connection if we just default to plaintext and insecure if their certs aren't able to be used.

sjberman · 2025-09-16T15:56:12Z

internal/controller/state/conditions/conditions.go

+
+	// GatewayReasonSecretRefNotPermitted is used with the "GatewayResolvedRefs" condition when the
+	// secretRef resource is not permitted by any ReferenceGrant.
+	GatewayReasonSecretRefNotPermitted v1.GatewayConditionReason = "SecretRefNotPermitted"


Are these available or will these be available to import from the gateway api?

No, these weren't available on gateway api only reason related to parametersRef so didn't want to use those to avoid confusion

internal/controller/state/graph/gateway.go

sjberman · 2025-09-16T16:01:34Z

internal/controller/state/graph/graph_test.go

-			name:     "gatewayclass belongs to a different controller",
+			store:        createStateWithGatewayClass(differentControllerGC),
+			expected:     &Graph{},
+			experimental: true,


Why set this to true here? Is it really changing anything?

Did this for code coverage. Does not change anything but that's also a path to check

internal/controller/state/graph/multiple_gateways_test.go

sjberman · 2025-09-16T16:23:53Z

internal/controller/state/dataplane/configuration_test.go

+			assertBuildConfiguration(g, result, test.expConf)
+		})
+	}
+}


Thanks for cleaning this up a bit. But holy cow, nearly 5500 lines. This is getting unwieldy.

sjberman · 2025-09-18T18:15:48Z

internal/controller/state/graph/graph_test.go

-			name:         "gatewayclass belongs to a different controller",
+			store:               createStateWithGatewayClass(differentControllerGC),
+			expected:            &Graph{},
+			experimentalEnabled: true,


It would be nice if we could actually verify that this does something, instead of purely trying to get code coverage

github-project-automation bot added this to NGINX Gateway Fabric Sep 12, 2025

nginx-bot bot added the release-notes label Sep 12, 2025

github-project-automation bot moved this to 🆕 New in NGINX Gateway Fabric Sep 12, 2025

github-actions bot added the enhancement New feature or request label Sep 12, 2025

salonichf5 self-assigned this Sep 12, 2025

salonichf5 force-pushed the feat/gateway-backendTLS branch from 6e0570b to 81d973e Compare September 12, 2025 16:04

salonichf5 requested a review from Copilot September 12, 2025 16:11

Copilot AI reviewed Sep 12, 2025

View reviewed changes

internal/controller/state/graph/gateway_test.go Outdated Show resolved Hide resolved

internal/controller/state/dataplane/configuration.go Show resolved Hide resolved

salonichf5 requested a review from Copilot September 12, 2025 16:24

Copilot AI reviewed Sep 12, 2025

View reviewed changes

internal/controller/state/graph/gateway_test.go Show resolved Hide resolved

internal/controller/state/graph/gateway_test.go Show resolved Hide resolved

internal/controller/state/dataplane/configuration_test.go Show resolved Hide resolved

salonichf5 commented Sep 15, 2025

View reviewed changes

sjberman reviewed Sep 16, 2025

View reviewed changes

salonichf5 added 3 commits September 16, 2025 11:04

add tls.backend field to specify gateway cert

6bb7974

update unit tests

e351248

code reviews

8168ac4

salonichf5 force-pushed the feat/gateway-backendTLS branch from af9ad2e to 8168ac4 Compare September 16, 2025 17:04

salonichf5 requested a review from sjberman September 18, 2025 16:11

update test paramaters name

64dc1b0

sjberman reviewed Sep 18, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add support for backend tls config for Gateways #3900

Add support for backend tls config for Gateways #3900

salonichf5 commented Sep 12, 2025 •

edited

Loading

Uh oh!

codecov bot commented Sep 12, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

salonichf5 Sep 15, 2025

Uh oh!

sjberman Sep 15, 2025

Uh oh!

sjberman Sep 16, 2025

Uh oh!

salonichf5 Sep 16, 2025

Uh oh!

Uh oh!

sjberman Sep 16, 2025

Uh oh!

salonichf5 Sep 16, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

sjberman Sep 16, 2025

Uh oh!

sjberman Sep 18, 2025

Uh oh!

Uh oh!

Add support for backend tls config for Gateways #3900

Are you sure you want to change the base?

Add support for backend tls config for Gateways #3900

Conversation

salonichf5 commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Checklist

Release notes

Uh oh!

codecov bot commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

salonichf5 Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

sjberman Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

sjberman Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

salonichf5 Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sjberman Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

salonichf5 Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sjberman Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

sjberman Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

salonichf5 commented Sep 12, 2025 •

edited

Loading

codecov bot commented Sep 12, 2025 •

edited

Loading

salonichf5 Sep 16, 2025 •

edited

Loading