Skip to content

Conversation

@08volt
Copy link
Member

@08volt 08volt commented Nov 21, 2025

Objective

Improve observability of L4 ILB and NetLB services by propagating the names of underlying GCP resources directly into Service.Status.Conditions.

Design

Introduces a new feature gate, EnableL4LBConditions (default: false), to control the population of these conditions.

When enabled, the L4 controller sync loop tracks the provisioning success of individual GCP components. The L4ILBSyncResult and L4NetLBSyncResult structs has been extended to accumulate metav1.Condition updates throughout the sync lifecycle.

Key Changes:

Feature Flag: Added --enable-l4lb-svc-conditions to flags.go and propagated it through the ControllerContext.

Condition Management:

Leverages k8s.io/apimachinery/pkg/api/meta to Set/Remove conditions.

State Transitions: The logic correctly handles protocol updates (e.g., switching a Service from TCP to UDP) by explicitly removing conditions associated with the previous protocol to prevent stale status reporting.

Implementation Details

  • Modified ensureIPv4Resources and ensureInternalLoadBalancer in pkg/l4resources to return specific conditions upon resource creation.
  • Refactored L4ILBSyncResult and L4NetLBSyncResult to initialize from existing Service conditions and merge updates.
  • Updated updateServiceStatus to persist these conditions to the API server.

Test Plan

Added coverage in l4_test.go and l4netlb_test.go to verify:

  • Conditions are correctly set when resources are created.
  • Conditions are removed when resources are deleted or protocols change.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 21, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 08volt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 21, 2025
@08volt 08volt force-pushed the svc-health-condition branch 2 times, most recently from 54d68bc to f8f2767 Compare November 21, 2025 11:44
@08volt
Copy link
Member Author

08volt commented Nov 21, 2025

/retest

@08volt 08volt force-pushed the svc-health-condition branch from f8f2767 to 1b97b12 Compare November 21, 2025 13:01
@08volt
Copy link
Member Author

08volt commented Nov 21, 2025

/retest

@08volt
Copy link
Member Author

08volt commented Nov 21, 2025

/assign @TortillaZHawaii

Copy link
Member

@TortillaZHawaii TortillaZHawaii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First round


if ipv6fr.IPProtocol == string(corev1.ProtocolTCP) {
syncResult.Annotations[annotations.TCPForwardingRuleIPv6Key] = ipv6fr.Name
metaapi.SetStatusCondition(&syncResult.Conditions, utils.NewTCPIPv6ForwardingRuleCondition(ipv6fr.Name))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the api has to be iterative? Can we have declarative, similar to what is set above for the annotations? meaning that we would not care about deleting existing ones, but just always set whatever was there

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be the official/safe way to handle conditions

Copy link
Member

@TortillaZHawaii TortillaZHawaii Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we hide "official/safe" way in some kind of abstraction? I'm wondering if we could just get away with some struct that would read the annotations which are set in the result structs. Based on the annotations map it could create conditions and calculate the diff between the got set of conditions and the wanted set, perform "official/safe" condition updates, and in the end make a PATCH to k8s API.

In the current form we have a lot of messy code, especially with Forwarding Rules, where we need to delete 2 other variants. In my opinion, if another resource comes along, it will become dreadful to find every place that needs to be updated for both conditions and annotations to work, especially given that you need to think about all the possible previous states of the LB. I worry, that if someone does something that is supported, but is non obvious (for example going from IPv4 to IPv6, or ILB to NetLB), current approach would result with leftover or conflicting conditions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats right, I kept the slice type but change the initilization to empty slice

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted it back, as the controller should also make sure that other conditions in the service are not deleted. The updates mentioned should be tested

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they are tested, we can check that the conditions in the service are always only the one we expect to have

@08volt 08volt force-pushed the svc-health-condition branch 2 times, most recently from 6e572b5 to 2624378 Compare November 24, 2025 13:04
@08volt 08volt changed the title Title: feat(l4): expose GCP resource state via Service Status Conditions Title: feat(L4): expose GCP resource state via Service Status Conditions Nov 24, 2025
@08volt 08volt changed the title Title: feat(L4): expose GCP resource state via Service Status Conditions feat(L4): expose GCP resource state via Service Status Conditions Nov 24, 2025
@08volt 08volt force-pushed the svc-health-condition branch 3 times, most recently from 432ec08 to 6c9bd7a Compare November 27, 2025 15:56
@08volt 08volt force-pushed the svc-health-condition branch from 6c9bd7a to 6548129 Compare November 27, 2025 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants