CMP-4112: Move kubeletconfig symlink from /etc/kubernetes to /var/run by Vincent056 · Pull Request #1128 · ComplianceAsCode/compliance-operator

Vincent056 · 2026-03-27T15:56:30Z

Move the runtime kubeletconfig symlink path from
/etc/kubernetes/compliance-operator to /var/run/compliance-operator to avoid writing to /etc on the host node. The new path is still excluded from FIO AIDE scanning under the blanket /var exclusion.

Fixes # CMP-4112, the old path will trigger the File Integrity Operator alert on first Compliance Operator install scan

openshift-ci-robot · 2026-03-27T15:56:34Z

@Vincent056: This pull request references CMP-4112 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Move the runtime kubeletconfig symlink path from
/etc/kubernetes/compliance-operator to /var/run/compliance-operator to avoid writing to /etc on the host node. The new path is still excluded from FIO AIDE scanning under the blanket /var exclusion.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

yuumasato · 2026-03-31T10:57:09Z

/retest

Vincent056 · 2026-03-31T14:54:25Z

/retest

xiaojiey · 2026-04-02T12:12:00Z

Tested PR #1128 and PR
ComplianceAsCode/content#14606 together. Verification pass.

##1. Install FIO, create a fileintegrity.

##2. Install CO, trigger a scan, and check the /etc/kubernetes/compliance-operator/ and /var/run/compliance-operator/ on nodes:
$ oc get fileintegritynodestatus -A
NAMESPACE                  NAME                                                              NODE                                        STATUS
openshift-file-integrity   example-fileintegrity-ip-10-0-29-252.us-east-2.compute.internal   ip-10-0-29-252.us-east-2.compute.internal   Succeeded
openshift-file-integrity   example-fileintegrity-ip-10-0-5-5.us-east-2.compute.internal      ip-10-0-5-5.us-east-2.compute.internal      Succeeded
openshift-file-integrity   example-fileintegrity-ip-10-0-52-233.us-east-2.compute.internal   ip-10-0-52-233.us-east-2.compute.internal   Succeeded
openshift-file-integrity   example-fileintegrity-ip-10-0-61-156.us-east-2.compute.internal   ip-10-0-61-156.us-east-2.compute.internal   Succeeded
openshift-file-integrity   example-fileintegrity-ip-10-0-67-71.us-east-2.compute.internal    ip-10-0-67-71.us-east-2.compute.internal    Succeeded
openshift-file-integrity   example-fileintegrity-ip-10-0-88-58.us-east-2.compute.internal    ip-10-0-88-58.us-east-2.compute.internal    Succeeded
$ for node in `oc get node --no-headers | awk '{print $1}'`; do oc debug node/$node -- chroot /host bash -c 'ls -la /etc/kubernetes/compliance-operator/ 2>&1'; done
Temporary namespace openshift-debug-8q6qs is created for debugging node...
Starting pod/ip-10-0-29-252us-east-2computeinternal-debug-dhxfm ...
To use host binaries, run `chroot /host`. Instead, if you need to access host namespaces, run `nsenter -a -t 1`.
ls: cannot access '/etc/kubernetes/compliance-operator/': No such file or directory

Removing debug pod ...
Temporary namespace openshift-debug-8q6qs was removed.
...
$ for node in `oc get node --no-headers | awk '{print $1}'`; do oc debug node/$node -- chroot /host bash -c 'ls -la /var/run/compliance-operator/ 2>&1'; done
Temporary namespace openshift-debug-ljjrq is created for debugging node...
Starting pod/ip-10-0-29-252us-east-2computeinternal-debug-pq2lk ...
To use host binaries, run `chroot /host`. Instead, if you need to access host namespaces, run `nsenter -a -t 1`.
total 0
drwxr-xr-x.  2 1001 root   60 Apr  2 10:26 .
drwxr-xr-x. 60 root root 1540 Apr  2 10:26 ..
lrwxrwxrwx.  1 1001 root   14 Apr  2 10:26 kubeletconfig -> /kubeletconfig

Removing debug pod ...
Temporary namespace openshift-debug-ljjrq was removed.
...

##3. Create kubeletconfig to trigger a kubeletconfig rule fail; apply the autoremediation and rescan:
$  get ccr | grep kubelet| grep protectte
upstream-ocp4-moderate-node-master-kubelet-enable-protect-kernel-defaults                           FAIL     medium
upstream-ocp4-moderate-node-master-kubelet-enable-protect-kernel-sysctl                             PASS     medium
upstream-ocp4-moderate-node-worker-kubelet-enable-protect-kernel-defaults                           FAIL     medium
upstream-ocp4-moderate-node-worker-kubelet-enable-protect-kernel-sysctl                             PASS     medium
$ oc get cr   | grep kubelet| grep protect
upstream-ocp4-moderate-node-master-kubelet-enable-protect-kernel-defaults                           NotApplied
upstream-ocp4-moderate-node-worker-kubelet-enable-protect-kernel-defaults                           NotApplied
$ oc get mcp -w
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-b00480b7215d57be30ed10ea2e7423ca   True      False      False      3              3                   3                     0                      8h
worker   rendered-worker-c32c6600d5c5130417b44c814552cc69   True      False      False      3              3                   3                     0                      8h
master   rendered-master-b00480b7215d57be30ed10ea2e7423ca   True      False      False      3              3                   3                     0                      8h
worker   rendered-worker-c32c6600d5c5130417b44c814552cc69   True      False      False      3              3                   3                     0                      8h
master   rendered-master-b00480b7215d57be30ed10ea2e7423ca   False     True       False      3              0                   0                     0                      8h
worker   rendered-worker-c32c6600d5c5130417b44c814552cc69   False     True       False      3              0                   0                     0                      8
...
$ oc-compliance rerun-now scansettingbinding test-moderate
Rerunning scans from 'test-moderate': upstream-ocp4-moderate, upstream-ocp4-moderate-node-master, upstream-ocp4-moderate-node-worker, upstream-ocp4-moderate
Re-running scan 'openshift-compliance/upstream-ocp4-moderate'
Re-running scan 'openshift-compliance/upstream-ocp4-moderate-node-master'
Re-running scan 'openshift-compliance/upstream-ocp4-moderate-node-worker'
Re-running scan 'openshift-compliance/upstream-ocp4-moderate'
$ oc get suite test-moderate
NAME            PHASE   RESULT
test-moderate   DONE    NON-COMPLIANT
$ oc get ccr | grep kubelet | grep protect
upstream-ocp4-moderate-node-master-kubelet-enable-protect-kernel-defaults                           PASS     medium
upstream-ocp4-moderate-node-master-kubelet-enable-protect-kernel-sysctl                             PASS     medium
upstream-ocp4-moderate-node-worker-kubelet-enable-protect-kernel-defaults                           PASS     medium
upstream-ocp4-moderate-node-worker-kubelet-enable-protect-kernel-sysctl                             PASS     medium

##4. Check fileintegrity and check the /etc/kubernetes/compliance-operator/ and /var/run/compliance-operator/ on nodes
$ oc get fileintegritynodestatus -n openshift-file-integrity
NAME                                                              NODE                                        STATUS
example-fileintegrity-ip-10-0-29-252.us-east-2.compute.internal   ip-10-0-29-252.us-east-2.compute.internal   Succeeded
example-fileintegrity-ip-10-0-5-5.us-east-2.compute.internal      ip-10-0-5-5.us-east-2.compute.internal      Succeeded
example-fileintegrity-ip-10-0-52-233.us-east-2.compute.internal   ip-10-0-52-233.us-east-2.compute.internal   Succeeded
example-fileintegrity-ip-10-0-61-156.us-east-2.compute.internal   ip-10-0-61-156.us-east-2.compute.internal   Succeeded
example-fileintegrity-ip-10-0-67-71.us-east-2.compute.internal    ip-10-0-67-71.us-east-2.compute.internal    Succeeded
example-fileintegrity-ip-10-0-88-58.us-east-2.compute.internal    ip-10-0-88-58.us-east-2.compute.internal    Succeeded

yuumasato · 2026-04-02T13:09:17Z

@Vincent056 Do we expect kubletconfig test to be failing?

 2026/03/31 16:48:41 waiting until suite kubelet-remediation-test-suite-node reaches target status 'DONE'. Current status: AGGREGATING
    main_test.go:1746: expecting one of [COMPLIANT] got NON-COMPLIANT
--- FAIL: TestKubeletConfigRemediation (255.25s)

Vincent056 · 2026-04-02T15:10:39Z

@Vincent056 Do we expect kubletconfig test to be failing?

 2026/03/31 16:48:41 waiting until suite kubelet-remediation-test-suite-node reaches target status 'DONE'. Current status: AGGREGATING
    main_test.go:1746: expecting one of [COMPLIANT] got NON-COMPLIANT
--- FAIL: TestKubeletConfigRemediation (255.25s)

I need to update the test image, let me see if I can build locally for that tag

compliance-operator/tests/e2e/serial/main_test.go

Line 1635 in 5746e95

    
           var baselineImage = fmt.Sprintf("%s:%s", brokenContentImagePath, "new_kubeletconfig")

yuumasato · 2026-04-02T16:01:42Z

/retest

Move the runtime kubeletconfig symlink path from /etc/kubernetes/compliance-operator to /var/run/compliance-operator to avoid writing to /etc on the host node. The new path is still excluded from FIO AIDE scanning under the blanket /var exclusion.

Vincent056 · 2026-04-03T05:39:54Z

/retest

github-actions · 2026-04-03T05:48:03Z

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1128-cdc1aebf2821110da5f05235d100b422b9430ded

xiaojiey · 2026-04-03T09:11:14Z

/retest

xiaojiey · 2026-04-03T09:13:20Z

The serial test cases failed due to PB failed to reach state VALID

2026/04/03 08:05:20 waiting ProfileBundle test-kubelet-config-remediation to become VALID (PENDING)
    main_test.go:1646: ProfileBundle test-kubelet-config-remediation failed to reach state VALID
--- FAIL: TestKubeletConfigRemediation (1800.02s)
=== RUN   TestSuspendScanSetting
...
2026/04/03 08:40:06 waiting ProfileBundle test-scan-deprecated-profile to become VALID (PENDING)
    main_test.go:2307: failed waiting for the ProfileBundle to become available: ProfileBundle test-scan-deprecated-profile failed to reach state VALID
--- FAIL: TestScanDeprecatedProfile (1800.03s)

And the parallel test cases failed for the same reason as well:

2026/04/03 10:34:18 waiting ProfileBundle test-scan-setting-binding-tailoring-many-enabling-rule-pass to become VALID (PENDING)
2026/04/03 10:34:18 waiting ProfileBundle test-scan-setting-binding-tailoring-many-enabling-rule-pass to become VALID (PENDING)
=== NAME  TestScanSettingBindingTailoringManyEnablingRulePass
    main_test.go:4796: failed to parse ProfileBundle test-scan-setting-binding-tailoring-many-enabling-rule-pass: ProfileBundle test-scan-setting-binding-tailoring-many-enabling-rule-pass failed to reach state VALID
--- FAIL: TestScanSettingBindingTailoringManyEnablingRulePass (1830.12s)
...
2026/04/03 10:34:29 waiting ProfileBundle test-parsing-error-restarts-parser-init-container to become VALID (INVALID)
=== NAME  TestParsingErrorRestartsParserInitContainer
    main_test.go:545: ProfileBundle test-parsing-error-restarts-parser-init-container failed to reach state VALID
--- FAIL: TestParsingErrorRestartsParserInitContainer

github-actions · 2026-04-03T22:48:15Z

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1128-ca6c130e4a86e6dc166b8d227eb9cc399f060c76

Vincent056 · 2026-04-06T07:37:08Z

/retest

Vincent056 · 2026-04-06T12:13:08Z

/retest

openshift-ci · 2026-04-06T12:47:52Z

@Vincent056: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-rosa	`ca6c130`	link	true	`/test e2e-rosa`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

rhmdnd

One comment on cleaning up the old directory. We could remove the cleanup logic in a subsequent release (1.10.0?), but it would leave the nodes in a cleaner state.

Thoughts?

pkg/controller/compliancescan/config.go

rhmdnd

/lgtm

openshift-ci · 2026-04-07T14:44:09Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rhmdnd, Vincent056

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [Vincent056,rhmdnd]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…ComplianceAsCode#1128) * CMP-4112: Move kubeletconfig symlink from /etc/kubernetes to /var/run Move the runtime kubeletconfig symlink path from /etc/kubernetes/compliance-operator to /var/run/compliance-operator to avoid writing to /etc on the host node. The new path is still excluded from FIO AIDE scanning under the blanket /var exclusion. * Update bundle file

…#1128) (#1164) * CMP-4112: Move kubeletconfig symlink from /etc/kubernetes to /var/run Move the runtime kubeletconfig symlink path from /etc/kubernetes/compliance-operator to /var/run/compliance-operator to avoid writing to /etc on the host node. The new path is still excluded from FIO AIDE scanning under the blanket /var exclusion. * Update bundle file

openshift-ci-robot added the jira/valid-reference label Mar 27, 2026

openshift-ci bot requested review from mrogers950 and rhmdnd March 27, 2026 15:56

openshift-ci bot added the approved label Mar 27, 2026

xiaojiey mentioned this pull request Apr 2, 2026

CMP-4112: Update kubeletconfig OVAL path to /var/run/compliance-operator ComplianceAsCode/content#14606

Merged

yuumasato added this to the 1.9.0 milestone Apr 2, 2026

Vincent056 force-pushed the CMP-4112-move-kubeletconfig-symlink branch from 4b0497f to cdc1aeb Compare April 3, 2026 05:40

Update bundle file

ca6c130

This comment was marked as duplicate.

Sign in to view

rhmdnd requested changes Apr 6, 2026

View reviewed changes

pkg/controller/compliancescan/config.go Show resolved Hide resolved

openshift-ci bot assigned rhmdnd Apr 6, 2026

ComplianceAsCode deleted a comment from not-abot-it Apr 6, 2026

xiaojiey mentioned this pull request Apr 7, 2026

CMP-3916: Add runtime SSH configuration for compliance scans #961

Merged

rhmdnd approved these changes Apr 7, 2026

View reviewed changes

openshift-ci bot added the lgtm label Apr 7, 2026

rhmdnd merged commit 5b8bb46 into ComplianceAsCode:master Apr 7, 2026
20 of 26 checks passed

Vincent056 mentioned this pull request Apr 7, 2026

CMP-4112: cherrypick Move kubeletconfig symlink from /etc/kubernetes to /var/run #1164

Merged

Conversation

Vincent056 commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Mar 27, 2026

Uh oh!

yuumasato commented Mar 31, 2026

Uh oh!

Vincent056 commented Mar 31, 2026

Uh oh!

xiaojiey commented Apr 2, 2026

Uh oh!

yuumasato commented Apr 2, 2026

Uh oh!

Vincent056 commented Apr 2, 2026

Uh oh!

yuumasato commented Apr 2, 2026

Uh oh!

Vincent056 commented Apr 3, 2026

Uh oh!

github-actions bot commented Apr 3, 2026

Uh oh!

xiaojiey commented Apr 3, 2026

Uh oh!

xiaojiey commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 3, 2026

Uh oh!

Vincent056 commented Apr 6, 2026

Uh oh!

This comment was marked as duplicate.

Vincent056 commented Apr 6, 2026

Uh oh!

openshift-ci bot commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rhmdnd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rhmdnd left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Apr 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Vincent056 commented Mar 27, 2026 •

edited

Loading

xiaojiey commented Apr 3, 2026 •

edited

Loading

openshift-ci bot commented Apr 6, 2026 •

edited

Loading