CMP-3799: Add e2e suite for odd tests for pre-release testing

[Anna-Koudelkova]

Adding a prerelease testing suite (name is debatable, I was not able to come up with anything better, will taky any advice here) for the test cases that are not directly related to the CO code, but still need to be tested before release, such as DAST tests, stability or profile support on different architectures.

Added a testcase for the latest for start (OCP-45729 in the downstream suite). Test case is passing on OCP 4.21:

=== RUN   TestProfilesOnArch
    main_test.go:36: Detected cluster architecture: amd64
2026/03/23 13:22:48 ProfileBundle ready (VALID)
    main_test.go:42: ProfileBundle ocp4 is VALID
2026/03/23 13:22:53 ProfileBundle ready (VALID)
    main_test.go:49: ProfileBundle rhcos4 is VALID
    main_test.go:68: Profile ocp4-cis exists
    main_test.go:68: Profile ocp4-cis-1-7 exists
    main_test.go:68: Profile ocp4-cis-node exists
    main_test.go:68: Profile ocp4-cis-node-1-7 exists
    main_test.go:68: Profile ocp4-e8 exists
    main_test.go:68: Profile ocp4-high exists
    main_test.go:68: Profile ocp4-high-node exists
    main_test.go:68: Profile ocp4-high-node-rev-4 exists
    main_test.go:68: Profile ocp4-high-rev-4 exists
    main_test.go:68: Profile ocp4-moderate exists
    main_test.go:68: Profile ocp4-moderate-node exists
    main_test.go:68: Profile ocp4-moderate-node-rev-4 exists
    main_test.go:68: Profile ocp4-moderate-rev-4 exists
    main_test.go:68: Profile ocp4-nerc-cip exists
    main_test.go:68: Profile ocp4-nerc-cip-node exists
    main_test.go:68: Profile ocp4-pci-dss exists
    main_test.go:68: Profile ocp4-pci-dss-3-2 exists
    main_test.go:68: Profile ocp4-pci-dss-4-0 exists
    main_test.go:68: Profile ocp4-pci-dss-node exists
    main_test.go:68: Profile ocp4-pci-dss-node-3-2 exists
    main_test.go:68: Profile ocp4-pci-dss-node-4-0 exists
    main_test.go:68: Profile ocp4-stig exists
    main_test.go:68: Profile ocp4-stig-node exists
    main_test.go:68: Profile ocp4-stig-node-v2r2 exists
    main_test.go:68: Profile ocp4-stig-node-v2r3 exists
    main_test.go:68: Profile ocp4-stig-v2r2 exists
    main_test.go:68: Profile ocp4-stig-v2r3 exists
    main_test.go:68: Profile rhcos4-e8 exists
    main_test.go:68: Profile rhcos4-high exists
    main_test.go:68: Profile rhcos4-high-rev-4 exists
    main_test.go:68: Profile rhcos4-moderate exists
    main_test.go:68: Profile rhcos4-moderate-rev-4 exists
    main_test.go:68: Profile rhcos4-nerc-cip exists
    main_test.go:68: Profile rhcos4-stig exists
    main_test.go:68: Profile rhcos4-stig-v2r2 exists
    main_test.go:68: Profile rhcos4-stig-v2r3 exists
    main_test.go:68: Profile ocp4-bsi exists
    main_test.go:68: Profile ocp4-bsi-2022 exists
    main_test.go:68: Profile ocp4-bsi-node exists
    main_test.go:68: Profile ocp4-bsi-node-2022 exists
    main_test.go:70: All 40 expected profiles exist for architecture amd64
--- PASS: TestProfilesOnArch (103.29s)
PASS
...
ok  	github.com/ComplianceAsCode/compliance-operator/tests/e2e/prerelease	384.139s

The next logical step would be to set up a prow job with f999 frequency (that is what it was marked as in the downstream repo) so that the job never runs unless triggered and wire it up to be part of the integration tests.

Assisted-by: Cursor

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Anna-Koudelkova

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [Anna-Koudelkova]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[github-actions]

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1111-5bf01e446e384394fdec8314f880717d21f6d1c7

[openshift-ci]

@Anna-Koudelkova: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-rosa	5bf01e4	link	true	/test e2e-rosa
ci/prow/e2e-aws-parallel-arm	5bf01e4	link	true	/test e2e-aws-parallel-arm
ci/prow/e2e-aws-parallel	5bf01e4	link	true	/test e2e-aws-parallel

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-ci-robot]

@Anna-Koudelkova: This pull request references CMP-3799 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target either version "4.22." or "openshift-4.22.", but it targets "compliance-operator-1.9.0" instead.

Details

In response to this:

Adding a prerelease testing suite (name is debatable, I was not able to come up with anything better, will taky any advice here) for the test cases that are not directly related to the CO code, but still need to be tested before release, such as DAST tests, stability or profile support on different architectures.

Added a testcase for the latest for start (OCP-45729 in the downstream suite). Test case is passing on OCP 4.21:

=== RUN   TestProfilesOnArch
   main_test.go:36: Detected cluster architecture: amd64
2026/03/23 13:22:48 ProfileBundle ready (VALID)
   main_test.go:42: ProfileBundle ocp4 is VALID
2026/03/23 13:22:53 ProfileBundle ready (VALID)
   main_test.go:49: ProfileBundle rhcos4 is VALID
   main_test.go:68: Profile ocp4-cis exists
   main_test.go:68: Profile ocp4-cis-1-7 exists
   main_test.go:68: Profile ocp4-cis-node exists
   main_test.go:68: Profile ocp4-cis-node-1-7 exists
   main_test.go:68: Profile ocp4-e8 exists
   main_test.go:68: Profile ocp4-high exists
   main_test.go:68: Profile ocp4-high-node exists
   main_test.go:68: Profile ocp4-high-node-rev-4 exists
   main_test.go:68: Profile ocp4-high-rev-4 exists
   main_test.go:68: Profile ocp4-moderate exists
   main_test.go:68: Profile ocp4-moderate-node exists
   main_test.go:68: Profile ocp4-moderate-node-rev-4 exists
   main_test.go:68: Profile ocp4-moderate-rev-4 exists
   main_test.go:68: Profile ocp4-nerc-cip exists
   main_test.go:68: Profile ocp4-nerc-cip-node exists
   main_test.go:68: Profile ocp4-pci-dss exists
   main_test.go:68: Profile ocp4-pci-dss-3-2 exists
   main_test.go:68: Profile ocp4-pci-dss-4-0 exists
   main_test.go:68: Profile ocp4-pci-dss-node exists
   main_test.go:68: Profile ocp4-pci-dss-node-3-2 exists
   main_test.go:68: Profile ocp4-pci-dss-node-4-0 exists
   main_test.go:68: Profile ocp4-stig exists
   main_test.go:68: Profile ocp4-stig-node exists
   main_test.go:68: Profile ocp4-stig-node-v2r2 exists
   main_test.go:68: Profile ocp4-stig-node-v2r3 exists
   main_test.go:68: Profile ocp4-stig-v2r2 exists
   main_test.go:68: Profile ocp4-stig-v2r3 exists
   main_test.go:68: Profile rhcos4-e8 exists
   main_test.go:68: Profile rhcos4-high exists
   main_test.go:68: Profile rhcos4-high-rev-4 exists
   main_test.go:68: Profile rhcos4-moderate exists
   main_test.go:68: Profile rhcos4-moderate-rev-4 exists
   main_test.go:68: Profile rhcos4-nerc-cip exists
   main_test.go:68: Profile rhcos4-stig exists
   main_test.go:68: Profile rhcos4-stig-v2r2 exists
   main_test.go:68: Profile rhcos4-stig-v2r3 exists
   main_test.go:68: Profile ocp4-bsi exists
   main_test.go:68: Profile ocp4-bsi-2022 exists
   main_test.go:68: Profile ocp4-bsi-node exists
   main_test.go:68: Profile ocp4-bsi-node-2022 exists
   main_test.go:70: All 40 expected profiles exist for architecture amd64
--- PASS: TestProfilesOnArch (103.29s)
PASS
...
ok  	github.com/ComplianceAsCode/compliance-operator/tests/e2e/prerelease	384.139s

The next logical step would be to set up a prow job with f999 frequency (that is what it was marked as in the downstream repo) so that the job never runs unless triggered and wire it up to be part of the integration tests.

Assisted-by: Cursor

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.