Skip to content

Conversation

hongkailiu
Copy link
Member

/hold

need to rebase after #30296 gets in

This is to cover the node rebooting case from the rule [1] that
is introduced recently:

```
Operators should not report Progressing only because DaemonSets
owned by them are adjusting to a new node from cluster scaleup or
a node rebooting from cluster upgrade.
```

The test fails if

- `co/machine-config` never became Progressing=True during a
  cluster upgrade, or
- some CO left Progressing=False during the upgrade after
  `machine-config` became Progressing=True. This should not
  have taken place as `machine-config` was rebooting the nodes
  which was the only thing ongoing to the cluster during that
  time.

[1]. https://github.com/openshift/api/blob/61248d910ff74aef020492922d14e6dadaba598b/config/v1/types_cluster_operator.go#L163-L164
@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 2, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 2, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 2, 2025

@hongkailiu: This pull request references OTA-1643 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

/hold

need to rebase after #30296 gets in

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from deads2k and p0lyn0mial October 2, 2025 21:31
Copy link
Contributor

openshift-ci bot commented Oct 2, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: hongkailiu
Once this PR has been reviewed and has the lgtm label, please assign xueqzhan for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

openshift-trt bot commented Oct 3, 2025

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New Test Risks for sha: b188bf9

Job Name New Test Risk
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade High - "[Monitor:legacy-cvo-invariants][bz-Bare Metal Hardware Provisioning] clusteroperator/baremetal must go Progressing=True during an upgrade test" is a new test, was only seen in one job, and failed 1 time(s) against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Bare Metal Hardware Provisioning] clusteroperator/baremetal should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade High - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/cloud-controller-manager must go Progressing=True during an upgrade test" is a new test, was only seen in one job, and failed 1 time(s) against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/cloud-controller-manager should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade High - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/cluster-autoscaler must go Progressing=True during an upgrade test" is a new test, was only seen in one job, and failed 1 time(s) against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/cluster-autoscaler should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade High - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/control-plane-machine-set must go Progressing=True during an upgrade test" is a new test, was only seen in one job, and failed 1 time(s) against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/control-plane-machine-set should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/machine-api must go Progressing=True during an upgrade test" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/machine-api should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade High - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/machine-approver must go Progressing=True during an upgrade test" is a new test, was only seen in one job, and failed 1 time(s) against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/machine-approver should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade High - "[Monitor:legacy-cvo-invariants][bz-Cloud Credential Operator] clusteroperator/cloud-credential must go Progressing=True during an upgrade test" is a new test, was only seen in one job, and failed 1 time(s) against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Cloud Credential Operator] clusteroperator/cloud-credential should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade High - "[Monitor:legacy-cvo-invariants][bz-DNS] clusteroperator/dns must go Progressing=True during an upgrade test" is a new test, was only seen in one job, and failed 1 time(s) against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-DNS] clusteroperator/dns should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade High - "[Monitor:legacy-cvo-invariants][bz-Etcd] clusteroperator/etcd must go Progressing=True during an upgrade test" is a new test, was only seen in one job, and failed 1 time(s) against the current commit.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Etcd] clusteroperator/etcd should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Image Registry] clusteroperator/image-registry must go Progressing=True during an upgrade test" is a new test, and was only seen in one job.
pull-ci-openshift-origin-main-e2e-gcp-ovn-upgrade Medium - "[Monitor:legacy-cvo-invariants][bz-Image Registry] clusteroperator/image-registry should stay Progressing=False while MCO is Progressing=True" is a new test, and was only seen in one job.
(...showing 20 of 65 rows)

New tests seen in this PR at sha: b188bf9

  • "[Monitor:legacy-cvo-invariants][bz-Bare Metal Hardware Provisioning] clusteroperator/baremetal must go Progressing=True during an upgrade test" [Total: 1, Pass: 0, Fail: 1, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Bare Metal Hardware Provisioning] clusteroperator/baremetal should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/cloud-controller-manager must go Progressing=True during an upgrade test" [Total: 1, Pass: 0, Fail: 1, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/cloud-controller-manager should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/cluster-autoscaler must go Progressing=True during an upgrade test" [Total: 1, Pass: 0, Fail: 1, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/cluster-autoscaler should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/control-plane-machine-set must go Progressing=True during an upgrade test" [Total: 1, Pass: 0, Fail: 1, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/control-plane-machine-set should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/machine-api must go Progressing=True during an upgrade test" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/machine-api should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/machine-approver must go Progressing=True during an upgrade test" [Total: 1, Pass: 0, Fail: 1, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Compute] clusteroperator/machine-approver should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Credential Operator] clusteroperator/cloud-credential must go Progressing=True during an upgrade test" [Total: 1, Pass: 0, Fail: 1, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Cloud Credential Operator] clusteroperator/cloud-credential should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-DNS] clusteroperator/dns must go Progressing=True during an upgrade test" [Total: 1, Pass: 0, Fail: 1, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-DNS] clusteroperator/dns should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Etcd] clusteroperator/etcd must go Progressing=True during an upgrade test" [Total: 1, Pass: 0, Fail: 1, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Etcd] clusteroperator/etcd should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Image Registry] clusteroperator/image-registry must go Progressing=True during an upgrade test" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • "[Monitor:legacy-cvo-invariants][bz-Image Registry] clusteroperator/image-registry should stay Progressing=False while MCO is Progressing=True" [Total: 1, Pass: 1, Fail: 0, Flake: 0]
  • (...showing 20 of 65 tests)

Copy link
Contributor

openshift-ci bot commented Oct 3, 2025

@hongkailiu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-gcp-ovn-upgrade b188bf9 link true /test e2e-gcp-ovn-upgrade
ci/prow/e2e-aws-ovn-single-node-upgrade b188bf9 link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/okd-scos-e2e-aws-ovn b188bf9 link false /test okd-scos-e2e-aws-ovn
ci/prow/e2e-openstack-ovn b188bf9 link false /test e2e-openstack-ovn
ci/prow/e2e-metal-ipi-ovn-kube-apiserver-rollout b188bf9 link false /test e2e-metal-ipi-ovn-kube-apiserver-rollout
ci/prow/e2e-metal-ipi-ovn-ipv6 b188bf9 link true /test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-aws-ovn-single-node b188bf9 link false /test e2e-aws-ovn-single-node
ci/prow/e2e-aws-csi b188bf9 link true /test e2e-aws-csi
ci/prow/e2e-gcp-csi b188bf9 link true /test e2e-gcp-csi
ci/prow/e2e-aws-ovn-single-node-serial b188bf9 link false /test e2e-aws-ovn-single-node-serial

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants