OCPBUGS-59757: OCPBUGS-59758: Improve MCN test stability

[isabella-janssen]

This is a manual cherry-pick of #29981, since the automated cherry-pick never completed.

---

Work included:
OCPBUGS-59203
This updates the MCN happy scope path test, Should properly update the MCN from the associated MCD, to be a part of the serial test suite instead of the parallel test suite. This update is necessary since the test makes changes to MCN objects that can cause interference with other MCN tests when run in parallel.

OCPBUGS-57524
This does two things to improve the stability of the slow, disruptive MCN tests

• Increase the node readiness timeout to reduce failures in the Should properly create and remove MCN on node creation and deletion due to the newly provisioned node not becoming ready within two minutes.
• Add debugging logs to help with troubleshooting in the occasionally flaky Should properly report MCN conditions on node degrade test.

Test locally:
To run the updated tests locally, run them against any 4.19 cluster with the following command.

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Suite:openshift/conformance/serial][Serial]Should properly update the MCN from the associated MCD [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly report MCN conditions on node degrade [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly create and remove MCN on node creation and deletion [apigroup:machineconfiguration.openshift.io]"

Test with payload:
All MCN tests should consistently pass in the MCO disruptive test suite & default serial test suite.

• Disruptive suite: https://pr-payload-tests.ci.openshift.org/runs/ci/8f303eb0-6888-11f0-93ac-4d919f688b32-0
• Default serial: https://pr-payload-tests.ci.openshift.org/runs/ci/92b81070-6889-11f0-83f3-d2115506bc53-0

[isabella-janssen]

/payload-job periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-aws-mco-disruptive-techpreview periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-azure-mco-disruptive-techpreview periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-gcp-mco-disruptive-techpreview periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-vsphere-mco-disruptive-techpreview

[openshift-ci]

@isabella-janssen: trigger 4 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-aws-mco-disruptive-techpreview
• periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-azure-mco-disruptive-techpreview
• periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-gcp-mco-disruptive-techpreview
• periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-vsphere-mco-disruptive-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/8f303eb0-6888-11f0-93ac-4d919f688b32-0

[isabella-janssen]

/cherry-pick jira OCPBUGS-59203

[openshift-cherrypick-robot]

@isabella-janssen: once the present PR merges, I will cherry-pick it on top of jira in a new PR and assign it to you.

In response to this:

/cherry-pick jira OCPBUGS-59203

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[isabella-janssen]

/jira cherry-pick OCPBUGS-59203

[openshift-ci-robot]

@isabella-janssen: Jira Issue OCPBUGS-59203 has been cloned as Jira Issue OCPBUGS-59757. Will retitle bug to link to clone.
/retitle OCPBUGS-59757: Improve MCN test stability

In response to this:

/jira cherry-pick OCPBUGS-59203

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@isabella-janssen: This pull request references Jira Issue OCPBUGS-59757, which is invalid:

• expected dependent Jira Issue OCPBUGS-59203 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This is a manual cherry-pick of #29981, since the automated cherry-pick never completed.

---

Work included:
OCPBUGS-59203
This updates the MCN happy scope path test, Should properly update the MCN from the associated MCD, to be a part of the serial test suite instead of the parallel test suite. This update is necessary since the test makes changes to MCN objects that can cause interference with other MCN tests when run in parallel.

OCPBUGS-57524
This does two things to improve the stability of the slow, disruptive MCN tests

• Increase the node readiness timeout to reduce failures in the Should properly create and remove MCN on node creation and deletion due to the newly provisioned node not becoming ready within two minutes.
• Add debugging logs to help with troubleshooting in the occasionally flaky Should properly report MCN conditions on node degrade test.

Test locally:
To run the updated tests locally, run them against any 4.19 cluster with the following command.

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Suite:openshift/conformance/serial][Serial]Should properly update the MCN from the associated MCD [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly report MCN conditions on node degrade [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly create and remove MCN on node creation and deletion [apigroup:machineconfiguration.openshift.io]"

Test with payload:
All MCN tests should consistently pass in the MCO disruptive test suite & default serial test suite.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[isabella-janssen]

/jira cherry-pick OCPBUGS-57524

[openshift-ci-robot]

@isabella-janssen: Jira Issue OCPBUGS-57524 has been cloned as Jira Issue OCPBUGS-59758. Will retitle bug to link to clone.
/retitle OCPBUGS-59758: OCPBUGS-59757: Improve MCN test stability

In response to this:

/jira cherry-pick OCPBUGS-57524

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@isabella-janssen: This pull request references Jira Issue OCPBUGS-59758, which is invalid:

• expected dependent Jira Issue OCPBUGS-57524 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This is a manual cherry-pick of #29981, since the automated cherry-pick never completed.

---

Work included:
OCPBUGS-59203
This updates the MCN happy scope path test, Should properly update the MCN from the associated MCD, to be a part of the serial test suite instead of the parallel test suite. This update is necessary since the test makes changes to MCN objects that can cause interference with other MCN tests when run in parallel.

OCPBUGS-57524
This does two things to improve the stability of the slow, disruptive MCN tests

• Increase the node readiness timeout to reduce failures in the Should properly create and remove MCN on node creation and deletion due to the newly provisioned node not becoming ready within two minutes.
• Add debugging logs to help with troubleshooting in the occasionally flaky Should properly report MCN conditions on node degrade test.

Test locally:
To run the updated tests locally, run them against any 4.19 cluster with the following command.

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Suite:openshift/conformance/serial][Serial]Should properly update the MCN from the associated MCD [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly report MCN conditions on node degrade [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly create and remove MCN on node creation and deletion [apigroup:machineconfiguration.openshift.io]"

Test with payload:
All MCN tests should consistently pass in the MCO disruptive test suite & default serial test suite.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[isabella-janssen]

/payload-job periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview-serial periodic-ci-openshift-release-master-nightly-4.19-e2e-aws-ovn-single-node-techpreview-serial periodic-ci-openshift-release-master-ci-4.19-e2e-azure-ovn-techpreview-serial periodic-ci-openshift-release-master-ci-4.19-e2e-gcp-ovn-techpreview-serial periodic-ci-openshift-release-master-nightly-4.19-e2e-vsphere-ovn-techpreview-serial

[openshift-ci]

@isabella-janssen: trigger 5 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview-serial
• periodic-ci-openshift-release-master-nightly-4.19-e2e-aws-ovn-single-node-techpreview-serial
• periodic-ci-openshift-release-master-ci-4.19-e2e-azure-ovn-techpreview-serial
• periodic-ci-openshift-release-master-ci-4.19-e2e-gcp-ovn-techpreview-serial
• periodic-ci-openshift-release-master-nightly-4.19-e2e-vsphere-ovn-techpreview-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/92b81070-6889-11f0-83f3-d2115506bc53-0

[yuqi-zhang]

The diff looked a bit weird, but cross-comparing the two branches the content seem to align, so

/lgtm

[isabella-janssen]

/jira refresh

[openshift-ci-robot]

@isabella-janssen: This pull request references Jira Issue OCPBUGS-59758, which is invalid:

• expected dependent Jira Issue OCPBUGS-57524 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[isabella-janssen]

/jira refresh

[openshift-ci-robot]

@isabella-janssen: This pull request references Jira Issue OCPBUGS-59757, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.19.z) matches configured target version for branch (4.19.z)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
• release note type set to "Release Note Not Required"
• dependent bug Jira Issue OCPBUGS-59203 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
• dependent Jira Issue OCPBUGS-59203 targets the "4.20.0" version, which is one of the valid target versions: 4.20.0
• bug has dependents

Requesting review from QA contact:
/cc @sergiordlr

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This is a manual cherry-pick of #29981, since the automated cherry-pick never completed.

---

Work included:
OCPBUGS-59203
This updates the MCN happy scope path test, Should properly update the MCN from the associated MCD, to be a part of the serial test suite instead of the parallel test suite. This update is necessary since the test makes changes to MCN objects that can cause interference with other MCN tests when run in parallel.

OCPBUGS-57524
This does two things to improve the stability of the slow, disruptive MCN tests

• Increase the node readiness timeout to reduce failures in the Should properly create and remove MCN on node creation and deletion due to the newly provisioned node not becoming ready within two minutes.
• Add debugging logs to help with troubleshooting in the occasionally flaky Should properly report MCN conditions on node degrade test.

Test locally:
To run the updated tests locally, run them against any 4.19 cluster with the following command.

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Suite:openshift/conformance/serial][Serial]Should properly update the MCN from the associated MCD [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly report MCN conditions on node degrade [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly create and remove MCN on node creation and deletion [apigroup:machineconfiguration.openshift.io]"

Test with payload:
All MCN tests should consistently pass in the MCO disruptive test suite & default serial test suite.

• Disruptive suite: https://pr-payload-tests.ci.openshift.org/runs/ci/8f303eb0-6888-11f0-93ac-4d919f688b32-0
• Default serial: https://pr-payload-tests.ci.openshift.org/runs/ci/92b81070-6889-11f0-83f3-d2115506bc53-0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[isabella-janssen]

/retest-required

[isabella-janssen]

/retest-required

[isabella-janssen]

/test e2e-metal-ipi-ovn-ipv6

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD fcd77b9 and 2 for PR HEAD eea68de in total

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD fcd77b9 and 2 for PR HEAD eea68de in total

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD fcd77b9 and 2 for PR HEAD eea68de in total

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD fcd77b9 and 2 for PR HEAD eea68de in total

[isabella-janssen]

/retest-required

[openshift-ci]

@isabella-janssen: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-etcd-scaling	eea68de	link	false	/test e2e-aws-ovn-etcd-scaling
ci/prow/e2e-metal-ipi-ovn-kube-apiserver-rollout	eea68de	link	false	/test e2e-metal-ipi-ovn-kube-apiserver-rollout
ci/prow/e2e-openstack-serial	eea68de	link	false	/test e2e-openstack-serial
ci/prow/e2e-azure-ovn-upgrade	eea68de	link	false	/test e2e-azure-ovn-upgrade
ci/prow/e2e-vsphere-ovn-etcd-scaling	eea68de	link	false	/test e2e-vsphere-ovn-etcd-scaling
ci/prow/e2e-gcp-disruptive	eea68de	link	false	/test e2e-gcp-disruptive
ci/prow/e2e-openstack-ovn	eea68de	link	false	/test e2e-openstack-ovn
ci/prow/e2e-metal-ipi-serial	eea68de	link	false	/test e2e-metal-ipi-serial
ci/prow/e2e-metal-ipi-virtualmedia	eea68de	link	false	/test e2e-metal-ipi-virtualmedia
ci/prow/e2e-vsphere-ovn-dualstack-primaryv6	eea68de	link	false	/test e2e-vsphere-ovn-dualstack-primaryv6
ci/prow/e2e-azure-ovn-etcd-scaling	eea68de	link	false	/test e2e-azure-ovn-etcd-scaling
ci/prow/e2e-gcp-fips-serial	eea68de	link	false	/test e2e-gcp-fips-serial
ci/prow/e2e-aws-disruptive	eea68de	link	false	/test e2e-aws-disruptive
ci/prow/e2e-gcp-ovn-etcd-scaling	eea68de	link	false	/test e2e-gcp-ovn-etcd-scaling

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[isabella-janssen]

/payload-job periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-aws-mco-disruptive-techpreview periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-azure-mco-disruptive-techpreview periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-gcp-mco-disruptive-techpreview periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-vsphere-mco-disruptive-techpreview periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview-serial

[openshift-ci]

@isabella-janssen: trigger 6 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

• periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-aws-mco-disruptive-techpreview
• periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-azure-mco-disruptive-techpreview
• periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-gcp-mco-disruptive-techpreview
• periodic-ci-openshift-machine-config-operator-release-4.19-periodics-e2e-vsphere-mco-disruptive-techpreview
• periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview
• periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/cc14dd70-7a05-11f0-9929-75a49a3c495f-0

[pablintino]

/diff
The helpers diff is a bit weird but it's fine.

[pablintino]

/lgtm

[djoshy]

/lgtm

/honk

[openshift-ci]

@djoshy:

In response to this:

/lgtm

/honk

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djoshy, isabella-janssen, pablintino, xueqzhan, yuqi-zhang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [xueqzhan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[isabella-janssen]

/retest-required

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD fcd77b9 and 2 for PR HEAD de4b9ef in total

[openshift-ci-robot]

@isabella-janssen: Jira Issue OCPBUGS-59757: All pull requests linked via external trackers have merged:

• openshift/origin#30018

Jira Issue OCPBUGS-59757 has been moved to the MODIFIED state.

In response to this:

This is a manual cherry-pick of #29981, since the automated cherry-pick never completed.

---

Work included:
OCPBUGS-59203
This updates the MCN happy scope path test, Should properly update the MCN from the associated MCD, to be a part of the serial test suite instead of the parallel test suite. This update is necessary since the test makes changes to MCN objects that can cause interference with other MCN tests when run in parallel.

OCPBUGS-57524
This does two things to improve the stability of the slow, disruptive MCN tests

• Increase the node readiness timeout to reduce failures in the Should properly create and remove MCN on node creation and deletion due to the newly provisioned node not becoming ready within two minutes.
• Add debugging logs to help with troubleshooting in the occasionally flaky Should properly report MCN conditions on node degrade test.

Test locally:
To run the updated tests locally, run them against any 4.19 cluster with the following command.

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Suite:openshift/conformance/serial][Serial]Should properly update the MCN from the associated MCD [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly report MCN conditions on node degrade [apigroup:machineconfiguration.openshift.io]"

./openshift-tests run-test "[Suite:openshift/machine-config-operator/disruptive][sig-mco][OCPFeatureGate:MachineConfigNodes] [Serial][Slow]Should properly create and remove MCN on node creation and deletion [apigroup:machineconfiguration.openshift.io]"

Test with payload:
All MCN tests should consistently pass in the MCO disruptive test suite & default serial test suite.

• Disruptive suite: https://pr-payload-tests.ci.openshift.org/runs/ci/8f303eb0-6888-11f0-93ac-4d919f688b32-0
• Default serial: https://pr-payload-tests.ci.openshift.org/runs/ci/92b81070-6889-11f0-83f3-d2115506bc53-0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-cherrypick-robot]

@isabella-janssen: cannot checkout jira: error checking out "jira": exit status 1 error: pathspec 'jira' did not match any file(s) known to git

In response to this:

/cherry-pick jira OCPBUGS-59203

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: openshift-enterprise-tests
This PR has been included in build openshift-enterprise-tests-container-v4.19.0-202508190013.p0.g0768bba.assembly.stream.el9.
All builds following this will include this PR.

[openshift-merge-robot]

Fix included in accepted release 4.19.0-0.nightly-2025-08-19-141357