Skip to content

Conversation

openshift-bot
Copy link
Contributor

disruptions Information

There were (56) added jobs and (112) were removed.

Comparisons were above allowed leeway of 30.00%

Note: For P99, disruptions had 402 jobs increased and 1661 jobs decreased.

Click To Show Table
Name Release From Arch Network Platform Topology Job Results P95 P95 % Increase P99 P99 % Increase
azure-network-liveness-new-connections 4.21 4.21 amd64 ovn aws ha 458 54.15s 345.68% 8m1.01s 790.76%
gcp-network-liveness-new-connections 4.21 4.21 amd64 ovn aws single 411 41s 1950.00% 4m55.1s 662.53%
gcp-network-liveness-new-connections 4.21 4.21 amd64 ovn aws ha 458 40.75s 1795.35% 4m58.17s 645.42%
kube-api-http2-localhost-reused-connections 4.21 4.21 amd64 ovn aws ha 458 1m5s 76.87% 1m36.3s 160.62%
azure-network-liveness-new-connections 4.21 4.21 amd64 ovn aws single 411 54.5s 101.85% 7m43.3s 757.96%
azure-network-liveness-new-connections 4.21 amd64 ovn aws ha 171 49.5s 113.36% 6m5.9s 361.99%
openshift-api-http2-localhost-new-connections 4.21 4.21 amd64 ovn azure ha 513 1m20s 45.19% 1m30s 61.23%
kube-api-http2-localhost-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 1m3s 64.49% 1m9.49s 78.82%
gcp-network-liveness-new-connections 4.21 amd64 ovn aws ha 171 33.5s 193.86% 3m54.8s 371.86%
openshift-api-new-connections 4.21 amd64 ovn azure external 100 35.55s 122.19% 14m5.52s 209.49%
cache-openshift-api-new-connections 4.21 amd64 ovn azure external 100 34.5s 126.97% 13m51.53s 214.26%
cache-openshift-api-reused-connections 4.21 amd64 ovn azure external 100 34.4s 126.32% 13m49.58s 216.39%
gcp-network-liveness-new-connections 4.21 4.21 arm64 ovn aws ha 149 23.6s 386.60% 3m52.08s 429.38%
openshift-api-reused-connections 4.21 amd64 ovn azure external 100 35.45s 101.42% 14m4.55s 209.50%
azure-network-liveness-new-connections 4.21 4.21 arm64 ovn aws ha 149 46.4s 56.76% 7m28.4s 581.67%
azure-network-liveness-new-connections 4.21 4.20 amd64 ovn azure ha 1055 16s 1500.00% 3m34.92s 56.88%
gcp-network-liveness-new-connections 4.21 4.20 amd64 ovn azure ha 1055 10s 3233.33% 2m6.46s 74.86%
azure-network-liveness-new-connections 4.21 multi ovn aws ha 240 10.2s 920.00% 6m41.71s 677.00%
oauth-api-reused-connections 4.21 amd64 ovn azure external 100 24.1s 36.93% 13m35.82s 210.29%
cache-oauth-api-new-connections 4.21 amd64 ovn azure external 100 23.15s 36.18% 13m34.68s 210.61%
cache-oauth-api-reused-connections 4.21 amd64 ovn azure external 100 23.1s 31.25% 13m33.71s 210.62%
cache-kube-api-new-connections 4.21 amd64 ovn azure external 100 12s 71.43% 16.03s 50.66%
cache-kube-api-reused-connections 4.21 amd64 ovn azure external 100 7.05s 135.00% 10.04s 43.43%
kube-api-new-connections 4.21 amd64 ovn azure external 100 13.05s 45.00% 17.01s 50.80%
openshift-api-http2-service-network-new-connections 4.21 4.21 amd64 ovn azure ha 513 4s 321.05% 6s 506.06%
openshift-api-new-connections 4.21 ppc64le ovn none ha 181 3s 200.00% 3.2s 87.13%
cache-oauth-api-new-connections 4.21 ppc64le ovn none ha 181 2s 263.64% 4s 133.92%
cache-openshift-api-new-connections 4.21 ppc64le ovn none ha 181 2s 100.00% 4.8s 380.00%
oauth-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 2s 100.00% 12.88s 329.33%
metrics-api-new-connections 4.21 ppc64le ovn none ha 181 2s 100.00% 3.4s 98.83%
cache-kube-api-new-connections 4.21 ppc64le ovn none ha 181 2s 100.00% 4.4s 340.00%
cache-openshift-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 2s 100.00% 13.94s 271.73%
cache-kube-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1.7s 70.00% 12.88s 186.22%
kube-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
cache-openshift-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
cache-oauth-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
azure-network-liveness-new-connections 4.20 s390x ovn none ha 101 0s 0.00% 17s 44.93%
oauth-api-reused-connections 4.20 s390x ovn none ha 101 3m24s 0.00% 15m38s 88.41%
ingress-to-console-reused-connections 4.20 s390x ovn none ha 101 3m46s 0.00% 15m37s 80.13%
metrics-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 67.24%
ingress-to-oauth-server-new-connections 4.20 s390x ovn none ha 101 3m48s 0.00% 15m37s 63.93%
cache-kube-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
metrics-api-new-connections 4.20 4.20 amd64 ovn aws ha 361 0s 0.00% 1s 203.03%
kube-api-http2-internal-lb-reused-connections 4.20 4.20 amd64 ovn azure ha 404 1s 0.00% 1.97s 97.00%
cache-oauth-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m37s 87.84%
cache-openshift-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m37s 90.06%
cache-openshift-api-new-connections 4.20 4.19 amd64 ovn azure ha 884 0s 0.00% 5.17s 72.33%
service-load-balancer-with-pdb-reused-connections 4.20 4.20 amd64 ovn azure ha 404 3s 0.00% 3.97s 32.33%
cache-kube-api-reused-connections 4.20 s390x ovn none ha 101 3m24s 0.00% 15m37s 88.21%
ingress-to-oauth-server-reused-connections 4.20 s390x ovn none ha 101 3m44s 0.00% 15m37s 80.13%
cache-oauth-api-new-connections 4.20 4.19 amd64 ovn azure ha 884 0s 0.00% 5.17s 72.33%
kube-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m38s 90.26%
metrics-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m38s 82.03%
gcp-network-liveness-new-connections 4.20 s390x ovn none ha 101 0s 0.00% 12s 106.19%
openshift-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
openshift-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m38s 89.89%
gcp-network-liveness-new-connections 4.20 ppc64le ovn none ha 241 0s 0.00% 4.8s 1746.15%
metrics-api-new-connections 4.20 4.20 amd64 ovn azure ha 404 0s 0.00% 20.82s 37.15%
service-load-balancer-with-pdb-new-connections 4.20 4.20 amd64 ovn gcp ha 954 1s 0.00% 3s 50.00%
openshift-api-http2-internal-lb-reused-connections 4.20 4.20 amd64 ovn azure ha 404 1s 0.00% 2s 96.08%
ingress-to-console-new-connections 4.20 s390x ovn none ha 101 3m48s 0.00% 15m37s 63.93%
oauth-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
azure-network-liveness-new-connections 4.20 ppc64le ovn none ha 241 0s 0.00% 3.6s 80.00%
aws-network-liveness-reused-connections 4.21 4.21 amd64 ovn aws single 411 0s 0.00% 900ms 157.14%
kube-api-http1-internal-lb-new-connections 4.21 4.20 amd64 ovn gcp ha 607 0s 0.00% 1s 33.33%
azure-network-liveness-new-connections 4.21 amd64 ovn aws external 672 5.45s 0.00% 1m50.25s 108.02%
gcp-network-liveness-new-connections 4.21 4.20 amd64 ovn aws ha 1027 5.7s 0.00% 3m35.4s 180.03%
azure-network-liveness-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 6.45s 0.00% 4m45.1s 774.54%
azure-network-liveness-new-connections 4.21 ppc64le ovn none ha 181 0s 0.00% 1m51.2s 2137.42%
kube-api-http2-internal-lb-reused-connections 4.21 4.21 amd64 ovn aws ha 458 6.15s 0.00% 8s 60.32%
oauth-api-new-connections 4.21 ppc64le ovn none ha 181 2s 0.00% 3.2s 87.13%
aws-network-liveness-new-connections 4.21 4.20 amd64 ovn gcp ha 607 0s 0.00% 5.76s 476.00%
kube-api-reused-connections 4.21 amd64 ovn azure external 100 10.05s 0.00% 14.01s 45.33%
metrics-api-new-connections 4.21 amd64 ovn aws ha 171 0s 0.00% 24s 53.45%
openshift-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1s 0.00% 12.88s 329.33%
cache-oauth-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1s 0.00% 12.94s 331.33%
gcp-network-liveness-new-connections 4.21 ppc64le ovn none ha 181 0s 0.00% 1m25s 2294.37%
ci-cluster-network-liveness-new-connections 4.21 4.20 amd64 ovn gcp ha 607 2.7s 0.00% 33s 123.73%
oauth-api-new-connections 4.21 amd64 ovn azure external 100 24.05s 0.00% 13m36.82s 210.67%
gcp-network-liveness-new-connections 4.21 amd64 ovn none single 104 0s 0.00% 4.88s 9660.00%
azure-network-liveness-new-connections 4.21 4.20 amd64 ovn aws ha 1027 5.7s 0.00% 5m42.08s 163.14%
ingress-to-console-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 1s 0.00% 7.98s 706.06%
ingress-to-console-new-connections 4.21 amd64 ovn aws ha 171 0s 0.00% 2.5s 150.00%
ingress-to-console-reused-connections 4.21 amd64 ovn aws external 672 0s 0.00% 10s 44.93%
ci-cluster-network-liveness-new-connections 4.21 amd64 ovn azure external 100 0s 0.00% 1.01s 57.81%
openshift-api-reused-connections 4.21 4.21 amd64 ovn aws ha 458 2s 0.00% 5.43s 174.24%
ingress-to-console-new-connections 4.21 amd64 ovn none single 104 1m51s 0.00% 2m26.1s 9265.38%
kube-api-new-connections 4.21 4.21 amd64 ovn azure ha 513 0s 0.00% 2.88s 188.00%
ingress-to-console-new-connections 4.21 ppc64le ovn none ha 181 1s 0.00% 4s 300.00%
kube-api-new-connections 4.21 ppc64le ovn none ha 181 2s 0.00% 3.2s 220.00%
cache-oauth-api-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 1s 0.00% 6s 100.00%
gcp-network-liveness-new-connections 4.21 multi ovn aws ha 240 12.15s 0.00% 4m17.44s 744.07%
kube-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1s 0.00% 12.88s 368.36%
metrics-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1s 0.00% 11s 266.67%
ci-cluster-network-liveness-new-connections 4.21 4.21 arm64 ovn aws ha 149 0s 0.00% 1.52s 52.00%
oauth-api-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 1s 0.00% 6s 51.90%
ingress-to-oauth-server-new-connections 4.21 ppc64le ovn none ha 181 1s 0.00% 2s 100.00%
cache-openshift-api-new-connections 4.21 amd64 ovn aws ha 171 0s 0.00% 1s 47.06%
ingress-to-console-new-connections 4.21 amd64 ovn aws external 672 0s 0.00% 10s 44.93%
service-load-balancer-with-pdb-reused-connections 4.21 4.20 amd64 ovn gcp ha 607 3s 0.00% 3.94s 31.33%
metrics-api-reused-connections 4.21 amd64 ovn aws ha 171 0s 0.00% 24s 53.45%

alerts Information

There were (2790) added jobs and (3122) were removed.

Comparisons were above allowed leeway of 30.00%

Note: For P99, alerts had 42 jobs increased and 282 jobs decreased.

Click To Show Table
Name Release From Arch Network Platform Topology Job Results P95 P95 % Increase P99 P99 % Increase
TechPreviewNoUpgrade openshift-kube-apiserver-operator Warning 4.20 arm64 ovn aws ha 129 3h42m18.4s 218.29% 3h53m52.56s 153.33%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.21 arm64 ovn aws ha 111 40m3s 363.54% 44m1.6s 99.40%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.21 amd64 ovn aws ha 319 18m10.2s 439.44% 33m53.72s 65.60%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.21 amd64 ovn vsphere ha 180 40m16.1s 52.55% 1h5m19.04s 54.77%
NodeSystemSaturation openshift-monitoring Warning 4.21 arm64 ovn aws ha 111 10m41s 681.71% 19m55s 69.26%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.21 4.21 amd64 ovn aws ha 161 14m30s 131.08% 25m1.6s 31.96%
KubeDaemonSetMisScheduled openshift-machine-config-operator Warning 4.21 4.21 amd64 ovn aws ha 161 4m10s 224.68% 5m20.4s 56.06%
NodeSystemSaturation openshift-monitoring Warning 4.20 amd64 ovn azure ha 126 21s 0.00% 7m20.5s 659.48%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.20 amd64 ovn aws ha 254 4m59.4s 0.00% 28m29.98s 63.68%
KubeJobFailed openshift-insights Warning 4.20 amd64 ovn aws ha 254 0s 0.00% 5h23m58.16s 64.65%
etcdGRPCRequestsSlow openshift-etcd Critical 4.20 4.19 amd64 ovn azure ha 211 0s 0.00% 5m46s 1135.71%
NodeSystemSaturation openshift-monitoring Warning 4.20 4.19 amd64 ovn azure ha 211 0s 0.00% 9m55s 149.69%
KubeDaemonSetMisScheduled openshift-machine-config-operator Warning 4.20 4.19 amd64 ovn gcp ha 152 43.4s 0.00% 2m52.94s 59.72%
KubeDaemonSetMisScheduled openshift-machine-config-operator Warning 4.20 4.19 amd64 ovn azure ha 211 0s 0.00% 1m49.2s 51.33%
ExtremelyHighIndividualControlPlaneCPU openshift-kube-apiserver Warning 4.21 4.21 amd64 ovn aws ha 161 0s 0.00% 28s 81.82%
KubeDaemonSetMisScheduled openshift-machine-config-operator Warning 4.21 4.20 amd64 ovn azure ha 245 0s 0.00% 56.24s 59.05%
CannotRetrieveUpdates openshift-cluster-version Warning 4.21 amd64 ovn gcp ha 167 0s 0.00% 26m14.28s 298.65%
NodeSystemSaturation openshift-monitoring Warning 4.21 4.21 amd64 ovn aws single 114 4m58s 0.00% 11m41.2s 135.62%
NodeSystemSaturation openshift-monitoring Warning 4.21 amd64 ovn aws ha 319 0s 0.00% 6m1s 60.76%
/cc @dgoodwin, @neisw, @DennisPeriquet

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 3, 2025
@openshift-ci-robot
Copy link

@openshift-bot: This pull request explicitly references no jira issue.

In response to this:

disruptions Information

There were (56) added jobs and (112) were removed.

Comparisons were above allowed leeway of 30.00%

Note: For P99, disruptions had 402 jobs increased and 1661 jobs decreased.

Click To Show Table
Name Release From Arch Network Platform Topology Job Results P95 P95 % Increase P99 P99 % Increase
azure-network-liveness-new-connections 4.21 4.21 amd64 ovn aws ha 458 54.15s 345.68% 8m1.01s 790.76%
gcp-network-liveness-new-connections 4.21 4.21 amd64 ovn aws single 411 41s 1950.00% 4m55.1s 662.53%
gcp-network-liveness-new-connections 4.21 4.21 amd64 ovn aws ha 458 40.75s 1795.35% 4m58.17s 645.42%
kube-api-http2-localhost-reused-connections 4.21 4.21 amd64 ovn aws ha 458 1m5s 76.87% 1m36.3s 160.62%
azure-network-liveness-new-connections 4.21 4.21 amd64 ovn aws single 411 54.5s 101.85% 7m43.3s 757.96%
azure-network-liveness-new-connections 4.21 amd64 ovn aws ha 171 49.5s 113.36% 6m5.9s 361.99%
openshift-api-http2-localhost-new-connections 4.21 4.21 amd64 ovn azure ha 513 1m20s 45.19% 1m30s 61.23%
kube-api-http2-localhost-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 1m3s 64.49% 1m9.49s 78.82%
gcp-network-liveness-new-connections 4.21 amd64 ovn aws ha 171 33.5s 193.86% 3m54.8s 371.86%
openshift-api-new-connections 4.21 amd64 ovn azure external 100 35.55s 122.19% 14m5.52s 209.49%
cache-openshift-api-new-connections 4.21 amd64 ovn azure external 100 34.5s 126.97% 13m51.53s 214.26%
cache-openshift-api-reused-connections 4.21 amd64 ovn azure external 100 34.4s 126.32% 13m49.58s 216.39%
gcp-network-liveness-new-connections 4.21 4.21 arm64 ovn aws ha 149 23.6s 386.60% 3m52.08s 429.38%
openshift-api-reused-connections 4.21 amd64 ovn azure external 100 35.45s 101.42% 14m4.55s 209.50%
azure-network-liveness-new-connections 4.21 4.21 arm64 ovn aws ha 149 46.4s 56.76% 7m28.4s 581.67%
azure-network-liveness-new-connections 4.21 4.20 amd64 ovn azure ha 1055 16s 1500.00% 3m34.92s 56.88%
gcp-network-liveness-new-connections 4.21 4.20 amd64 ovn azure ha 1055 10s 3233.33% 2m6.46s 74.86%
azure-network-liveness-new-connections 4.21 multi ovn aws ha 240 10.2s 920.00% 6m41.71s 677.00%
oauth-api-reused-connections 4.21 amd64 ovn azure external 100 24.1s 36.93% 13m35.82s 210.29%
cache-oauth-api-new-connections 4.21 amd64 ovn azure external 100 23.15s 36.18% 13m34.68s 210.61%
cache-oauth-api-reused-connections 4.21 amd64 ovn azure external 100 23.1s 31.25% 13m33.71s 210.62%
cache-kube-api-new-connections 4.21 amd64 ovn azure external 100 12s 71.43% 16.03s 50.66%
cache-kube-api-reused-connections 4.21 amd64 ovn azure external 100 7.05s 135.00% 10.04s 43.43%
kube-api-new-connections 4.21 amd64 ovn azure external 100 13.05s 45.00% 17.01s 50.80%
openshift-api-http2-service-network-new-connections 4.21 4.21 amd64 ovn azure ha 513 4s 321.05% 6s 506.06%
openshift-api-new-connections 4.21 ppc64le ovn none ha 181 3s 200.00% 3.2s 87.13%
cache-oauth-api-new-connections 4.21 ppc64le ovn none ha 181 2s 263.64% 4s 133.92%
cache-openshift-api-new-connections 4.21 ppc64le ovn none ha 181 2s 100.00% 4.8s 380.00%
oauth-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 2s 100.00% 12.88s 329.33%
metrics-api-new-connections 4.21 ppc64le ovn none ha 181 2s 100.00% 3.4s 98.83%
cache-kube-api-new-connections 4.21 ppc64le ovn none ha 181 2s 100.00% 4.4s 340.00%
cache-openshift-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 2s 100.00% 13.94s 271.73%
cache-kube-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1.7s 70.00% 12.88s 186.22%
kube-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
cache-openshift-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
cache-oauth-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
azure-network-liveness-new-connections 4.20 s390x ovn none ha 101 0s 0.00% 17s 44.93%
oauth-api-reused-connections 4.20 s390x ovn none ha 101 3m24s 0.00% 15m38s 88.41%
ingress-to-console-reused-connections 4.20 s390x ovn none ha 101 3m46s 0.00% 15m37s 80.13%
metrics-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 67.24%
ingress-to-oauth-server-new-connections 4.20 s390x ovn none ha 101 3m48s 0.00% 15m37s 63.93%
cache-kube-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
metrics-api-new-connections 4.20 4.20 amd64 ovn aws ha 361 0s 0.00% 1s 203.03%
kube-api-http2-internal-lb-reused-connections 4.20 4.20 amd64 ovn azure ha 404 1s 0.00% 1.97s 97.00%
cache-oauth-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m37s 87.84%
cache-openshift-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m37s 90.06%
cache-openshift-api-new-connections 4.20 4.19 amd64 ovn azure ha 884 0s 0.00% 5.17s 72.33%
service-load-balancer-with-pdb-reused-connections 4.20 4.20 amd64 ovn azure ha 404 3s 0.00% 3.97s 32.33%
cache-kube-api-reused-connections 4.20 s390x ovn none ha 101 3m24s 0.00% 15m37s 88.21%
ingress-to-oauth-server-reused-connections 4.20 s390x ovn none ha 101 3m44s 0.00% 15m37s 80.13%
cache-oauth-api-new-connections 4.20 4.19 amd64 ovn azure ha 884 0s 0.00% 5.17s 72.33%
kube-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m38s 90.26%
metrics-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m38s 82.03%
gcp-network-liveness-new-connections 4.20 s390x ovn none ha 101 0s 0.00% 12s 106.19%
openshift-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
openshift-api-reused-connections 4.20 s390x ovn none ha 101 3m23s 0.00% 15m38s 89.89%
gcp-network-liveness-new-connections 4.20 ppc64le ovn none ha 241 0s 0.00% 4.8s 1746.15%
metrics-api-new-connections 4.20 4.20 amd64 ovn azure ha 404 0s 0.00% 20.82s 37.15%
service-load-balancer-with-pdb-new-connections 4.20 4.20 amd64 ovn gcp ha 954 1s 0.00% 3s 50.00%
openshift-api-http2-internal-lb-reused-connections 4.20 4.20 amd64 ovn azure ha 404 1s 0.00% 2s 96.08%
ingress-to-console-new-connections 4.20 s390x ovn none ha 101 3m48s 0.00% 15m37s 63.93%
oauth-api-new-connections 4.20 s390x ovn none ha 101 3m41s 0.00% 15m38s 66.94%
azure-network-liveness-new-connections 4.20 ppc64le ovn none ha 241 0s 0.00% 3.6s 80.00%
aws-network-liveness-reused-connections 4.21 4.21 amd64 ovn aws single 411 0s 0.00% 900ms 157.14%
kube-api-http1-internal-lb-new-connections 4.21 4.20 amd64 ovn gcp ha 607 0s 0.00% 1s 33.33%
azure-network-liveness-new-connections 4.21 amd64 ovn aws external 672 5.45s 0.00% 1m50.25s 108.02%
gcp-network-liveness-new-connections 4.21 4.20 amd64 ovn aws ha 1027 5.7s 0.00% 3m35.4s 180.03%
azure-network-liveness-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 6.45s 0.00% 4m45.1s 774.54%
azure-network-liveness-new-connections 4.21 ppc64le ovn none ha 181 0s 0.00% 1m51.2s 2137.42%
kube-api-http2-internal-lb-reused-connections 4.21 4.21 amd64 ovn aws ha 458 6.15s 0.00% 8s 60.32%
oauth-api-new-connections 4.21 ppc64le ovn none ha 181 2s 0.00% 3.2s 87.13%
aws-network-liveness-new-connections 4.21 4.20 amd64 ovn gcp ha 607 0s 0.00% 5.76s 476.00%
kube-api-reused-connections 4.21 amd64 ovn azure external 100 10.05s 0.00% 14.01s 45.33%
metrics-api-new-connections 4.21 amd64 ovn aws ha 171 0s 0.00% 24s 53.45%
openshift-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1s 0.00% 12.88s 329.33%
cache-oauth-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1s 0.00% 12.94s 331.33%
gcp-network-liveness-new-connections 4.21 ppc64le ovn none ha 181 0s 0.00% 1m25s 2294.37%
ci-cluster-network-liveness-new-connections 4.21 4.20 amd64 ovn gcp ha 607 2.7s 0.00% 33s 123.73%
oauth-api-new-connections 4.21 amd64 ovn azure external 100 24.05s 0.00% 13m36.82s 210.67%
gcp-network-liveness-new-connections 4.21 amd64 ovn none single 104 0s 0.00% 4.88s 9660.00%
azure-network-liveness-new-connections 4.21 4.20 amd64 ovn aws ha 1027 5.7s 0.00% 5m42.08s 163.14%
ingress-to-console-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 1s 0.00% 7.98s 706.06%
ingress-to-console-new-connections 4.21 amd64 ovn aws ha 171 0s 0.00% 2.5s 150.00%
ingress-to-console-reused-connections 4.21 amd64 ovn aws external 672 0s 0.00% 10s 44.93%
ci-cluster-network-liveness-new-connections 4.21 amd64 ovn azure external 100 0s 0.00% 1.01s 57.81%
openshift-api-reused-connections 4.21 4.21 amd64 ovn aws ha 458 2s 0.00% 5.43s 174.24%
ingress-to-console-new-connections 4.21 amd64 ovn none single 104 1m51s 0.00% 2m26.1s 9265.38%
kube-api-new-connections 4.21 4.21 amd64 ovn azure ha 513 0s 0.00% 2.88s 188.00%
ingress-to-console-new-connections 4.21 ppc64le ovn none ha 181 1s 0.00% 4s 300.00%
kube-api-new-connections 4.21 ppc64le ovn none ha 181 2s 0.00% 3.2s 220.00%
cache-oauth-api-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 1s 0.00% 6s 100.00%
gcp-network-liveness-new-connections 4.21 multi ovn aws ha 240 12.15s 0.00% 4m17.44s 744.07%
kube-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1s 0.00% 12.88s 368.36%
metrics-api-new-connections 4.21 4.20 amd64 ovn gcp ha 607 1s 0.00% 11s 266.67%
ci-cluster-network-liveness-new-connections 4.21 4.21 arm64 ovn aws ha 149 0s 0.00% 1.52s 52.00%
oauth-api-new-connections 4.21 4.21 amd64 ovn gcp ha 1152 1s 0.00% 6s 51.90%
ingress-to-oauth-server-new-connections 4.21 ppc64le ovn none ha 181 1s 0.00% 2s 100.00%
cache-openshift-api-new-connections 4.21 amd64 ovn aws ha 171 0s 0.00% 1s 47.06%
ingress-to-console-new-connections 4.21 amd64 ovn aws external 672 0s 0.00% 10s 44.93%
service-load-balancer-with-pdb-reused-connections 4.21 4.20 amd64 ovn gcp ha 607 3s 0.00% 3.94s 31.33%
metrics-api-reused-connections 4.21 amd64 ovn aws ha 171 0s 0.00% 24s 53.45%

alerts Information

There were (2790) added jobs and (3122) were removed.

Comparisons were above allowed leeway of 30.00%

Note: For P99, alerts had 42 jobs increased and 282 jobs decreased.

Click To Show Table
Name Release From Arch Network Platform Topology Job Results P95 P95 % Increase P99 P99 % Increase
TechPreviewNoUpgrade openshift-kube-apiserver-operator Warning 4.20 arm64 ovn aws ha 129 3h42m18.4s 218.29% 3h53m52.56s 153.33%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.21 arm64 ovn aws ha 111 40m3s 363.54% 44m1.6s 99.40%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.21 amd64 ovn aws ha 319 18m10.2s 439.44% 33m53.72s 65.60%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.21 amd64 ovn vsphere ha 180 40m16.1s 52.55% 1h5m19.04s 54.77%
NodeSystemSaturation openshift-monitoring Warning 4.21 arm64 ovn aws ha 111 10m41s 681.71% 19m55s 69.26%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.21 4.21 amd64 ovn aws ha 161 14m30s 131.08% 25m1.6s 31.96%
KubeDaemonSetMisScheduled openshift-machine-config-operator Warning 4.21 4.21 amd64 ovn aws ha 161 4m10s 224.68% 5m20.4s 56.06%
NodeSystemSaturation openshift-monitoring Warning 4.20 amd64 ovn azure ha 126 21s 0.00% 7m20.5s 659.48%
HighOverallControlPlaneMemory openshift-machine-config-operator Warning 4.20 amd64 ovn aws ha 254 4m59.4s 0.00% 28m29.98s 63.68%
KubeJobFailed openshift-insights Warning 4.20 amd64 ovn aws ha 254 0s 0.00% 5h23m58.16s 64.65%
etcdGRPCRequestsSlow openshift-etcd Critical 4.20 4.19 amd64 ovn azure ha 211 0s 0.00% 5m46s 1135.71%
NodeSystemSaturation openshift-monitoring Warning 4.20 4.19 amd64 ovn azure ha 211 0s 0.00% 9m55s 149.69%
KubeDaemonSetMisScheduled openshift-machine-config-operator Warning 4.20 4.19 amd64 ovn gcp ha 152 43.4s 0.00% 2m52.94s 59.72%
KubeDaemonSetMisScheduled openshift-machine-config-operator Warning 4.20 4.19 amd64 ovn azure ha 211 0s 0.00% 1m49.2s 51.33%
ExtremelyHighIndividualControlPlaneCPU openshift-kube-apiserver Warning 4.21 4.21 amd64 ovn aws ha 161 0s 0.00% 28s 81.82%
KubeDaemonSetMisScheduled openshift-machine-config-operator Warning 4.21 4.20 amd64 ovn azure ha 245 0s 0.00% 56.24s 59.05%
CannotRetrieveUpdates openshift-cluster-version Warning 4.21 amd64 ovn gcp ha 167 0s 0.00% 26m14.28s 298.65%
NodeSystemSaturation openshift-monitoring Warning 4.21 4.21 amd64 ovn aws single 114 4m58s 0.00% 11m41.2s 135.62%
NodeSystemSaturation openshift-monitoring Warning 4.21 amd64 ovn aws ha 319 0s 0.00% 6m1s 60.76%
/cc @dgoodwin, @neisw, @DennisPeriquet

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from p0lyn0mial and sjenning October 3, 2025 01:05
@neisw
Copy link
Contributor

neisw commented Oct 3, 2025

/lgtm
/verified bypass

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 3, 2025
@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Oct 3, 2025
@openshift-ci-robot
Copy link

@neisw: The verified label has been added.

In response to this:

/lgtm
/verified bypass

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Contributor

openshift-ci bot commented Oct 3, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neisw, openshift-bot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 3, 2025
@neisw
Copy link
Contributor

neisw commented Oct 3, 2025

/retest-required

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 3ded19e and 2 for PR HEAD 731d981 in total

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD a21249f and 1 for PR HEAD 731d981 in total

Copy link
Contributor

openshift-ci bot commented Oct 4, 2025

@openshift-bot: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-single-node-upgrade 731d981 link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/okd-scos-e2e-aws-ovn 731d981 link false /test okd-scos-e2e-aws-ovn
ci/prow/e2e-aws-ovn-fips 731d981 link unknown /test e2e-aws-ovn-fips
ci/prow/e2e-vsphere-ovn-upi 731d981 link unknown /test e2e-vsphere-ovn-upi

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants