[OCPBUGS-57585]CVO protects /metrics with kube-rbac-proxy

[hongkailiu]

This pull adds kube-rbac-proxy as a sidecar in the CVO pod which allows only HTTPS requests to the /metrics endpoint for the Prometheus service account from OpenShift monitoring.

See https://rhobs-handbook.netlify.app/products/openshiftmonitoring/collecting_metrics.md/#kube-rbac-proxy-sidecar for details.

The sidecar has three volumeMounts:

• serving-cert and service-ca are taken from existing volumes.
• secret-kube-rbac-proxy-metric is from the introduced secret openshift-cluster-version-kube-rbac-proxy-metric.

Since proxy servers HTTPS, CVO servers only HTTP request.

I'm not changing --listen in the bootstrap manifest, because we don't need to serve metrics then (it's long before we have Prometheus around to scrape us).

The proxy container serves listens to the 9099 port of the pod's IP which was for the CVO container. The latter now listens to the same port of the loopback address (127.0.0.1) since CVO has hostNetwork: true. We learn this from node-exporter.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hongkailiu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [hongkailiu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hongkailiu]

/retest

[petr-muller]

/cc

[hongkailiu]

/retest

[hongkailiu]

/retest

[hongkailiu]

launch 4.20,openshift/cluster-version-operator#1214 gcp,single-node

The cluster bot job:
https://prow.ci.openshift.org/view/gs/test-platform-results/logs/release-openshift-origin-installer-launch-gcp-modern/1947749334703411200

$ oc debug node/ci-ln-444hbxt-72292-26f5g-master-0
Starting pod/ci-ln-444hbxt-72292-26f5g-master-0-debug-gf24s ...
To use host binaries, run `chroot /host`. Instead, if you need to access host namespaces, run `nsenter -a -t 1`.
Pod IP: 10.0.0.4
If you don't see a command prompt, try pressing enter.
sh-5.1# chroot /host
sh-5.1# curl -k https://10.0.0.4:9099/metrics
Unauthorized

[hongkailiu]

In favour of #1215

[openshift-ci]

@hongkailiu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-hypershift	4da6ccc	link	true	/test e2e-hypershift
ci/prow/e2e-agnostic-operator-devpreview	4da6ccc	link	false	/test e2e-agnostic-operator-devpreview
ci/prow/e2e-hypershift-conformance	4da6ccc	link	true	/test e2e-hypershift-conformance

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.