Custom Kube State Metrics: Increase Resources

[martysp21]

Increase the CPU allocation to prevent CPU throttling and memory spikes.
The change is made to adhere for latest resource recommendation stated
in the KSM docs. This change was not triggered by any other problem than
prevention.

https://github.com/kubernetes/kube-state-metrics#resource-recommendation

[github-actions]

Kustomize Render Diff

Comparing 5e6627780 → 499237fdb

Component	Environment	Changes
components/monitoring/custom-kube-state-metrics/staging	development	+2 -2
components/monitoring/custom-kube-state-metrics/production	production	+2 -2
components/monitoring/custom-kube-state-metrics/staging	staging	+2 -2

Total: 3 components, +6 -6 lines

📋 Full diff available in the workflow summary and as a downloadable artifact.

[qodo-for-redhat-appstudio]

PR Summary by Qodo

Increase custom kube-state-metrics CPU requests/limits to reduce throttling

⚙️ Configuration changes 🐞 Bug fix 🕐 Less than 10 minutes

AI Description

• Increase kube-state-metrics CPU request to reduce runtime throttling under load
• Raise CPU limit to better absorb spikes without impacting scrape latency

Diagram

graph TD
  Dev["Ops/Platform"] --> Manifest["deployment.yaml"] --> Deploy["KSM Deployment"] --> Pod["KSM Pod"] --> Metrics["Cluster metrics"]
  subgraph Legend
    direction LR
    _cfg["Config"] ~~~ _k8s["Kubernetes"] ~~~ _out["Output"]
  end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Enable Vertical Pod Autoscaler (VPA) recommendations

• ➕ Automatically adapts resource requests to observed usage
• ➕ Reduces manual tuning churn across environments
• ➖ Operational overhead and policy decisions (update mode, disruption)
• ➖ May be harder to roll out in tightly controlled clusters

2. Add HPA with CPU-based scaling (if stateless/replicable)

• ➕ Scales out to handle bursts without raising single-pod limits
• ➕ Can improve availability during spikes
• ➖ kube-state-metrics scaling characteristics may be limited by API/QPS and duplicate work
• ➖ Requires validating correctness/performance with multiple replicas

Recommendation: For an immediate mitigation to CPU throttling, the static request/limit bump in this PR is the simplest and lowest-risk option. If throttling recurs or usage varies widely over time, consider VPA recommendations (and potentially controlled VPA updates) as a follow-up to avoid repeated manual adjustments.

Files changed (1) +2 / -2

Other (1) +2 / -2

deployment.yamlIncrease CPU request/limit for custom kube-state-metrics +2/-2

Increase CPU request/limit for custom kube-state-metrics

• Updates the Deployment resources to request more CPU (50m → 100m) and allow a higher CPU limit (200m → 250m). This aims to reduce CPU throttling and associated performance/memory side effects under load.

components/monitoring/custom-kube-state-metrics/base/deployment.yaml

[qodo-for-redhat-appstudio]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Tip of the day

💡 Did you know, you can route each action level your way: inline, summary, both, or drop

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Previous review results

Review updated until commit 3b65d33 ⚖️ Balanced

Results up to commit e90d741 ⚖️ Balanced

No changes from previous review

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: martysp21, pumahaka

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• components/monitoring/OWNERS [martysp21,pumahaka]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kubasikus]

/hold

[kubasikus]

@martysp21 Can you share some evidence of the CPU throttling occuring?

[martysp21]

@martysp21 Can you share some evidence of the CPU throttling occuring?

Heya, I am not stating that any throttling is happening at the moment. This is a prevention and minimal recommended resource allocation taken from the official docs of KSM. https://github.com/kubernetes/kube-state-metrics#resource-recommendation

[raks-tt]

/hold
Requesting for JIRA/ references why this is being done

[qodo-for-redhat-appstudio]

Code review by qodo was updated up to the latest commit 3b65d33

[kubasikus]

/unhold

[kubasikus]

/lgtm

[konflux-ci-qe-bot]

🤖 Pipeline Failure Analysis

Category: Configuration

The pipeline failed during Konflux installation because the ArgoCD application 'vector-kubearchive-log-collector-in-cluster-local' remained in an 'Unknown' state, blocking component synchronization.

📋 Technical Details

Immediate Cause

The ArgoCD application 'vector-kubearchive-log-collector-in-cluster-local' failed to deploy or reach a healthy state during the konflux-ci-install-konflux step. The application status was reported as 'Unknown', and critically, it did not exhibit a 'context deadline exceeded' error, suggesting a persistent configuration or deployment issue rather than a simple timeout.

Contributing Factors

No specific contributing factors were identified from the provided build logs or the supplemental diagnostic artifacts. The application's 'Unknown' state without a clear error message suggests a potential internal misconfiguration or an underlying environmental issue that was not captured in the available logs.

Impact

The inability of the vector-kubearchive-log-collector-in-cluster-local ArgoCD application to achieve a healthy state directly prevented the successful synchronization of Konflux components. This failure was a blocker for the konflux-ci-install-konflux step, stopping further execution of the e2e test pipeline.

🔍 Evidence

appstudio-e2e-tests/konflux-ci-install-konflux

Category: configuration
Root Cause: The ArgoCD application 'vector-kubearchive-log-collector-in-cluster-local' failed to deploy correctly or reach a healthy state, remaining in an 'Unknown' state without a 'context deadline exceeded' error, preventing the completion of Konflux component synchronization.

Logs:

artifacts/appstudio-e2e-tests/konflux-ci-install-konflux/build-log.txt

[2026-08-17 16:36:27] [ERROR] Application 'vector-kubearchive-log-collector-in-cluster-local' is in Unknown state without 'context deadline exceeded'

---

Analysis powered by prow-failure-analysis | Build: 2089385918925377536

[raks-tt]

/unhold
/lgtm

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.38%. Comparing base (50fdd86) to head (3b65d33).
⚠️ Report is 92 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #13553   +/-   ##
=======================================
  Coverage   60.38%   60.38%           
=======================================
  Files          24       24           
  Lines        1628     1628           
=======================================
  Hits          983      983           
  Misses        563      563           
  Partials       82       82           

Flag	Coverage Δ
go	60.38% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[nhaklik]

Production Approval Record

Field	Value
Action	APPROVED
Reviewer	@nhaklik
Timestamp	2026-08-17T14:03:10.351Z
Criteria Override	Risk assessment not provided, Staging validation not found

[redhat-appstudio-qe-bot]

❗ Detected an outage of the following critical component(s)❗

• github: API Requests, Pull Requests

Due to this issue E2E tests will probably fail. Please keep an eye on the following status pages:

• https://www.githubstatus.com

and add a comment /retest-required once the reported issues are solved

[eedri]

@martysp21 this PR was approved for production 3d ago but is still open.

Its required CI checks aren't passing yet.

Once that's resolved it should merge; close the PR if it's no longer needed.

[martysp21]

/retest