Skip to content

[PodLevelResources] Add validation for Windows OS #133046

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

toVersus
Copy link
Contributor

What type of PR is this?

/kind bug

What this PR does / why we need it:

This PR is taking over from #132627.

Adds the following checks based on the KEP description:

  • API server to reject Pod with PodLevelResources if Pod targets Windows OS.
  • kubelet to reject Pod with PodLevelResources on nodes other than Linux OS at admission phase.

Which issue(s) this PR is related to:

Fixes: #132582

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Added validation to reject Pods using the `PodLevelResources` feature on Windows OS due to lack of support. The API server rejects Pods with Pod-level resources and a `Pod.spec.os.name` targeting Windows. Kubelet on nodes running Windows also rejects Pods with Pod-level resources at admission phase.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[KEP]: https://github.com/kubernetes/enhancements/issues/2837

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jul 18, 2025
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot
Copy link
Contributor

Hi @toVersus. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jul 18, 2025
@k8s-ci-robot k8s-ci-robot added area/kubelet sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/node Categorizes an issue or PR as relevant to SIG Node. labels Jul 18, 2025
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Jul 18, 2025
@github-project-automation github-project-automation bot moved this to Needs Triage in SIG Apps Jul 18, 2025
@toVersus
Copy link
Contributor Author

/assign @tallclair @ndixita

@toVersus toVersus changed the title Reject windows in api server [PodLevelResources] Add validation for Windows OS Jul 18, 2025
Copy link
Contributor

@HirazawaUi HirazawaUi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that validatePodResources currently doesn't have a dedicated unit test. Given that it's complex enough, would you be willing to add one?

@toVersus toVersus force-pushed the reject-windows-in-api-server branch from 14f7dab to d81b37b Compare July 18, 2025 05:43
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: toVersus
Once this PR has been reviewed and has the lgtm label, please ask for approval from tallclair and additionally assign msau42 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@toVersus
Copy link
Contributor Author

I noticed that validatePodResources currently doesn't have a dedicated unit test. Given that it's complex enough, would you be willing to add one?

Thanks for the suggestion! I've added a new unit test specifically for validatePodResources. Since unit tests for the functions called within validatePodResources already exist, I focused on adding basic test cases and ones focused on validatePodResources.

@toVersus toVersus force-pushed the reject-windows-in-api-server branch from d81b37b to 02399ed Compare July 18, 2025 06:55
@HirazawaUi
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 18, 2025
Reject Pod with PodLevelResources in spec if Pod targets Windows OS.
@toVersus toVersus force-pushed the reject-windows-in-api-server branch 2 times, most recently from f043643 to bf4bc17 Compare July 18, 2025 13:57
@tallclair tallclair moved this to Needs Review in SIG Node: Pod Level Resources Jul 18, 2025
@@ -1009,6 +1010,9 @@ func NewMainKubelet(kubeCfg *kubeletconfiginternal.KubeletConfiguration,
// AppArmor is a Linux kernel security module and it does not support other operating systems.
klet.appArmorValidator = apparmor.NewValidator()
handlers = append(handlers, lifecycle.NewAppArmorAdmitHandler(klet.appArmorValidator))
} else if goos == "windows" {
// PodLevelResources feature is not supported for Windows
handlers = append(handlers, lifecycle.NewPodLevelResourcesHandler())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to take a slightly different approach to this:

  1. Create a new generic handler for checking feature compatibility, maybe something like PodFeaturesAdmitHandler.
  2. Create a features_{linux,windows,unsupported}.go in the lifecycle directory, similar to what was in https://github.com/kubernetes/kubernetes/pull/132627/files#diff-00fbb8cb4fcf4ea260d7497ed65513ff4ac77278797330b0ec6383e474c9b579 (don't reuse the one in allocation_manager though)
  3. The generic admission handler should check if pod-level resources are set, and if so call the features helper to see whether they're supported.

Does that make sense?

Copy link
Contributor Author

@toVersus toVersus Jul 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I tried making the changes. Is this along the lines of what you were expecting? If so, I'll add unit tests if needed.

Reject pods with PodLevelResources running on Windows nodes
at kubelet admission phase
@toVersus toVersus force-pushed the reject-windows-in-api-server branch from bf4bc17 to fb3bafc Compare July 19, 2025 00:14
@HirazawaUi
Copy link
Contributor

Do you also need to update the comment for the podSpec.Resources field in the API file to indicate that it is not supported on Windows OS?

@toVersus
Copy link
Contributor Author

@thockin
Should the limitation that this field cannot be set on Windows pods be documented in the API docs? Windows support may be relaxed in the future, and there are other functional limitations as well. So I'm wondering how much detail we should include.
Personally, I feel that it would be sufficient to document it under Limitations section in the official documentation, like what's done for InPlacePodVerticalScaling. InPlacePodVerticalScaling also rejects Windows pods that specify the resizePolicy field through validation, but the API docs do not mention that Windows is unsupported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/node Categorizes an issue or PR as relevant to SIG Node. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
Status: Needs Triage
Status: Needs Review
Development

Successfully merging this pull request may close these issues.

[PodLevelResources] Validation for Windows OS
6 participants