Skip to content

feat(charts): add Helm chart for Kubeflow Pipelines#12787

Open
jsonmp-k8 wants to merge 5 commits intokubeflow:masterfrom
jsonmp-k8:helm-chart
Open

feat(charts): add Helm chart for Kubeflow Pipelines#12787
jsonmp-k8 wants to merge 5 commits intokubeflow:masterfrom
jsonmp-k8:helm-chart

Conversation

@jsonmp-k8
Copy link
Contributor

@jsonmp-k8 jsonmp-k8 commented Feb 7, 2026

Summary

  • Add a comprehensive Helm chart (charts/kubeflow-pipelines/) that provides full functional parity with the existing Kustomize manifests
  • Supports all 4 deployment combinations: single/multi-user × MySQL/PostgreSQL × SeaweedFS/MinIO
  • Includes 93 template files covering all KFP components: API server, UI, cache server, metadata (MLMD), persistence agent, scheduled workflow, viewer CRD, visualization server, pipeline runner, metadata writer, cache deployer, SeaweedFS, MySQL, multi-user Istio/RBAC resources, and webhooks

Test plan

  • helm lint passes for all 4 mode combinations
  • helm template renders correct resources for single-user MySQL+SeaweedFS
  • helm template renders correct resources for single-user PostgreSQL+MinIO
  • helm template renders correct resources for multi-user MySQL+SeaweedFS
  • helm template renders correct resources for multi-user PostgreSQL+MinIO
  • Verify Istio AuthorizationPolicies and DestinationRules render only in multi-user mode
  • Verify conditional MySQL/SeaweedFS Istio resources render based on database/objectStore type
  • Verify SeaweedFS NetworkPolicy matches Kustomize restrictive rules
  • Deploy to a test cluster and validate end-to-end pipeline execution

Signed-off-by: Jaison Paul paul.jaison@gmail.com

@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign chensun for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions
Copy link

github-actions bot commented Feb 7, 2026

🎉 Welcome to the Kubeflow Pipelines repo! 🎉

Thanks for opening your first PR! We're excited to have you onboard 🚀

Next steps:

Feel free to ask questions in the comments.

@google-oss-prow
Copy link

Hi @jsonmp-k8. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Add a comprehensive Helm chart that provides full functional parity
with the existing Kustomize manifests. Supports all 4 deployment
combinations: single/multi-user x MySQL/PostgreSQL x SeaweedFS/MinIO.

Signed-off-by: Jaison Paul <paul.jaison@gmail.com>
@jsonmp-k8
Copy link
Contributor Author

#12728

@jeffspahr
Copy link
Contributor

Code Review: Helm Chart for Kubeflow Pipelines

Overview

+4,780 lines, 93 templates, covering all 4 deployment combinations (single/multi-user × MySQL/PostgreSQL × SeaweedFS/MinIO). This is a significant contribution that aims for full functional parity with the existing Kustomize manifests.

I compared each finding against the existing Kustomize manifests to separate genuinely new issues from inherited behavior. Issues that already exist in the Kustomize manifests are noted as such and should not block this PR.


Issues Inherited from Kustomize (Not Blocking)

The following issues exist identically in the Kustomize manifests under manifests/kustomize/. Since the Helm chart is faithfully porting existing behavior, these should be tracked separately if they need fixing:

Issue Kustomize Location
sync.py "True" or "False" precedence bug (always evaluates to "True") base/installs/multi-user/pipelines-profile-controller/sync.py
Webhook port 8443 not exposed on ml-pipeline service (only 8888/8887) base/webhook/*.yaml + base/pipeline/ml-pipeline-apiserver-service.yaml
Empty MySQL root password + MYSQL_ALLOW_EMPTY_PASSWORD=true base/installs/generic/mysql-secret.yaml + third-party/mysql/base/mysql-deployment.yaml
MinIO minio/minio123 default credentials third-party/minio/base/mlpipeline-minio-artifact-secret.yaml
PostgreSQL password default base/metadata/overlays/postgres/secrets.env
Wildcard apiGroups: ["*"] in viewer-crd role base/pipeline/ml-pipeline-viewer-crd-role.yaml
Wildcard verbs: ["*"] in pipeline-runner role base/pipeline/pipeline-runner-role.yaml
seaweedfs AuthorizationPolicy - {} empty rule (bypasses restrictions) third-party/seaweedfs/istio/istio-authorization-policy.yaml
cache-server / metadata-grpc rules: [{}] base/installs/multi-user/istio-authorization-config.yaml
sync.py hardcodes kubeflow namespace Both sync.py files in manifests
Profile controller hardcodes seaweedfs URLs regardless of objectStore type base/installs/multi-user/pipelines-profile-controller/deployment.yaml
sync.py hardcodes bucket name mlpipeline manifests sync.py
seaweedfs `
No resource limits on most components Same across Kustomize deployments
No health probes on cache-deployer, metadata-writer, persistence-agent, viewer-crd, scheduled-workflow Same in Kustomize
Inconsistent imagePullPolicy (Always vs IfNotPresent) Same mix in Kustomize

New Issues Introduced by the Helm Chart

P1 — Should fix before merge

1. Unquoted secret values in templates

templates/object-store-secret.yaml — both keys and values in stringData are unquoted. Special characters (#, :, {) in user-provided access keys will break YAML parsing. The Kustomize manifests don't have this issue because their values are static strings, but in a Helm template with user-provided values, quoting is necessary:

# Current (broken for special chars)
{{ .Values.objectStoreSecret.accessKeyKey }}: {{ .Values.objectStoreSecret.accessKey }}
# Should be
{{ .Values.objectStoreSecret.accessKeyKey }}: {{ .Values.objectStoreSecret.accessKey | quote }}

Same issue in templates/mysql/db-secret.yaml — password is quoted but username is not.

2. Argo workflowNamespaces hardcoded to kubeflow

values.yaml — The Kustomize manifests use a $(kfp-namespace) parameter for the Argo config, but the Helm chart hardcodes kubeflow in the subchart values. If installed in any other namespace, Argo will watch the wrong namespace and miss workflows. This should default to the release namespace.

3. Dead seaweedfs.enabled config knob

values.yaml defines seaweedfs.enabled: true but no template ever checks this value — they all check objectStore.type == "seaweedfs" instead. This dead configuration will confuse users. Either wire it into the conditionals or remove it.

P2 — Should fix (Helm-specific best practices)

4. No fullnameOverride/nameOverride support — Resource names are hardcoded (e.g., ml-pipeline, cache-server). Two releases in the same namespace will collide. This is a Helm-specific concern that doesn't apply to Kustomize.

5. Missing standard Kubernetes labels_helpers.tpl omits app.kubernetes.io/name and app.kubernetes.io/version from the common labels helper. No selectorLabels helper exists either.

6. Committed .tgz subchart archives — 5 binary chart archives are checked into charts/. These should be added to .gitignore and fetched via helm dependency build in CI instead.

7. Loose dependency version rangesmysql: 12.x.x, postgresql: 16.x.x, etc. Bitnami charts frequently have breaking changes in minor bumps. Consider tightening to ~12.3.0-style constraints.

8. No imagePullSecrets support — No global or per-component imagePullSecrets configuration, which blocks private registry usage.

9. No Helm tests — No templates/tests/ directory. A basic connection test for the API server health endpoint would catch deployment failures early.

10. No values.schema.json — With 4 deployment combinations and complex conditionals, a JSON schema would catch configuration errors at helm install time rather than at runtime.

11. publicConfig.kubeflowPipelinesVersion defaults to "dev" — Should default to appVersion (2.15.0).

12. No replicas parameter — All deployments hardcoded to 1 replica with no override in values.

13. No PodDisruptionBudgets — Critical-path components (api-server, metadata-grpc) should have PDB support for production HA.

14. No global values block — No cross-cutting support for node selectors, tolerations, or affinity that would apply to all components.

@jeffspahr
Copy link
Contributor

KEP Alignment Review

I compared this PR against the KEP: Helm Charts for Kubeflow Pipelines to assess how well it aligns with the agreed-upon direction.


Where the PR aligns with the KEP

KEP Requirement PR Status
Charts live in kubeflow/pipelines under /charts/kubeflow-pipelines ✅ Fully aligned
kubeflow/pipelines is the source of truth for chart structure, values, versioning ✅ Fully aligned
Align chart versions with KFP releases appVersion: "2.15.0", version: 0.1.0 is reasonable for a first chart release
Publish as versioned OCI Helm artifacts Deferred — this is a CI/release concern, not chart code. Fine to defer.
Downstream repos (e.g., kubeflow/manifests) may consume/synchronize but won't be the primary owner ✅ Aligned by virtue of living in the pipelines repo

Where the PR diverges from the KEP

1. Minimalism — "Start with only the core components" (High gap)

The KEP explicitly calls for a minimal initial scope:

"For the first iteration, the scope will be intentionally small: core components required for a working Pipelines setup, reasonable default configuration exposed via Helm values, configuration for common use cases only. Additional features and configuration options can be added incrementally over time."

The PR takes the opposite approach — 93 templates covering all 4 deployment combinations (single/multi-user × MySQL/PostgreSQL × SeaweedFS/MinIO) with a compat layer for legacy MySQL deployments. This is comprehensive, not minimal.

A KEP-aligned V1 might look like:

  • Single-user mode only
  • One database backend (e.g., MySQL via Bitnami subchart)
  • One object store (e.g., MinIO or SeaweedFS)
  • Multi-user, PostgreSQL, and the compat layer added in follow-up PRs

2. Maintainability — "Avoid direct one-to-one templating of all existing manifests" (High gap)

The KEP explicitly warns against 1:1 porting:

"Maintainability: Avoid direct one-to-one templating of all existing manifests. Charts should be structured for readability, reviewability, and long-term maintenance."

The PR appears to be a fairly direct port of each Kustomize manifest into a Helm template. Evidence:

  • The compat layer duplicates the third-party MySQL deployment path alongside the Bitnami subchart, creating two code paths for the same component
  • sync.py is included verbatim via .Files.Get, inheriting all hardcoded values rather than being restructured for Helm (namespace, bucket name, endpoints)
  • 93 template files largely mirror the Kustomize directory structure
  • Complex conditionals like {{- if and .Values.compat.enabled .Values.compat.mysql.pvc (or .Values.mysql.enabled .Values.compat.mysql.deployment) }} suggest the compat layer adds significant maintenance burden

A more maintainable approach would consolidate — e.g., pick one DB deployment strategy rather than supporting both compat MySQL and the Bitnami subchart simultaneously.

3. Standardization — "Follow established Kubernetes and Kubeflow best practices" (Medium gap)

Several standard Helm patterns are missing:

  • No fullnameOverride/nameOverride support (two releases in the same namespace will collide)
  • Missing app.kubernetes.io/name and app.kubernetes.io/version standard labels
  • No selectorLabels helper
  • No values.schema.json for input validation
  • No global values block for cross-cutting concerns

These are established patterns in the Helm ecosystem and in other Kubeflow components.

4. Testing and Validation (Medium gap)

The KEP states:

"Helm chart testing infrastructure currently used in kubeflow/manifests should be reused or adapted for kubeflow/pipelines. This may include: Helm template validation, basic installation checks, minimal functional verification."

The PR includes no Helm tests (templates/tests/), no values.schema.json, and no CI integration. The PR description lists a manual test plan with helm lint and helm template checks, but these aren't automated.

5. "This KEP does not assume any existing pull request as the final solution" (Note)

The KEP's Relationship to Existing Helm Work section says:

"This KEP does not assume any existing pull request as the final solution. Instead: existing work can be used as reference or input, final implementations should follow the direction and scope agreed through this KEP."

This means the PR should be evaluated against the KEP's principles, not just on whether it achieves functional parity with Kustomize.


Summary

KEP Principle PR Alignment Gap
Location (kubeflow/pipelines) ✅ Fully aligned
Ownership (pipelines repo is source of truth) ✅ Fully aligned
Version alignment with KFP releases ✅ Aligned
OCI publishing Deferred (OK)
Minimalism (start small, iterate) ❌ Ships all 4 deployment combos + compat layer High
Maintainability (avoid 1:1 templating) ❌ Largely a direct port of Kustomize manifests High
Standardization (K8s/Kubeflow best practices) ⚠️ Missing standard Helm patterns Medium
Testing (template validation, install checks) ⚠️ No automated tests or schema validation Medium

The PR delivers on the KEP's structural goals (location, ownership, versioning) but diverges from its design philosophy. The KEP envisions a minimal, well-structured starting point that grows incrementally, while the PR delivers a comprehensive 1:1 port of all Kustomize variants.

The PR could be brought into alignment by either:

  1. Scoping down: Ship a single deployment combo (e.g., single-user + MySQL + SeaweedFS) as V1, with multi-user and alternative backends in follow-up PRs
  2. Restructuring: If the full scope is kept, refactor to reduce the 1:1 mapping (consolidate the compat layer, template sync.py values via env vars, add standard Helm patterns)

Either path should include values.schema.json and at least basic Helm tests.

@droctothorpe
Copy link
Collaborator

/ok-to-test

jsonmp-k8 added a commit to jsonmp-k8/pipelines that referenced this pull request Feb 11, 2026
P1 fixes:
- Quote all user-provided secret values in object-store-secret and db-secret
- Replace hardcoded kubeflow namespace in Argo workflowNamespaces with
  empty list (defaults to release namespace via singleNamespace: true)
- Remove dead seaweedfs.enabled knob (templates use objectStore.type)

P2 Helm best-practice improvements:
- Add fullnameOverride/nameOverride support in _helpers.tpl
- Add standard K8s labels (app.kubernetes.io/name, version, selectorLabels)
- Add .gitignore for chart archives
- Tighten dependency versions to ~X.Y.0 ranges
- Add global.imagePullSecrets with helper included in all deployments
- Add Helm test for API server health endpoint
- Add values.schema.json for config validation
- Default publicConfig version to .Chart.AppVersion instead of "dev"
- Add per-component replicas parameter in values and deployments
- Add PodDisruptionBudgets for apiServer, metadataGrpc, cacheServer
- Add global nodeSelector/tolerations/affinity with component-level fallback

Signed-off-by: Jaison Paul <paul.jaison@gmail.com>
Copilot AI review requested due to automatic review settings February 11, 2026 04:43
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Helm chart (charts/kubeflow-pipelines/) intended to provide a Helm-based installation path for Kubeflow Pipelines with parity across single-/multi-user modes and different DB/object-store backends.

Changes:

  • Introduces chart metadata, default values, and a values JSON schema.
  • Adds Helm templates for core KFP components (API server, UI, cache, metadata/MLMD, persistence, scheduled workflow, viewer controller) plus multi-user Istio/RBAC/metacontroller resources.
  • Vendors/locks Helm dependencies (Argo Workflows, Bitnami MySQL/PostgreSQL, MinIO, Metacontroller) and adds KFP CRDs.

Reviewed changes

Copilot reviewed 109 out of 115 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
charts/kubeflow-pipelines/Chart.yaml Defines the new Helm chart and its dependencies.
charts/kubeflow-pipelines/Chart.lock Locks dependency versions/digest for reproducible builds.
charts/kubeflow-pipelines/README.md Documents intent and known differences vs Kustomize.
charts/kubeflow-pipelines/values.yaml Adds default configuration for all components and subcharts.
charts/kubeflow-pipelines/values.schema.json Adds a JSON schema for chart values validation.
charts/kubeflow-pipelines/.gitignore Ignores packaged dependency archives under charts/*.tgz.
charts/kubeflow-pipelines/templates/_helpers.tpl Centralizes naming/labels and endpoint/secret helper logic.
charts/kubeflow-pipelines/templates/NOTES.txt Provides post-install guidance for single- vs multi-user.
charts/kubeflow-pipelines/templates/apiserver/deployment.yaml Deploys the KFP API server.
charts/kubeflow-pipelines/templates/apiserver/pdb.yaml Optional PDB for the API server.
charts/kubeflow-pipelines/templates/apiserver/role.yaml Namespaced RBAC Role for API server.
charts/kubeflow-pipelines/templates/apiserver/rolebinding.yaml Binds API server Role to its ServiceAccount.
charts/kubeflow-pipelines/templates/apiserver/sa.yaml ServiceAccount for API server.
charts/kubeflow-pipelines/templates/apiserver/service.yaml Service for API server HTTP/gRPC endpoints.
charts/kubeflow-pipelines/templates/cache/deployment.yaml Deploys the cache server webhook component.
charts/kubeflow-pipelines/templates/cache/pdb.yaml Optional PDB for cache server.
charts/kubeflow-pipelines/templates/cache/role.yaml Namespaced RBAC Role for cache server.
charts/kubeflow-pipelines/templates/cache/rolebinding.yaml Binds cache Role to cache ServiceAccount.
charts/kubeflow-pipelines/templates/cache/sa.yaml ServiceAccount for cache server.
charts/kubeflow-pipelines/templates/cache/service.yaml Service exposing cache server webhook port.
charts/kubeflow-pipelines/templates/cache-deployer/clusterrole.yaml ClusterRole for cache-deployer to manage webhooks/CSRs.
charts/kubeflow-pipelines/templates/cache-deployer/clusterrolebinding.yaml Binds cache-deployer ClusterRole to its SA.
charts/kubeflow-pipelines/templates/cache-deployer/deployment.yaml Deploys cache-deployer controller.
charts/kubeflow-pipelines/templates/cache-deployer/role.yaml Namespaced Role for cache-deployer.
charts/kubeflow-pipelines/templates/cache-deployer/rolebinding.yaml Binds cache-deployer Role to its SA.
charts/kubeflow-pipelines/templates/cache-deployer/sa.yaml ServiceAccount for cache-deployer.
charts/kubeflow-pipelines/templates/compat/argo-binding.yaml Adds Argo RoleBinding compatibility resources.
charts/kubeflow-pipelines/templates/compat/argo-priorityclass.yaml Adds Argo PriorityClass compatibility resource.
charts/kubeflow-pipelines/templates/compat/argo-role.yaml Adds Argo Role compatibility resource.
charts/kubeflow-pipelines/templates/container-builder-sa.yaml Adds ServiceAccount for container-builder.
charts/kubeflow-pipelines/templates/kfp-launcher-configmap.yaml ConfigMap for launcher default pipeline root.
charts/kubeflow-pipelines/templates/kubeflow-pipelines-public-configmap.yaml Public config ConfigMap for version reporting.
charts/kubeflow-pipelines/templates/metadata/envoy-configmap.yaml Envoy config for MLMD gRPC-web proxy.
charts/kubeflow-pipelines/templates/metadata/envoy-deployment.yaml Deploys Envoy proxy in front of MLMD gRPC.
charts/kubeflow-pipelines/templates/metadata/envoy-service.yaml Service for Envoy proxy.
charts/kubeflow-pipelines/templates/metadata/grpc-configmap.yaml ConfigMap exposing MLMD gRPC service host/port.
charts/kubeflow-pipelines/templates/metadata/grpc-deployment.yaml Deploys MLMD gRPC server.
charts/kubeflow-pipelines/templates/metadata/grpc-pdb.yaml Optional PDB for MLMD gRPC server.
charts/kubeflow-pipelines/templates/metadata/grpc-sa.yaml ServiceAccount for MLMD gRPC server.
charts/kubeflow-pipelines/templates/metadata/grpc-service.yaml Service for MLMD gRPC server.
charts/kubeflow-pipelines/templates/metadata-writer/deployment.yaml Deploys metadata-writer controller.
charts/kubeflow-pipelines/templates/metadata-writer/role.yaml Namespaced RBAC Role for metadata-writer.
charts/kubeflow-pipelines/templates/metadata-writer/rolebinding.yaml Binds metadata-writer Role to its SA.
charts/kubeflow-pipelines/templates/metadata-writer/sa.yaml ServiceAccount for metadata-writer.
charts/kubeflow-pipelines/templates/multi-user/api-server-config-configmap.yaml Multi-user API server config (authz, runner SA, etc.).
charts/kubeflow-pipelines/templates/multi-user/apiserver-clusterrole.yaml Multi-user cluster-scoped RBAC for API server.
charts/kubeflow-pipelines/templates/multi-user/cache-clusterrole.yaml Multi-user cluster-scoped RBAC for cache components.
charts/kubeflow-pipelines/templates/multi-user/decorator-controller.yaml Metacontroller DecoratorController for per-namespace installs.
charts/kubeflow-pipelines/templates/multi-user/istio-authorization-policies.yaml Istio AuthorizationPolicies for multi-user isolation.
charts/kubeflow-pipelines/templates/multi-user/istio-destination-rules.yaml Istio DestinationRules for mTLS in multi-user mode.
charts/kubeflow-pipelines/templates/multi-user/metadata-virtual-service.yaml Istio VirtualService routing for metadata endpoints.
charts/kubeflow-pipelines/templates/multi-user/metadata-writer-clusterrole.yaml Cluster RBAC for metadata-writer in multi-user mode.
charts/kubeflow-pipelines/templates/multi-user/persistence-agent-clusterrole.yaml Cluster RBAC for persistence-agent in multi-user mode.
charts/kubeflow-pipelines/templates/multi-user/profile-controller-code-configmap.yaml ConfigMap embedding sync.py for the profile controller.
charts/kubeflow-pipelines/templates/multi-user/profile-controller-deployment.yaml Deploys the multi-user profile controller.
charts/kubeflow-pipelines/templates/multi-user/profile-controller-env-configmap.yaml Env ConfigMap for profile controller behavior.
charts/kubeflow-pipelines/templates/multi-user/profile-controller-service.yaml Service for profile controller webhook.
charts/kubeflow-pipelines/templates/multi-user/scheduled-workflow-clusterrole.yaml Cluster RBAC for scheduled workflow controller in multi-user.
charts/kubeflow-pipelines/templates/multi-user/ui-clusterrole.yaml Cluster RBAC for UI in multi-user.
charts/kubeflow-pipelines/templates/multi-user/view-edit-clusterroles.yaml Aggregated ClusterRoles for Kubeflow Pipelines view/edit.
charts/kubeflow-pipelines/templates/multi-user/viewer-controller-clusterrole.yaml Cluster RBAC for viewer controller in multi-user.
charts/kubeflow-pipelines/templates/multi-user/virtual-service.yaml Istio VirtualService routing for UI in multi-user mode.
charts/kubeflow-pipelines/templates/mysql/db-secret.yaml Creates DB Secret for MySQL/PostgreSQL based on values.
charts/kubeflow-pipelines/templates/mysql/mysql-deployment.yaml Optional compatibility MySQL Deployment (when not using subchart).
charts/kubeflow-pipelines/templates/mysql/mysql-pvc.yaml Optional compatibility PVC for MySQL parity with Kustomize.
charts/kubeflow-pipelines/templates/mysql/mysql-sa.yaml Optional compatibility MySQL ServiceAccount.
charts/kubeflow-pipelines/templates/mysql/mysql-service.yaml Optional compatibility MySQL Service.
charts/kubeflow-pipelines/templates/object-store-secret.yaml Creates object store Secret (unless existingSecret is provided).
charts/kubeflow-pipelines/templates/persistence-agent/deployment.yaml Deploys persistence-agent controller.
charts/kubeflow-pipelines/templates/persistence-agent/role.yaml Namespaced RBAC Role for persistence-agent.
charts/kubeflow-pipelines/templates/persistence-agent/rolebinding.yaml Binds persistence-agent Role to its SA.
charts/kubeflow-pipelines/templates/persistence-agent/sa.yaml ServiceAccount for persistence-agent.
charts/kubeflow-pipelines/templates/pipeline-install-config.yaml ConfigMap wiring DB/object-store/app settings for components.
charts/kubeflow-pipelines/templates/pipeline-runner/role.yaml Namespaced RBAC Role for pipeline-runner.
charts/kubeflow-pipelines/templates/pipeline-runner/rolebinding.yaml Binds pipeline-runner Role to its SA.
charts/kubeflow-pipelines/templates/pipeline-runner/sa.yaml ServiceAccount for pipeline-runner.
charts/kubeflow-pipelines/templates/public-configmap-role.yaml Role granting read access to the public configmap.
charts/kubeflow-pipelines/templates/public-configmap-rolebinding.yaml RoleBinding granting authenticated users access to public config.
charts/kubeflow-pipelines/templates/scheduled-workflow/deployment.yaml Deploys scheduled workflow controller.
charts/kubeflow-pipelines/templates/scheduled-workflow/role.yaml Namespaced RBAC Role for scheduled workflow controller.
charts/kubeflow-pipelines/templates/scheduled-workflow/rolebinding.yaml Binds scheduled workflow Role to its SA.
charts/kubeflow-pipelines/templates/scheduled-workflow/sa.yaml ServiceAccount for scheduled workflow controller.
charts/kubeflow-pipelines/templates/seaweedfs/deployment.yaml Deploys in-chart SeaweedFS (when selected).
charts/kubeflow-pipelines/templates/seaweedfs/minio-compat-service.yaml Provides minio-service compat endpoint for SeaweedFS S3 gateway.
charts/kubeflow-pipelines/templates/seaweedfs/networkpolicy.yaml NetworkPolicy for SeaweedFS ingress restrictions.
charts/kubeflow-pipelines/templates/seaweedfs/pvc.yaml PVC for SeaweedFS storage.
charts/kubeflow-pipelines/templates/seaweedfs/sa.yaml ServiceAccount for SeaweedFS.
charts/kubeflow-pipelines/templates/seaweedfs/service.yaml Service exposing SeaweedFS endpoints.
charts/kubeflow-pipelines/templates/tests/test-api-connection.yaml Adds a Helm test pod for API server health check.
charts/kubeflow-pipelines/templates/ui/configmap.yaml UI config (viewer pod template) differing by multi-user mode.
charts/kubeflow-pipelines/templates/ui/deployment.yaml Deploys the KFP UI and configures object-store credentials/env.
charts/kubeflow-pipelines/templates/ui/role.yaml Namespaced RBAC Role for UI.
charts/kubeflow-pipelines/templates/ui/rolebinding.yaml Binds UI Role to UI ServiceAccount.
charts/kubeflow-pipelines/templates/ui/sa.yaml ServiceAccount for UI.
charts/kubeflow-pipelines/templates/ui/service.yaml Service exposing UI.
charts/kubeflow-pipelines/templates/visualization/deployment.yaml Deploys visualization server.
charts/kubeflow-pipelines/templates/visualization/sa.yaml ServiceAccount for visualization server.
charts/kubeflow-pipelines/templates/visualization/service.yaml Service for visualization server.
charts/kubeflow-pipelines/templates/viewer-crd/deployment.yaml Deploys viewer CRD controller.
charts/kubeflow-pipelines/templates/viewer-crd/role.yaml Namespaced RBAC Role for viewer CRD controller.
charts/kubeflow-pipelines/templates/viewer-crd/rolebinding.yaml Binds viewer CRD Role to its SA.
charts/kubeflow-pipelines/templates/viewer-crd/sa.yaml ServiceAccount for viewer CRD controller.
charts/kubeflow-pipelines/templates/viewer-sa.yaml ServiceAccount for viewer workloads in single-user mode.
charts/kubeflow-pipelines/templates/webhooks/mutating-webhook.yaml Adds PipelineVersion mutating admission webhook configuration.
charts/kubeflow-pipelines/templates/webhooks/validating-webhook.yaml Adds PipelineVersion validating admission webhook configuration.
charts/kubeflow-pipelines/files/sync.py Adds profile controller sync logic (SeaweedFS-oriented).
charts/kubeflow-pipelines/crds/pipelines.yaml Adds Pipelines CRD.
charts/kubeflow-pipelines/crds/pipelineversions.yaml Adds PipelineVersions CRD.
charts/kubeflow-pipelines/crds/scheduledworkflows.yaml Adds ScheduledWorkflows CRD.
charts/kubeflow-pipelines/crds/viewers.yaml Adds Viewers CRD.
charts/kubeflow-pipelines/charts/minio-5.4.0.tgz Vendored MinIO dependency chart archive.
charts/kubeflow-pipelines/charts/metacontroller-helm-4.12.5.tgz Vendored Metacontroller dependency chart archive.

- name: grpc
port: 8887
protocol: TCP
targetPort: 8887
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The webhook configurations in this chart target ml-pipeline on port 8443, but the ml-pipeline Service only exposes ports 8888 and 8887. With failurePolicy: Fail, PipelineVersion CREATE/UPDATE requests will fail because the webhook endpoint is unreachable. Either add the webhook port (8443) to this Service (and ensure the backing pod listens on it), or gate rendering of these webhook configs behind a values flag that defaults to disabled (to match platform-agnostic Kustomize which doesn’t install the webhook resources by default).

Suggested change
targetPort: 8887
targetPort: 8887
- name: webhook
port: 8443
protocol: TCP
targetPort: 8443

Copilot uses AI. Check for mistakes.
- name: http
containerPort: 8888
- name: grpc
containerPort: 8887
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to the webhook configs: the API server Deployment doesn’t declare/expose a container port for 8443 (only 8888/8887). If the API server is expected to serve the admission webhooks, ensure it is configured to listen on 8443 and that the port is exposed so the Service can route traffic.

Suggested change
containerPort: 8887
containerPort: 8887
- name: https-webhook
containerPort: 8443

Copilot uses AI. Check for mistakes.
Comment on lines +3 to +7
metadata:
name: pipelineversions.pipelines.kubeflow.org
labels:
{{- include "kubeflow-pipelines.labels" . | nindent 4 }}
webhooks:
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The webhook configurations are missing the cert-manager CA injection annotation used by the Kustomize cert-manager overlays (e.g., cert-manager.io/inject-ca-from: <ns>/kfp-api-webhook-cert). Without either caBundle or CA injection, the apiserver won’t trust the webhook server cert and admissions will fail. Consider adding an optional annotation/caBundle mechanism (and documenting the expected secret/certificate source) when webhooks are enabled.

Copilot uses AI. Check for mistakes.
Comment on lines +44 to +49
- name: AWS_ENDPOINT_URL
value: http://seaweedfs.{{ .Release.Namespace }}:8111
- name: S3_ENDPOINT_URL
value: http://seaweedfs.{{ .Release.Namespace }}:8333
- name: AWS_REGION
value: us-east-1
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In multi-user mode, the profile-controller Deployment hardcodes SeaweedFS endpoints for AWS_ENDPOINT_URL / S3_ENDPOINT_URL. This breaks the advertised multi-user + MinIO combination (Kustomize uses different env values and even swaps in a different sync.py implementation for MinIO). These endpoints (and possibly the controller script) need to be conditional on .Values.objectStore.type.

Copilot uses AI. Check for mistakes.
Object store endpoint host
*/}}
{{- define "kubeflow-pipelines.objectStoreHost" -}}
minio-service.{{ .Release.Namespace }}
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kubeflow-pipelines.objectStoreHost always returns minio-service.<namespace>, but when .Values.objectStore.type=minio the chart relies on the MinIO subchart, which typically does not create a Service named minio-service unless explicitly overridden. To avoid a broken defaultPipelineRoot/artifact repository config, either (1) configure the minio subchart to use fullnameOverride: minio-service, or (2) compute the endpoint host based on the actual subchart service name and/or create a compatibility Service named minio-service when MinIO is enabled.

Suggested change
minio-service.{{ .Release.Namespace }}
{{- default (printf "minio-service.%s" .Release.Namespace) .Values.objectStore.host -}}

Copilot uses AI. Check for mistakes.
Comment on lines +53 to +62
- name: MINIO_ACCESS_KEY
valueFrom:
secretKeyRef:
name: {{ include "kubeflow-pipelines.objectStoreSecretName" . }}
key: accesskey
- name: MINIO_SECRET_KEY
valueFrom:
secretKeyRef:
name: {{ include "kubeflow-pipelines.objectStoreSecretName" . }}
key: secretkey
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The chart exposes objectStoreSecret.accessKeyKey / secretKeyKey in values, but consumers still reference hard-coded secret keys (accesskey / secretkey). If a user changes the key names, the UI pod will fail to start due to missing keys. Use the configured key names consistently everywhere you reference the secret data.

Copilot uses AI. Check for mistakes.
Comment on lines +393 to +394
},
})
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The profile controller script has a Python syntax error: the Secret object inserted into desired_resources is missing closing braces for the data map and the outer dict. As-is, sync.py will fail to start, breaking multi-user mode.

Suggested change
},
})
},
},
)

Copilot uses AI. Check for mistakes.
P1 fixes:
- Quote all user-provided secret values in object-store-secret and db-secret
- Replace hardcoded kubeflow namespace in Argo workflowNamespaces with
  empty list (defaults to release namespace via singleNamespace: true)
- Remove dead seaweedfs.enabled knob (templates use objectStore.type)

P2 Helm best-practice improvements:
- Add fullnameOverride/nameOverride support in _helpers.tpl
- Add standard K8s labels (app.kubernetes.io/name, version, selectorLabels)
- Add .gitignore for chart archives
- Tighten dependency versions to ~X.Y.0 ranges
- Add global.imagePullSecrets with helper included in all deployments
- Add Helm test for API server health endpoint
- Add values.schema.json for config validation
- Default publicConfig version to .Chart.AppVersion instead of "dev"
- Add per-component replicas parameter in values and deployments
- Add PodDisruptionBudgets for apiServer, metadataGrpc, cacheServer
- Add global nodeSelector/tolerations/affinity with component-level fallback

Signed-off-by: Jaison Paul <paul.jaison@gmail.com>
- Use kindIs "invalid" + ternary instead of | default for global
  scheduling fallback so explicit empty {} / [] overrides work
- Remove per-component nodeSelector/tolerations/affinity defaults
  from values.yaml (nil by default inherits global)
- Regenerate Chart.lock after dependency version range tightening
- Remove no-op nameOverride/fullnameOverride (resource names are
  hardcoded for Kustomize compatibility)

Signed-off-by: Jaison Paul <paul.jaison@gmail.com>
@juliusvonkohout
Copy link
Member

Hello, I see multiple ones. 1. the general one kubeflow/community#832 2. KFP specific ones #12842 #12787 #12723

and some requirements from GSOC https://www.kubeflow.org/events/upcoming-events/gsoc-2026/#project-5-helm-charts

"
Details: This project continues the KSC-approved initiative to provide Kubeflow platform and standalone components via Helm. The goal is to move beyond Kustomize-only deployments to offer minimalistic, maintainable Helm charts that reflect Kustomize defaults 1:1. Key tasks include: developing and testing Helm charts for KFP and Katib, implementing CI/CD testing infrastructure for Helm-based deployments and coordinating with component maintainers to ensure cross-project consistency.

This project will touch most components and continue the helm chart initiative started by Kunal Dugar who also helped a lot with the testing infrastructure. This will therefore also include working with maintainers of other components such as KFP maintainersfor the KFP helm charts, security and scalability topic or Katib maintainers for Katib helm charts. Some have already open PRs and there was a formal vote by the KSC (Kubeflow steering Committee) that we are moving forward with offering Kubeflow platform and standalone components as helm charts. Therefore it is not just the technical part, but also the coordination effort. The goal is to make minimalistic helm charts that are easy to maintain next to kustomize and only expose sensible settings relevant to most users. For the time being the rendered chart default values must replicate kustomize 1:1. The testing infrastructure has already been set up in the GSOC 2025 efforts in kubeflow/manifests where we already have a few helm charts.
"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants