Skip to content

feat: Create Makefile target for a fresh kind cluster #413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: notebooks-v2
Choose a base branch
from

Conversation

atheo89
Copy link

@atheo89 atheo89 commented Jun 10, 2025

Solves: #409

This Makefile target automates the creation and configuration of a local Kubernetes cluster using Kind. It's designed to set up a development environment with essential components pre-installed.

Key Features

  • Cluster Creation: It creates a Kind Kubernetes cluster. By default, the cluster is named kind-notebooks, but you can customize this by setting the CLUSTER_NAME variable (e.g., make kind-notebooks CLUSTER_NAME=my-dev-cluster).

  • Cert-Manager Installation: Automatically installs cert-manager, a crucial component for issuing and managing TLS certificates within the Kubernetes cluster. The Makefile waits for cert-manager to be fully available before proceeding.
    Note: The deployment of cert-manager should be patched by appending the --kubelet-insecure-tls arg, because the Metrics Server in Kind clusters, especially when the Kubelets' certificates don't contain the node's IP addresses as Subject Alternative Names (SANs) cause the following error:
    E0610 11:58:11.888437 1 scraper.go:149] "Failed to scrape node" err="Get \"https://10.89.0.22:10250/metrics/resource\": tls: failed to verify certificate: x509: cannot validate certificate for 10.89.0.22 because it doesn't contain any IP SANs" node="kind-notebooks-control-plane" I0610 11:58:18.413843 1 server.go:191] "Failed probe" probe="metric-storage-ready" err="no metrics to serve"

  • Metrics Server Installation (Optional): By default, it installs the Metrics Server, which enables Kubernetes to collect resource usage data from nodes and pods. This installation can be skipped by setting the INSTALL_METRICS_SERVER variable to false (e.g., make kind-notebooks INSTALL_METRICS_SERVER=false).

  • Kubectl Context: After setting up the cluster and its components, the Makefile automatically sets your kubectl context to the newly created Kind cluster, making it easy to interact with it immediately.
    Idempotency: The kind delete cluster command at the beginning ensures that if a cluster with the same name already exists, it's removed before a new one is created. This makes the target idempotent, meaning you can run it multiple times without issues.

How to Use
To use this Makefile, you'll need Kind and kubectl installed on your system.

To create the cluster with default settings:
make setup-kind-cluster

To create a cluster with a custom name:

make setup-kind-cluster CLUSTER_NAME=my-custom-cluster

To create a cluster without installing the Metrics Server:
make setup-kind-cluster INSTALL_METRICS_SERVER=false

Output logs of the target:

controller/ git:(local-setup-kind) $ make kind-notebooks 
KIND_EXPERIMENTAL_PROVIDER=podman kind delete cluster --name kind-notebooks || true
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Deleting cluster "kind-notebooks" ...
Deleted nodes: ["kind-notebooks-control-plane"]
KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster --name kind-notebooks
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Creating cluster "kind-notebooks" ...
 ✓ Ensuring node image (kindest/node:v1.32.2) 🖼 
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to "kind-kind-notebooks"
You can now use your cluster with:

kubectl cluster-info --context kind-kind-notebooks

Have a nice day! 👋
--- Installing cert-manager ---
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.yaml
namespace/cert-manager created
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrole.rbac.authorization.k8s.io/cert-manager-cluster-view created
clusterrole.rbac.authorization.k8s.io/cert-manager-view created
clusterrole.rbac.authorization.k8s.io/cert-manager-edit created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created
role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
role.rbac.authorization.k8s.io/cert-manager:leaderelection created
role.rbac.authorization.k8s.io/cert-manager-tokenrequest created
role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager-cert-manager-tokenrequest created
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
service/cert-manager-cainjector created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
kubectl wait --for=condition=Available deployment/cert-manager -n cert-manager --timeout=300s
deployment.apps/cert-manager condition met
Cert Manager installed. Version: v1.17.2
--- Installing metrics-server ---
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
kubectl patch deployment metrics-server --namespace kube-system --type='json' --patch='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--kubelet-insecure-tls"}]'
deployment.apps/metrics-server patched
kubectl wait --for=condition=Available deployment/metrics-server -n kube-system --timeout=300s
deployment.apps/metrics-server condition met
Metrics Server installed. Version: v0.7.2
--- Setting the current context to kind-notebooks cluster ---
kubectl config set-context kind-kind-notebooks --namespace=workspace-controller-system
Context "kind-kind-notebooks" modified.

Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kimwnasptd for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@atheo89 atheo89 force-pushed the local-setup-kind branch from 94fdbf5 to adbe8e1 Compare June 10, 2025 14:08
@atheo89 atheo89 changed the title Create Makefile target for a fresh kind cluster feat: Create Makefile target for a fresh kind cluster Jun 10, 2025
@andyatmiami
Copy link
Contributor

/lgtm

testing invoking this under a variety of scenarios as observed expected behavior:

  • gmake kind-notebooks CONTAINER_TOOL=podman CLUSTER_NAME=kind-e2e
  • gmake kind-notebooks CONTAINER_TOOL=podman CLUSTER_NAME=kind-e2e INSTALL_METRICS_SERVER=false
  • gmake kind-notebooks CONTAINER_TOOL=podman

@atheo89 atheo89 force-pushed the local-setup-kind branch from ec1b1e6 to 130a9d8 Compare June 13, 2025 11:30
@atheo89
Copy link
Author

atheo89 commented Jun 13, 2025

Hey, I made a few additional updates in this PR:

  • Added support for Kind config files — the cluster name now defaults to the name: field in the specified config file, but can still be overridden if needed. In the future may be used to configure many other things.
  • Renamed the Makefile target to setup-kind-cluster for clarity.
  • Introduced a new utils/ directory to organize helper scripts and configuration files, making the setup easier to maintain and extend.

Let me know if you have any feedback or suggestions!

@andyatmiami
Copy link
Contributor

/lgtm

Re-verified functionality based on the rework done - all operating as expected.

Following invocations were tested:

  • gmake setup-kind-cluster CONTAINER_TOOL=podman
  • gmake setup-kind-cluster CONTAINER_TOOL=podman CLUSTER_NAME=notebooks-e2e
  • gmake setup-kind-cluster CONTAINER_TOOL=podman CLUSTER_NAME=notebooks-e2e INSTALL_METRICS_SERVER=false
  • gmake setup-kind-cluster CONTAINER_TOOL=podman CONFIG_FILE=/Users/astonebe/Development/Test/kind-notebooks-config.yaml

On a personal note - recently a MacOS update or something hosed my podman machine - and I had to blow away my kind cluster... I used this PR to get me back up and running 💯

@google-oss-prow google-oss-prow bot added the lgtm label Jun 24, 2025
Copy link
Contributor

@andyatmiami andyatmiami left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can appreciate some folks may not "like" the presence of the workspaces/controller/utils directory and/or feel it should be contained within a different folder location - but the proposed layout in this PR seems reasonable to me. I personally prefer keeping as much scripting as possible outside the Makefile - so I like how this has been broken down.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Needs Triage
Development

Successfully merging this pull request may close these issues.

2 participants