feat: Clarify Rootless Runtime Requirements

site/content/docs/user/ingress.md

@@ -21,6 +21,10 @@ Ingress exposes HTTP and HTTPS routes from outside the cluster to services withi
 > **NOTE**: You may also want to consider using [Gateway API](https://gateway-api.sigs.k8s.io/) instead of Ingress.
 > Gateway API has an [Ingress migration guide](https://gateway-api.sigs.k8s.io/guides/migrating-from-ingress/).
 
+> **WARNING**: If you are using a [rootless container runtime], ensure your host is
+> properly configured before creating the KinD cluster. Most Ingress and Gateway controllers will
+> not work if these steps are skipped.
+
 ### Create Cluster
 
 #### Option 1: LoadBalancer
@@ -139,3 +143,4 @@ curl localhost/bar
 
 [LoadBalancer]: /docs/user/loadbalancer/
 [Cloud Provider KIND]: /docs/user/loadbalancer/
+[rootless container runtime]: /docs/user/rootless/

site/content/docs/user/quick-start.md

@@ -160,6 +160,10 @@ More usage can be discovered with `kind create cluster --help`.
 kind can auto-detect the [docker], [podman], or [nerdctl] installed and choose the available one. If you want to turn off the auto-detect, use the environment variable `KIND_EXPERIMENTAL_PROVIDER=docker`, `KIND_EXPERIMENTAL_PROVIDER=podman` or `KIND_EXPERIMENTAL_PROVIDER=nerdctl` to
 select the runtime.
 
+> **NOTE**: In some distributions (ex: Fedora), the container runtime operates in
+> [rootless mode](/docs/user/rootless) by default. Extra setup is needed for KinD clusters to be fully
+> functional.
+
 ## Interacting With Your Cluster
 
 After [creating a cluster](#creating-a-cluster), you can use [kubectl][kubectl]
@@ -501,4 +505,4 @@ kind, the Kubernetes cluster itself, etc.
 [Private Registries]: /docs/user/private-registries
 [customize control plane with kubeadm]: https://kubernetes.io/docs/setup/independent/control-plane-flags/
 [access multiple clusters]: https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/
-[release notes]: https://github.com/kubernetes-sigs/kind/releases
+[release notes]: https://github.com/kubernetes-sigs/kind/releases

site/content/docs/user/rootless.md

@@ -9,41 +9,58 @@ menu:
 Starting with kind 0.11.0, [Rootless Docker](https://docs.docker.com/go/rootless/), [Rootless Podman](https://github.com/containers/podman/blob/master/docs/tutorials/rootless_tutorial.md) and [Rootless nerdctl](https://github.com/containerd/nerdctl/blob/main/docs/rootless.md) can be used as the node provider of kind.
 
 ## Provider requirements
+
 - Docker: 20.10 or later
 - Podman: 3.0 or later
 - nerdctl: 1.7 or later
 
 ## Host requirements
+
+### cgroups v2
+
 The host needs to be running with cgroup v2.
 Make sure that the result of the `docker info` command contains `Cgroup Version: 2`.
 If it prints `Cgroup Version: 1`, try adding `GRUB_CMDLINE_LINUX="systemd.unified_cgroup_hierarchy=1"` to `/etc/default/grub` and
 running `sudo update-grub` to enable cgroup v2.
 
-Also, depending on the host configuration, the following steps might be needed:
+Your host may also need to enable [cgroup delegation](https://systemd.io/CGROUP_DELEGATION/) for daemon-based controller runtimes.
+This is not required for daemonless runtimes, such as podman. Note that this procedure may
+[negatively impact performance](https://lists.fedoraproject.org/archives/list/[email protected]/thread/ZMKLS7SHMRJLJ57NZCYPBAQ3UOYULV65/).
 
-- Create `/etc/systemd/system/[email protected]/delegate.conf` with the following content, and then run `sudo systemctl daemon-reload`:
+To enable cgroup delegation, perform the folowing actions:
 
-  ```ini
-  [Service]
-  Delegate=yes
-  ```
+1. As root, create the directory `/etc/systemd/system/[email protected]/` if it does not already exist
+
+   ```sh
+   sudo mdkir -p /etc/systemd/system/[email protected]/
+   ```
+2. As root, create the file `/etc/systemd/system/[email protected]/delegate.conf` with the following content:
+
+   ```ini
+   [Service]
+   Delegate=yes
+   ```
+
+3. Reload the systemd daemon:
+
+   ```sh
+   sudo systemctl daemon-reload
+   ```
 
-  (This is not enabled by default because ["the runtime impact of
-  [delegating the "cpu" controller] is still too
-  high"](https://lists.fedoraproject.org/archives/list/[email protected]/thread/ZMKLS7SHMRJLJ57NZCYPBAQ3UOYULV65/).
-  Beware that changing this configuration may affect system
-  performance.)
+4. If using docker, reload the user docker daemon:
 
-  Please note that:
+   ```sh
+   systemctl --user restart docker
+   ```
 
-  - `/etc/systemd/system/[email protected]/` directory needs to be created if not already present on your host
-  - If using Docker and it was already running when this step was done, a restart is needed for the changes to take
-    effect
-      {{< codeFromInline lang="bash" >}}
-      systemctl --user restart docker
-      {{< /codeFromInline >}}
+### Networking
 
-- Create `/etc/modules-load.d/iptables.conf` with the following content:
+Containers running in rootless mode are not typically loaded with host-level iptable modules.
+This breaks the behavior of most Ingress and Gateway controllers.
+
+To load the iptable modules into the KinD containers, do the following:
+
+1. As root, create the file `/etc/modules-load.d/iptables.conf` with the following content:
 
   ```
   ip6_tables
@@ -52,14 +69,62 @@ Also, depending on the host configuration, the following steps might be needed:
   iptable_nat
   ```
 
-- If using podman, be aware that by default there is a [limit](https://docs.podman.io/en/v4.3/markdown/options/pids-limit.html#pids-limit-limit) to the number of pids that can be created. This can cause problems like nginx workers inside a container not spawning correctly.
-    - If you want to disable this limit, edit your `containers.conf` file (generally located in `/etc/containers/containers.conf`). Note that this could cause things like pid exhaustion to happen on the host machine. Alternatively, change `0` to your desired new limit:
+2. Restart your system to ensure these changes take effect.
+
+### Increase PID Limits
+
+KinD nodes are represented as individual containers on their hosts. Runtimes such as podman set default
+[process id limits](https://docs.podman.io/en/v4.3/markdown/options/pids-limit.html#pids-limit-limit)
+that may be too low for the node or for a pod running on the node. The NGINX ingress controller is
+[particularly susceptible](https://github.com/kubernetes-sigs/kind/issues/3451) to this issue.
+
+To increase the PID limit, do the following:
+
+1. If using podman, edit your `containers.conf` file (generally located in
+   `/etc/containers/containers.conf` or `~/.config/containers/containers.conf`) to increase the PIDs
+   limit to a desired value (default 4096 on most systems).
 
     ```ini
     [containers]
-    pids_limit = 0
+    pids_limit = 65536
     ```
 
+
+### Increase inotify Limits
+
+As documented in [known issues](/docs/user/known-issues/#pod-errors-due-to-too-many-open-files), pods may
+fail by reaching inotify watch and instance limits. Ingress controllers such as NGINX and Contour
+are particularly susceptible to this issue.
+
+To increase the inotify limits, do the following:
+
+1. As root, create a `.conf` file in `/etc/systctl.d` that increases the `fs.inotify` max user settings:
+
+   ```
+   fs.inotify.max_user_watches = 524288
+   fs.inotify.max_user_instances = 512
+   ```
+
+2. Restart your system for these changes to take effect.
+
+
+### Allow Unprivileged Binding to HTTP(S) Ports
+
+If you use the `extraPortMappings` method to provide ingress to your KinD cluster, you can allow
+the KinD container to bind to ports 80 and 443 on the host. User containers cannot bind to these
+ports by default as they are considered privileged.
+
+To allow a KinD node to bind to ports 80 and/or 443 on the host, do the following:
+
+1. As root, create a `.conf` file in `/etc/systctl.d` that lowers the privileged port start number:
+
+   ```
+   net.ipv4.ip_unprivileged_port_start=80
+   ```
+
+2. Restart your system for these changes to take effect.
+
+
 ## Restrictions
 
 The restrictions of Rootless Docker apply to kind clusters as well.