Merge pull request #589 from karlkfi/karl-readme-update

chore: Update readme with features and packages

Author: k8s-ci-robot

README.md

@@ -1,27 +1,38 @@
 # cli-utils
 
-The cli-utils repository contains an actuation library, which wraps `kubectl apply` code.
-This library allows importers to easily execute `kubectl apply`, while also
-addressing several deficiencies of the current implementation of `kubectl apply`.
-The library enhances `kubectl apply` in the following ways:
+`cli-utils` is a collection of Go libraries designed to facilitate bulk
+actuation of Kubernetes resource objects by wraping and enahancing
+`kubectl apply` with a more user friendly abstraction.
 
-1. **Pruning**: adds new, experimental automatic object deletion functionality.
-2. **Sorting**: adds optional resource sorting functionality to apply or delete objects
-in a particular order.
-3. **Apply Time Mutation**: adds optional functionality to dynamically substitute fields
-from one resource config into another.
+While the name incidates a focus on CLI utilities, the project has evolved to
+encompass a broader scope, including CLI use and server-side use in GitOps
+controllers.
 
-TODO(seans): Add examples of API, once we've achieved an alpha API.
+## Features
+
+1. **Pruning**
+1. **Status Interpretation**
+1. **Status Lookup**
+1. **Diff & Preview**
+1. **Waiting for Reconciliation**
+1. **Resource Ordering**
+1. **Explicit Dependency Ordering**
+1. **Implicit Dependency Ordering**
+1. **Apply Time Mutation**
+1. **CLI Printers**
 
 ### Pruning
 
+The Applier automatically deletes objects that were previously applied and then
+removed from the input set on a subsequent apply.
+
 The current implementation of `kubectl apply --prune` uses labels to identify the
 set of previously applied objects in the prune set calculation. But the use of labels
 has significant downsides. The current `kubectl apply --prune` implemenation is alpha,
-and it is improbable that it will graduate to beta. This library attempts to address
+and it is improbable that it will graduate to beta. `cli-utils` attempts to address
 the current `kubectl apply --prune` deficiencies by storing the set of previously
 applied objects in an **inventory** object which is applied to the cluster. The
-**inventory** object is a `ConfigMap` with the `inventory-id` label, and references
+reference implimentation uses a `ConfigMap` as an **inventory** object, and references
 to the applied objects are stored in the `data` section of the `ConfigMap`.
 
 The following example illustrates a `ConfigMap` resource used as an inventory object:
@@ -47,15 +58,72 @@ metadata:
     cli-utils.sigs.k8s.io/inventory-id: 46d8946c-c1fa-4e1d-9357-b37fb9bae25f
 ```
 
-### Apply Sort Ordering
+### Status Interpretation
+
+The `kstatus` library can be used to read an object's current status and interpret
+whether that object has be reconciled (aka Current) or not, including whether it
+is expected to never reconcile (aka Failed).
+
+### Status Lookup
+
+In addition to performing interpritation of status from an object in-memory,
+`cli-utils` can also be used to query status from the server, allowing you to
+retrieve the status of previously or concurrently applied objects.
+
+### Diff & Preview
+
+`cli-utils` can be used to compare local object manifests with remote objects
+from the server. These can be compared locally with diff or remotely with
+preview (aka dry-run). This can be useful for discovering drift or previewing
+which changes would be made, if the loal manifests were applied.
+
+### Waiting for Reconciliation
+
+The Applier automatically watches applied and deleted objects and tracks their
+status, blocking until the objects have reconciled, failed, or been fully
+deleted.
+
+This functionality is similar to `kubectl delete <resource> <name> --wait`, in
+that is waits for all finalizers to complete, except it also works for creates
+and updates.
 
-Adding an optional `config.kubernetes.io/depends-on: <OBJECT>` annotation to a
-resource config provides apply ordering functionality. After manually specifying
-the dependency relationship among applied resources with this annotation, the
-library will sort the resources and apply/prune them in the correct order.
-Importantly, the library will wait for an object to reconcile successfully within
-the cluster before applying dependent resources. Prune (deletion) ordering is
-the opposite of apply ordering.
+While there is a `kubectl apply <resource> <name> --wait`, it only waits for
+deletes when combined with `--prune`. `cli-utils` provides an alternative that
+works for all spec changes, waiting for reconciliation, the convergence of
+status to the desired specification. After reconciliation, it is expected that
+the object has reached a steady state until the specification is changed again.
+
+### Resource Ordering
+
+The Applier and Destroyer use resource type to determine which order to apply
+and delete objects.
+
+In contrast, when using `kubectl apply`, the objects are applied in alphanumeric
+order of their file names, and top to bottom in each file. With `cli-utils`,
+this manual sorting is unnecessary for many common use cases.
+
+### Explicit Dependency Ordering
+
+While resource ordering provides a smart default user experience, sometimes
+resource type alone is not enough to determine desired ordering. In these cases,
+the user can use explicit dependency ordering by adding a
+`config.kubernetes.io/depends-on: <OBJECT_REFERENCE>` annotation to an object.
+
+The Applier and Destroyer use these explicit dependency directives to build a
+dependency tree and flatten it for determining apply ordering. When deleting,
+the order is reversed, ensuring that dependencies are not deleted before the
+objects that depend on them (aka dependents).
+
+In addition to ordering the applies and deletes, dependency ordering also waits
+for dependency reconciliation when applying and deletion finalization when
+deleting. This ensures that dependencies are not just applied first, but have
+reconciled before their dependents are applied. Likewise, dependents are not
+just deleted first, but have completed finalization before their dependencies
+are deleted.
+
+Also, because dependency ordering is enforced during actuation, a dependency
+cannot be pruned by the Applier unless all its dependents are also deleted. This
+prevents accidental premature deletion of objects that are still in active use.
 
 In the following example, the `config.kubernetes.io/depends-on` annotation
 identifies that `pod-c` must be successfully applied prior to `pod-a`
@@ -74,19 +142,46 @@ spec:
       image: k8s.gcr.io/pause:2.0
 ```
 
+### Implicit Dependency Ordering
+
+In addition to being able to specify explicit dependencies, `cli-utils`
+automatically detects some implicit dependencies.
+
+Implicit dependencies include:
+
+1. Namespace-scoped resource objects depend on their Namespace.
+2. Custom resource objects depend on their Custom Resource Definition
+
+Like resource ordering, implicit dependency ordering improves the apply and
+delete experience to reduce the need to manually specify ordering for many
+common use cases. This allows more objects to be applied together all at once,
+with less manual orchestration.
+
 ### Apply-Time Mutation
 
-**apply-time mutation** functionality allows library users to dynamically fill in
-resource field values from one object into another, even though they are applied
-at the same time. By adding a `config.kubernetes.io/apply-time-mutation` annotation,
-a resource specifies the field in another object as well as the location for the
-local field subsitution. For example, if an object's IP address is set during
-actuation, another object applied at the same time can reference that IP address.
-This functionality leverages the previously described **Apply Sort Ordering** to
-ensure the source resource field is populated before applying the target resource.
+The Applier can dynamically modify objects before applying them, performing
+field value substitution using input(s) from dependency fields.
+
+This allows for applying objects together in a set that you would otherwise need
+to seperate into multiple sets, with manual modifications between applies.
 
-In the following example, `pod-a` will substitute the IP address/port from the
-source `pod-b` into the `pod-a` SERVICE_HOST environment variable:
+Apply-Time Mutation is configured using the
+`config.kubernetes.io/apply-time-mutation` annotation on the target object to be
+modified. The annotation may specify one or more substitutions. Each
+substitution inncludes a source object, and source field path, and a parget
+field path, with an optional token. 
+
+If the token is specified, the token is
+replaced in the target field value string with the source field value. If the
+token is not specified, the whole target field value is replaced with the
+source field value. This alternatively allows either templated interpretation or
+type preservation.
+
+The source and target field paths are specified using JSONPath, allowing for
+robust navigation of complex resource field hierarchies using a familiar syntax.
+
+In the following example, `pod-a` will substitute the IP address and port from
+the spec and status of the source `pod-b` into the spec of the target `pod-a`:
 
 ```yaml
 kind: Pod
@@ -119,6 +214,62 @@ spec:
       value: "${pob-b-ip}:${pob-b-port}"
 ```
 
+The primary reason to do this with Apply-Time Mutation, instead of client-side
+manifest templating is that the pod IP is populated by a controller at runtime
+during reconciliation, and is not known before applying. 
+
+That said, this is a toy example using built-in types. For pods, you probably
+actually want to use DNS for service discovery instead. 
+
+Most use cases for Apply-Time Mutation are actually using custom resources, as a
+temporary alternative to building higher level abstractions, modifying
+interfaces, or creating dependencies between otherwise independent interfaces.
+
+### CLI Printers
+
+Since the original intent of `cli-utils` was to contain common code for CLIs,
+and end-to-end testing requires a reference implimentation, a few printers are
+included to translate from the primary event stream into STDOUT text:
+
+1. **Event Printer**: The event printer just prints text to STDOT whenever an
+    event is recieved.
+1. **JSON Printer**: The JSON printer converts events into a JSON string per
+    line, intended for automated interpretation by machine.
+1. **Table Printer**: The table  printer writes and updates in-place a table
+    with one object per line, intended for human consumption.
+
+## Packages
+
+├── **cmd**: the kapply CLI command
+├── **examples**: examples that serve as additional end-to-end tests using mdrip
+├── **hack**: hacky scripts used by make
+├── **pkg**
+│   ├── **apis**: API resources that satisfy the kubernetes Object interface
+│   ├── **apply**: bulk applier and destroyer
+│   ├── **common**: placeholder for common tools that should probably have their own package
+│   ├── **config**: inventory config bootstrapping
+│   ├── **errors**: error printing
+│   ├── **flowcontrol**: flow control enablement discovery
+│   ├── **inventory**: inventory resource reference implimentation
+│   ├── **jsonpath**: utility for using jsonpath to read & write Unstructured object fields
+│   ├── **kstatus**: object status event watcher with ability to reduce status to a single enum
+│   ├── **manifestreader**: bolk resource object manifest reading and parsing
+│   ├── **multierror**: error composition
+│   ├── **object**: library for dealing with Unstructured objects
+│   ├── **ordering**: sort functionality for objects
+│   ├── **print**: CLI output
+│   ├── **printers**: CLI output
+│   └── **testutil**: utility for facilitating testing
+├── **release**: goreleaser config
+├── **scripts**: scripts used by make
+└── **test**: end-to-end and stress tests
+
+## kapply
+
+To facilitate testing, this repository includes a reference CLI called `kapply`.
+The `kapply` tool is not intended for direct consumer use, but may be useful
+when trying to determine how to best utilize the `cli-utils` library packages.
+
 ## Community, discussion, contribution, and support
 
 Learn how to engage with the Kubernetes community on the [community page](http://kubernetes.io/community/).