Updated WebSockets KEP to move to GA in 1.35

Author: seans3

keps/prod-readiness/sig-api-machinery/4006.yaml

@@ -3,3 +3,5 @@ alpha:
   approver: "@deads2k"
 beta:
   approver: "@jpbetz"
+stable:
+  approver: "@jpbetz"

keps/sig-api-machinery/4006-transition-spdy-to-websockets/README.md

@@ -96,7 +96,6 @@ tags, and then generate with `hack/update-toc.sh`.
   - [Background: <code>PortForward</code> Subprotocol](#background-portforward-subprotocol)
   - [Proposal: New <code>PortForward</code> Tunneling Subprotocol Version - <code>v2.portforward.k8s.io</code>](#proposal-new-portforward-tunneling-subprotocol-version---v2portforwardk8sio)
   - [Proposal: API Server PortForward -- Stream Tunnel Proxy](#proposal-api-server-portforward----stream-tunnel-proxy)
-  - [Pre-GA: Kubelet <code>StreamTranslatorProxy</code>](#pre-ga-kubelet-streamtranslatorproxy)
   - [Test Plan](#test-plan)
       - [Prerequisite testing updates](#prerequisite-testing-updates)
       - [Unit tests](#unit-tests)
@@ -234,10 +233,6 @@ know that this has succeeded?
 `kubectl exec`, `kubectl attach`, `kubectl cp`, and `kubectl port-forward` for the
 communication leg between `kubectl` and the API Server.
 
-2. Extend the WebSockets communication leg from the API Server to Kubelet. After this
-extension, WebSockets streaming will occur between `kubectl` and Kubelet (proxied
-through the API Server). This plan is described at [Pre-GA: Kubelet](#pre-ga-kubelet-).
-
 ### Non-Goals
 
 <!--
@@ -246,6 +241,8 @@ and make progress.
 -->
 
 1. We will not make *any* changes to current WebSocket based browser/javascript clients.
+2. We will not extend the WebSockets communication leg from the API Server to Kubelet (in
+this KEP).
 
 ## Proposal
 
@@ -266,7 +263,7 @@ the current API Server proxy to translate or tunnel the `kubectl` WebSockets dat
 a SPDY upstream connection. In this way, the cluster components upstream from the
 API Server will not initially need to be changed. We intend to extend the communication
 path for WebSockets streaming from `kubectl` to Kubelet once the the initial leg
-is proven to work (i.e. that it goes GA).
+is proven to work (i.e. that it goes GA), but it will be in another KEP.
 
 ### User Stories (Optional)
 
@@ -510,23 +507,6 @@ as the downstream connection within the dual concurrent `io.Copy` proxying gorou
 The upstream connection is the same SPDY connection to the container (through the
 Kubelet and CRI).
 
-### Pre-GA: Kubelet `StreamTranslatorProxy`
-
-The eventual plan is to incrementally transition all SPDY communication legs to WebSockets.
-After the WebSocket communication leg from `kubectl` to the API Server is proven
-to work, the next communication leg to transition is the one from the API Server to
-the Kubelet. Both the API Server and the Kubelet stream data messages using the
-`UpgradeAwareProxy`. Since the initial plan is to modify the `UpgradeAwareProxy`
-in the API Server to delegate to the `StreamTranslatorProxy`, it will be straightforward
-to transition this next communication leg by moving the integrated `StreamTranslatorProxy`
-from the API Server to the Kubelet.
-
-The final communication leg to transition from SPDY to WebSockets will be the one
-from Kubelet to the Container Runtimes. Since this communication happens within a
-node (using Unix domain sockets), this path is not as critical. But this effort
-will be more work, since it will require modifying not just Kubelet, but **all**
-Container Runtimes.
-
 ### Test Plan
 
 <!--
@@ -735,6 +715,8 @@ in back-to-back releases.
 
 ##### v1.30 RemoteCommand Subprotocol (exec, cp, and attach)
 
+- `kubectl` environment variable KUBECTL_REMOTE_COMMAND_WEBSOCKETS is **ON** by default.
+- API Server feature flag `TranslateStreamCloseWebsocketRequests` is **ON** by default.
 - Additional `exec`, `cp`, and `attach` unit tests completed and enabled.
 - Additional `exec`, `cp`, and `attach` integration tests completed and enabled.
 - Additional `exec`, `cp`, and `attach` e2e tests completed and enabled.
@@ -754,6 +736,12 @@ in back-to-back releases.
 
 #### GA
 
+- `kubectl` environment variables and API Server feature gates are locked to on by default.
+- Deprecate `kubectl` environment variables and API Server feature gates for future removal.
+- Force synthetic RBAC `CREATE` authorization check for WebSocket upgrades on the following
+  subresources: `pods/exec`, `pods/attach`, and `pods/portforward`. This additional check
+  will be gated by the API Server `ForceRBACCreateCheck` feature flag, which defaults to
+  **TRUE**.
 - Add WebSocket support for HTTPS proxies.
   - See (https://github.com/kubernetes/kubernetes/issues/126134)
 - Conformance tests for `RemoteCommand` completed and enabled.
@@ -762,7 +750,6 @@ in back-to-back releases.
 - Conformance tests for `PortForward` completed and enabled.
 - Conformance tests for `PortForward` have been stable and
   non-flaky for two weeks.
-- Extend the WebSockets communication leg from the API Server to Kubelet.
 
 ### Upgrade / Downgrade Strategy
 
@@ -892,6 +879,8 @@ well as the [existing list] of feature gates.
   KUBECTL_REMOTE_COMMAND_WEBSOCKETS, TranslateStreamCloseWebsocketRequests
   - Feature gate name(s) for PortForward Subprotocol:
   KUBECTL_PORT_FORWARD_WEBSOCKETS, PortForwardWebsockets
+  - Feature gate name(s) for subresource endpoints `pods/exec`, `pods/attach`,
+  and `pods/portforward`: ForceRBACCreateCheck
 - Components depending on the feature gate: kubectl, API Server
 
 ###### Does enabling the feature change any default behavior?
@@ -910,7 +899,10 @@ variable set to **ON** for `exec`, `cp`, and `attach` commands. While the
 KUBECTL_PORT_FORWARD_WEBSOCKETS environment variable must be set to **ON** for
 `port-forward` command. These modifications, however, will be transparent to the
 user unless the `kubectl`/API Server communication is communicating through an
-intermediary such as a proxy (which is the whole reason for the feature).
+intermediary such as a proxy (which is the whole reason for the feature). The API Server
+feature flag `ForceRBACCreateCheck` forces a synthetic, secondary RBAC check for the `CREATE`
+verb permission on WebSocket upgrade requests. When this feature gate is **TRUE**, the additional
+permission check will apply to endpoints `pods/exec`, `pods/attach`, and `pods/portforward`.
 
 ###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?
 
@@ -928,7 +920,9 @@ NOTE: Also set `disable-supported` to `true` or `false` in `kep.yaml`.
 The features can be disabled for a single user by setting the `kubectl` environment
 variable associated with the feature to **OFF**. Or the features can be turned off
 for all `kubectl` users communicating with a cluster by turning off the feature flags
-for the API Server.
+for the API Server. A cluster operator can disable the more stringent permissions for
+subresources `pods/exec`, `pods/attach`, and `pods/portforward` by setting the
+`ForceRBACCreateCheck` feature flag to **FALSE**.
 
 ###### What happens if we reenable the feature if it was previously rolled back?
 
@@ -957,6 +951,9 @@ https://github.com/kubernetes/kubernetes/pull/97058/files#diff-7826f7adbc1996a05
 - There are unit tests in the API Server which exercise the feature gate within
   the `UpgradeAwareProxy`, which conditionally delegates to the `StreamTunneling`
   proxy for the PortForward subprotocol.
+- There will be unit tests in the API Server to verify the feature gate
+  forcing more stringent RBAC checks for `pods/exec`, `pods/attach`, and
+  `pods/portforward`.
 
 ### Rollout, Upgrade and Rollback Planning
 
@@ -1455,6 +1452,8 @@ Major milestones might include:
 - First Kubernetes release where PortForward over WebSockets described in KEP: v1.30
 - PortForward over WebSockets shipped as alpha: v1.30
 - PortForward over WebSockets shipped as beta: v1.31
+- WebSocket HTTPS Proxy functionality shipped: v1.33
+- Synthetic RBAC `CREATE` authz check for WebSocket upgrade requests: v1.35
 
 ## Drawbacks
 

keps/sig-api-machinery/4006-transition-spdy-to-websockets/kep.yaml

@@ -22,13 +22,13 @@ stage: beta
 # The most recent milestone for which work toward delivery of this KEP has been
 # done. This can be the current (upcoming) milestone, if it is being actively
 # worked on.
-latest-milestone: "v1.31"
+latest-milestone: "v1.35"
 
 # The milestone at which this feature was, or is targeted to be, at each stage.
 milestone:
   alpha: "v1.29"
   beta: "v1.31"
-  stable: "v1.32"
+  stable: "v1.36"
 
 # The following PRR answers are required at alpha release
 # List the feature gate name and the components for which it must be enabled
@@ -45,6 +45,9 @@ feature-gates:
   - name: PortForwardWebsockets
     components:
       - kube-apiserver
+  - name: ForceRBACCreateCheck
+    components:
+      - kube-apiserver
 disable-supported: true
 
 # The following PRR answers are required at beta release.