From 278dd59778ed4cef45e75dfbfb3d9847bc41f5eb Mon Sep 17 00:00:00 2001 From: Avinal Kumar Date: Tue, 6 May 2025 16:50:55 +0530 Subject: [PATCH] SHIP-0040: Extending Build Volumes Signed-off-by: Avinal Kumar --- ...g-build-volumes-for-build-and-buildruns.md | 208 ++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 ships/0042-extending-build-volumes-for-build-and-buildruns.md diff --git a/ships/0042-extending-build-volumes-for-build-and-buildruns.md b/ships/0042-extending-build-volumes-for-build-and-buildruns.md new file mode 100644 index 0000000..c4b2cf7 --- /dev/null +++ b/ships/0042-extending-build-volumes-for-build-and-buildruns.md @@ -0,0 +1,208 @@ + + +--- +title: extend-build-volumes-to-mount-directly-to-builds-and-buildruns +authors: + - "@avinal" +reviewers: + - TBD +approvers: + - TBD +creation-date: 2025-04-21 +last-updated: yyyy-mm-dd +status: provisional +see-also: + - "0022-build-strategy-volumes.md" +replaces: + - "/docs/proposals/that-less-than-great-idea.md" +superseded-by: + - "/docs/proposals/our-past-effort.md" +--- + +# Extending Build Volumes to mount directly to Builds and Buildruns +## Release Signoff Checklist + +- [ ] Enhancement is `implementable` +- [ ] Design details are appropriately documented from clear requirements +- [ ] Test plan is defined +- [ ] Graduation criteria for dev preview, tech preview, GA +- [ ] User-facing documentation is created in [docs](/docs/) + +## Open Questions [optional] + + +## Summary + +This proposal aims to enhance Shipwright by allowing users to define additional volumes directly +within Build and BuildRun resources. These volumes can then be mounted into the build containers. +This capability is essential for scenarios requiring access to external resources like self-signed +certificates for Git/image registries or other necessary build artifacts, independent of BuildStrategy +definitions, overcoming a current limitation where all volumes must be pre-defined or overridable via the strategy. + +## Motivation + +Currently, Shipwright Builds face challenges when needing to utilize resources like self-signed +certificates for interacting with private Git repositories or image registries. While BuildStrategy +resources can define volumes that Build and BuildRun resources can override, there's no mechanism +to introduce entirely new volumes directly at the Build or BuildRun level without them being +pre-declared in the BuildStrategy. These strategy-defined volumes, even when overridden, are typically +scoped to the build steps defined by the strategy. + +The original proposal for strategy volumes (SHIP-0022) explicitly listed adding arbitrary volumes to +builds as a non-goal to simplify its initial scope. This proposal addresses that gap. + +### Goals + +- Allow Build and BuildRun resources to declare their own volumes, in addition to those provided or +overridden from the BuildStrategy. +- Ensure these directly defined volumes are mountable and accessible to all primary containers involved +in executing the build steps throughout the lifecycle of the build's execution Pod. + +### Non-Goals + +What is out of scope for this proposal? Listing non-goals helps to focus discussion and make +progress. + +## Proposal + +### User Stories + +#### Using self-signed certificates + +As a user, I want to use Git repositories or image registries that are secured with self-signed +certificates. These certificates need to be accessible by the build process, potentially across +multiple steps (e.g., source cloning, image push/pull). + +#### Volumes indepedndent of Build Strategies + +As a user, I want to provide necessary artifacts or configurations (e.g., custom configuration files, pre-downloaded dependencies) +to my builds using standard Kubernetes volumes (ConfigMap, Secret, EmptyDir). I need the flexibility +to do this directly in my Build or BuildRun without requiring the BuildStrategy to pre-define a +corresponding volume for me to override. + +### Implementation Notes + +The current Build and BuildRun APIs will be extended to support a mountPath field for each volume +entry and to allow volume definitions that are not simply overrides of BuildStrategy volumes. + +The spec.volumes array already exists in Build and BuildRun. + +```yaml +apiVersion: shipwright.io/v1beta1 # Ensure this is the correct API group +kind: Build +metadata: + # ... +spec: + # ... + volumes: + - name: custom-certs + secret: + secretName: my-ca-certs + mountPath: /etc/ssl/certs/custom-ca.crt # Proposed field for direct mount + # For ConfigMap + - name: config + configMap: + name: my-build-configmap + mountPath: /etc/build/config + # For EmptyDir + - name: shared-workspace + emptyDir: {} + mountPath: /workspace/shared +``` + +The current implicit requirement that every volume in Build.spec.volumes or BuildRun.spec.volumes +must correspond to a volume name in the BuildStrategy.spec.volumes will be relaxed. + +- If a volume name in Build or BuildRun matches a volume name defined in the BuildStrategy, the +existing override logic and validation (e.g., ensuring the type is compatible if specified by the strategy) +will apply. The mountPath for these overridden strategy volumes would still be determined by the +BuildStrategy's step definitions. +- If a volume name in Build or BuildRun is new (i.e., does not match any volume in the BuildStrategy), +it will be treated as a direct volume mount. For these volumes, the mountPath field in the Build/BuildRun +volume definition becomes mandatory. + + +Directly defined volumes (those with a mountPath) will be added to the Pod template generated for the build. +These volumes will be mounted into all primary containers responsible for executing the build steps +defined by the BuildStrategy, as well as any essential helper containers injected by Shipwright +(e.g., for source checkout, if applicable and technically feasible to receive these mounts). +This ensures wide availability for common use cases like certificates. + +If a volume with the same name is defined at multiple levels, the following precedence will apply +for its definition (e.g., which Secret or ConfigMap to use): + +- BuildRun.spec.volumes (highest precedence) +- Build.spec.volumes +- BuildStrategy.spec.volumes (lowest precedence, acts as a default or template) + + +- For new, directly mounted volumes, a mountPath must be specified. +- The name of the volume must be unique within the list of directly defined volumes for a given Build or BuildRun. +- Standard Kubernetes validation for the volume source (e.g., configMap, secret) will apply. +- If a mountPath defined in a Build/BuildRun for a direct volume conflicts with a mountPath used by +a BuildStrategy volume, or another direct volume, this should result in a validation error during +Build/BuildRun admission to prevent ambiguity. + + +### Test Plan + + +### Release Criteria + + +#### Upgrade Strategy [if necessary] + + +### Risks and Mitigations + +What are the risks of this proposal and how do we mitigate? Think broadly. For example, consider +both security and how this will impact the larger Shipwright ecosystem. + +How will security be reviewed and by whom? How will UX be reviewed and by whom? + +## Drawbacks + +- All-Step Mounting: Mounting directly defined volumes to all primary build step containers by +default offers simplicity but lacks granularity. If a volume is only needed in one specific step, +it will still be mounted in others. This could slightly clutter the Pod spec and provide +wider-than-necessary access within the Pod's lifecycle. However, for common use-cases like certificates +or shared workspace, this broad availability is often desired. The alternative, per-step mount +configuration, would significantly increase API complexity. +- Potential for mountPath Overlap: While validation should prevent this, users need to be mindful +of mountPaths used by the underlying build tools within the strategy containers to avoid unintentional behavior. + +## Alternatives + +- Dedicated Certificate Management Feature: + + - Description: Introduce a specific API section within Build/BuildRun solely for providing CA + certificates or other trust bundles, rather than using general-purpose volumes. + - Reason for Not Choosing: While this would address the self-signed certificate use case directly, + it is less flexible. General-purpose volumes can solve a wider range of problems, such as providing + configuration files, tools, or shared workspace artifacts, beyond just certificates. The proposed + solution leverages existing Kubernetes volume concepts which are familiar to users. + +- Relying Solely on BuildStrategy Volume Overrides: + + - Description: Not implement this feature and require all volumes to be predefined in BuildStrategy + and then overridden in Build/BuildRun. + - Reason for Not Choosing: This lacks flexibility for users who don't control the BuildStrategy + or when a BuildStrategy is designed to be generic. It forces strategy authors to anticipate all + possible volume needs, which is impractical. + +## Infrastructure Needed [optional] + +Use this section if you need things from the project. Examples include a new subproject, repos +requested, github details, and/or testing infrastructure. + +Listing these here allows the community to get the process for these resources started right away. + +## Implementation History + +Major milestones in the life cycle of a proposal should be tracked in `Implementation History`. + +