Skip to content

Conversation

@gjenkins8
Copy link
Member

No description provided.

@gjenkins8 gjenkins8 force-pushed the gjenkins/registries.conf_support branch from 99c6d56 to cea4a1d Compare February 27, 2025 15:54
Copy link

@mtrmac mtrmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some drive-by notes.

hips/hip-XXXX.md Outdated
Comment on lines 50 to 62
```toml
# registries.conf
[[registry]]
prefix = "oci.example.com"
```
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In registries.conf, it’s not necessary to add a no-op entry for a registry. (In fact it’s currently somewhat annoying to do, although that would probably be best fixed in the parser package.)

```

```json
# auth.json
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For better or worse, auth.json is intentionally placed into tmpfs filesystems cleared on reboot. (The tools may read other locations in the home directory, but they, without specific instruction otherwise, don’t update them.)

That might be a fairly disruptive change to users of “login”, so it probably should be discussed as such.

hips/hip-XXXX.md Outdated

Including support for repository prefixes (allowing different credentials for different prefixes), registry mirrors, and registry "aliasing". Features Helm would like to introduce. But is currenly blocked by a lack of mechanism to store detail

(in particular, the existing registry configuration Helm uses, Docker’s `$HOME/.docker/config.json`, etc do not support registry aliases nor prefixes)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is do not support registry aliases nor prefixes the only limitation of Docker’s $HOME/.docker/config.json? I would expect that this proposal could clarify more on the problems and limitations here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FeynmanZhou are there other limitations you want to highlight?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what limitations @FeynmanZhou is referring to, but one limitation this may or may not address is namespaced auth. Like with Harbor, you might have one credential for oci://localhost/production and another for oci://localhost/development and config.json as I recall only supports auth at the domain level.

hips/hip-XXXX.md Outdated
## Specification

Helm will utilize the `registries.conf` specification when determinging OCI registry information (authentication credentials, etc):
<https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to reference a versioned link instead of main branch in case the link is broken in the main branch

hips/hip-XXXX.md Outdated

## Open issues

- Support for registies.conf in ORAS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we use ORAS for this, do we need to get agreement from them before this moves forward?

Copy link

@FeynmanZhou FeynmanZhou Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mattfarina ,

@sabre1041 brought up this proposal to the ORAS community meeting on Mar 5. There is also a GitHub issue tracked in oras-go oras-project/oras-go#918.

As this is a dependency for ORAS from Helm v4, what's the expected timeline to getregisties.conf supported in oras-go?

Copy link
Member

@scottrigby scottrigby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of supporting this in Helm.

We may want to note in this HIP that registry aliases would help with airgapped environments—an important use case for Helm users—where an in-cluster registry would replace external registries defined as the storage for chart dependencies.

There are existing means in Helm to overwrite a registry for container images in airgap—many charts make images/oci registries configurable, but even if they do not, this can be done with a post-renderer. However, there is currently no way to do this with the repository specified by chart dependencies without manually manipulating the Chart.yaml.

@scottrigby
Copy link
Member

Note that the Podman tools that standardize registries.conf have been accepted as a CNCF Sandbox project: cncf/sandbox#309

@scottrigby
Copy link
Member

Also note this related issue: helm/helm#13615

Signed-off-by: George Jenkins <[email protected]>
@gjenkins8 gjenkins8 force-pushed the gjenkins/registries.conf_support branch from 07c67e8 to 1f6c412 Compare November 3, 2025 22:47
@gjenkins8 gjenkins8 changed the title H4HIP: Helm support registries.conf for OCI configuration/auth info Helm support registries.conf for OCI configuration/auth info Nov 3, 2025
Signed-off-by: George Jenkins <[email protected]>
@gjenkins8 gjenkins8 force-pushed the gjenkins/registries.conf_support branch from 1f6c412 to 6335ac4 Compare November 3, 2025 22:59
@gjenkins8 gjenkins8 marked this pull request as ready for review November 3, 2025 23:00
hips/hip-XXXX.md Outdated
---
hip: 9999
title: "registries.conf support for OCI registry management"
authors: [ "George Jenkins <[email protected]>", "Andy Block <[email protected]>" ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
authors: [ "George Jenkins <[email protected]>", "Andy Block <andy@redhat.com>" ]
authors: [ "George Jenkins <[email protected]>", "Andrew Block <andy.block@gmail.com>" ]


- support for repository prefixes (allowing different credentials for different prefixes)
- registry mirrors
- registry "aliasing"—features
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add:

  • allowing/denying registries

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

hips/hip-XXXX.md Outdated
- registry mirrors
- registry "aliasing"—features

Helm would like to introduce features depending on these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Helm would like to introduce features depending on these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.
Helm would like to introduce features leaning upon these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.

hips/hip-XXXX.md Outdated
- registry "aliasing"—features

Helm would like to introduce features depending on these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.
With `registries.conf` having been standardized by other container ecosystem CLI tools (podman, buildah, skopeo, etc.).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also mention that these are also CNCF hosted tools/projects

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

included / done

hips/hip-XXXX.md Outdated

`registries.conf` was picked as being a format intended for consumption beyond the Docker application container ecosystem.

ORAS (the library Helm uses for supporting OCI functionality) has planned support for `registries.conf` client-side OCI registry management.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we include reference(s) to the ORAS specific features?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a TODO, need to go grab the link

hips/hip-XXXX.md Outdated
Helm will utilize the `registries.conf` specification when determining OCI registry information (authentication credentials, etc.):
<https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md>

`registries.conf` will be preferred either when a `registries.conf` file already exists on the user's system, or when Helm supports and a user utilizes functionality that cannot be supported by Docker's configuration file in the future.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we indicate where we will search for entries?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the file lookup specifications.

TODO: is whether we want to follow the same tmpfs pattern for linux systems as the specification

hips/hip-XXXX.md Outdated
}
```

Helm will use ORAS v3 for updating (and reading) `registries.conf` (TODO: link to ORAS v3 `registries.conf` implementation).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call out the fact that ORAS v3 development has yet to begin as of this date

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't characterize it as hasn't begun. An oras-go v2 branch was cut and there are several PRs out for v3. Nothing has merged though.

hips/hip-XXXX.md Outdated

To account for existing configuration in Docker’s registry configuration, Helm will prefer `registries.conf` when resolving OCI registries (including credentials), and fall back to the existing storage mechanism if `registries.conf` does not contain an entry for the required OCI registry (including if `registries.conf` does not exist).

Helm must expect (and even encourage) users to utilize other tooling to manage `registries.conf`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or at least provide some form of guidance

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite following?

Do you mean guidance on using other tools to manage? (if so, I think this is a detail we can leave for docs)

hips/hip-XXXX.md Outdated

1. A corrupt `registries.conf` will cause an error for existing workflows
2. An invalid or incompatible with Helm `registries.conf` entry for the given OCI registry will cause the user's workflow to fail
3. Helm’s preference for `registries.conf` will break users who assume credentials are stored in Docker’s registry configuration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to include an explicit option to opt out of the feature and fall back. That way, we are covered for all scenarios

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added an HELM_EXPERIMENTAL_OCI_CONTAINERS_CONFIG feature flag.

The goal (as written into the HIP) is for this to become default true. The alternative is a permanent feature flag style config (cli flag or environment var). I think we should only introduce such iff we think (from user feedback and implementation) that introducing Container Tools config support can't be done in a non-intrusive/non-backwards compatible manner.

"auth": "Zm9vOmJhcgo="
}
}
```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to mention policy.json around here since that is used for allow/deny?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess along these lines, hopefully certs.d would be supported. That could be out of the scope for this document though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, I want to focus this HIP on initial registries.conf/auth.json support (ie. feature parity and switch over from Docker conf).

And leave extending to support future functionality to either:

  1. ORAS improvements
  2. Explicit HIPs for additional Helm behavior changes

hips/hip-XXXX.md Outdated
}
```

Helm will use ORAS v3 for updating (and reading) `registries.conf` (TODO: link to ORAS v3 `registries.conf` implementation).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't characterize it as hasn't begun. An oras-go v2 branch was cut and there are several PRs out for v3. Nothing has merged though.

Helm's fallback to Docker's registry configuration ensures the vast majority of existing user workflows remain the same.

However, there are three potential incompatibility scenarios:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is another possibility here where the user doesn't have a registries.conf but has a auth.json My understanding is the registries.conf is not required.

Copy link
Member Author

@gjenkins8 gjenkins8 Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, when I originally wrote this, I thought registries.conf was always paired with auth.json.

https://github.com/helm/community/pull/391/files#r1978339008 mentions the same

I have (significantly) reworked the language to include auth.json. Strictly, auth.json is all that is needed to replace docker/config.json. But the overall motivation is to be able to utilize features implemented by registries.conf.

Signed-off-by: George Jenkins <[email protected]>

`auth.json`:

1. Location specified in environment variable `REGISTRY_AUTH_FILE` if set
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to consider that ${XDG_RUNTIME_DIR} is tmpfs on linux by default

https://github.com/helm/community/pull/391/files#r1978342074

Signed-off-by: George Jenkins <[email protected]>
Signed-off-by: George Jenkins <[email protected]>
Signed-off-by: George Jenkins <[email protected]>
@gjenkins8 gjenkins8 changed the title Helm support registries.conf for OCI configuration/auth info Helm support Container Tools project (auth.json, registries.conf, etc) for OCI configuration/auth info Nov 12, 2025
@gjenkins8 gjenkins8 changed the title Helm support Container Tools project (auth.json, registries.conf, etc) for OCI configuration/auth info Helm support Container Tools configuration (auth.json, registries.conf, etc) for OCI registry access management Nov 15, 2025
Signed-off-by: George Jenkins <[email protected]>
Signed-off-by: George Jenkins <[email protected]>

## Abstract

The [Container Tools][containers-tools-project] project defines an alternative specification for managing client OCI registry configuration. Supporting more advanced features compared to Docker's `docker/config.json` that Helm currently uses today.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The [Container Tools][containers-tools-project] project defines an alternative specification for managing client OCI registry configuration. Supporting more advanced features compared to Docker's `docker/config.json` that Helm currently uses today.
The [Container Tools][containers-tools-project] project defines an alternative specification for managing client OCI registry configuration. Supporting more advanced features compared to Docker's `.docker/config.json` that Helm currently uses today.

nit

Pertainently including [registries.conf][registries-conf], [auth.json][auth-json], as well as the other specifications in <https://github.com/containers/container-libs/blob/main/image/docs/>.

This HIP focuses on the initial implementation using ORAS to supports Container Tools OCI registry mamagement to supersede `docker/config.json` within Helm.
Today, `auth.json` provides the equivalent functionality to `docker/config.json` for storing OCI registry credentials.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while maintianing backward compatibility since it supports .docker/config.json


- ORAS `registries.conf` support: <https://github.com/oras-project/oras-go/issues/918>
- Is the release and backwards compatibility plan good enough?
- ~~Discuss Helm's/ORAS's potential usage of `registries.conf` with `registries.conf` owners~~

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are on board with the idea, but the timeline for release I do not know.

@TerryHowe
Copy link

looks good in general, my comments are nits really.

Copy link
Contributor

@sabre1041 sabre1041 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more final changes. Looking really good


## Motivation

Helm currently uses Docker's `docker/config.json` to store client OCI registry configuration today.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Helm currently uses Docker's `docker/config.json` to store client OCI registry configuration today.
Helm currently uses Docker's `.docker/config.json` file to store client OCI registry configuration today.

## Motivation

Helm currently uses Docker's `docker/config.json` to store client OCI registry configuration today.
With `docker/config.json`'s functionality being limited to mapping a registry domain to authentication credentials only.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
With `docker/config.json`'s functionality being limited to mapping a registry domain to authentication credentials only.
The functionality of the `.docker/config.json` file is limited to mapping a registry domain to authentication credentials only.

With `docker/config.json`'s functionality being limited to mapping a registry domain to authentication credentials only.

The [CNCF-hosted](https://www.cncf.io/projects/podman-container-tools/) Container Tools project has created several specifications for managing client OCI registry configuration.
These are `registries.conf`, `auth.json`, `policy.json`, etc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
These are `registries.conf`, `auth.json`, `policy.json`, etc.
These include `registries.conf`, `auth.json`, `policy.json`, etc


The [CNCF-hosted](https://www.cncf.io/projects/podman-container-tools/) Container Tools project has created several specifications for managing client OCI registry configuration.
These are `registries.conf`, `auth.json`, `policy.json`, etc.
Which enable more advanced functionality for client OCI registry management than `docker/config.json`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Which enable more advanced functionality for client OCI registry management than `docker/config.json`.
which enable more advanced functionality for client OCI registry management than `.docker/config.json`.

<https://github.com/containers/container-libs/blob/main/image/docs/containers-auth.json.5.md>
<https://github.com/containers/container-libs/blob/main/image/docs/containers-registry.conf.json.5.md>

Helm will write to `auth.json` when performing OCI registry operations that modify client OCI registry configuration (e.g., `helm registry login`). And as long as the operation can be supported by `docker/config.json`, Helm will dual-write to `docker/config.json` for compatibility purposes. Dual-write will be removed in a future version of Helm, and Helm will write only to `auth.json`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Helm will write to `auth.json` when performing OCI registry operations that modify client OCI registry configuration (e.g., `helm registry login`). And as long as the operation can be supported by `docker/config.json`, Helm will dual-write to `docker/config.json` for compatibility purposes. Dual-write will be removed in a future version of Helm, and Helm will write only to `auth.json`.
Helm will write to `auth.json` when performing OCI registry operations that modify client OCI registry configuration (e.g., `helm registry login`). And as long as the operation can be supported by `.docker/config.json`, Helm will dual-write to `.docker/config.json` for compatibility purposes. Dual-write will be removed in a future version of Helm, and Helm will write only to `auth.json`.

1. `$HOME/.config/containers/registries.conf` if it exists
2. Otherwise `/etc/containers/registries.conf`

Helm will use ORAS v3 for updating (and reading) `registries.conf` / `auth.json`, working with the ORAS project to build support.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to note that ORAS v3 is a future milestone for that project?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Helm must expect (and even encourage) users to utilize other tooling to manage `registries.conf`, `auth.json`, etc.

## `registries.conf`/`auth.json` vs. `docker/config.json` preference
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## `registries.conf`/`auth.json` vs. `docker/config.json` preference
## `registries.conf`/`auth.json` vs. `.docker/config.json`

## `registries.conf`/`auth.json` vs. `docker/config.json` preference

To manage the release, Helm will introduce an environment variable `HELM_EXPERIMENTAL_OCI_CONTAINERS_CONFIG`.
Initially, when unset or set to `false`, Helm will continue to use only `docker/config.json`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Initially, when unset or set to `false`, Helm will continue to use only `docker/config.json`.
Initially, when unset or set to `false`, Helm will continue to use only `.docker/config.json`.


## How to teach this

Helm's documentation will need to be updated with details of Helm's `registries.conf` / `auth.json` support and fallback to `docker/config.json`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Helm's documentation will need to be updated with details of Helm's `registries.conf` / `auth.json` support and fallback to `docker/config.json`.
Helm's documentation will need to be updated with details of Helm's `registries.conf` / `auth.json` support and fallback to `.docker/config.json`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants