starters: generate and publish local + aws Nebi workspaces

[viniciusdc]

Part of #560.

What

Adds the machinery to build and publish Nebi starter workspaces, scoped to the local and aws providers.

Starters are distributed as OCI bundles, so the rendered workspaces are deliberately not committed here. Only their source lives in the tree:

• scripts/gen-starters.sh renders examples/<provider>-config.yaml into a workspace, replacing the identity-bearing values with the CHANGEME sentinel and filling in pixi.toml. It uses line edits rather than a YAML round trip, so the examples' inline comments survive intact.
• starters/templates/ holds the pixi.toml template and the per-provider READMEs, which are copied verbatim.
• Output lands in dist/, which is gitignored.

examples/ stays the single source of truth for config content, so there is no second copy to drift.

Per-provider differences

local placeholders only project_name: kind runs everything locally, the certificate is self-signed and the GitOps repo is created automatically. It gets no opentofu dependency, since that provider drives kind through a Go library.

aws placeholders project_name, domain, the ACME email and the GitOps repository URL and path, and pins opentofu at >=1.11.3,<2. Pinning it is the point of a pinned toolchain: without it nic falls back to downloading an unpinned OpenTofu at deploy time. The floor has to clear pkg/tofu.MinVersion (1.11.3) - below that nic rejects the PATH binary and downloads one anyway, silently, which defeats the pin. Everything else (region, availability zones, instance types, Longhorn, EFS) stays a working default.

CI

validate-starters generates both starters and checks that no template token went unsubstituted, that every pixi.toml parses, and that an unedited starter is rejected by the placeholder gate naming exactly the expected fields (see Validation below).

publish-starters is tag-driven, and deliberately not dispatchable. A published v<x.y.z> bundle has to keep meaning the same bytes; a main push resolves to the previous tag, and a workflow_dispatch could target any ref - from a branch it would publish starter-*:vmain, from an existing tag it would overwrite a released bundle. Deployment-branch rules live in repo settings and can't be reviewed from the workflow file, so the guard is the if. A manual re-publish is a re-run of the tag's own run. The job runs under an environment gate with a concurrency group that is never cancelled mid-push, resolves pixi lock, publishes each workspace with nebi publish --local, then re-imports them and re-validates to confirm the placeholders survived the round trip.

Starters publish under the existing nebari project with a starters/ subdir, mirroring how the charts live at quay.io/nebari/charts.

Scope

Only local and aws. The other providers keep their examples/ entries and can be added to the generator's provider list when we want them; the generator fails loudly if a provider is added without declaring its placeholder fields.

The $schema modeline (#562) is not included; it needs the generated schema published first, and that work is itself still in flight (#507 was closed unmerged, superseded by #600).

Not covered here, and each a natural follow-up rather than a gap this PR hides: the end-to-end nebi import -> pixi install -> nebi run deploy kind test in CI, and the docs-site page for the import-edit-deploy flow. Both are #560 DoD items. #560 also still reads "one per provider" across all six - the local+aws cut wants ratifying on the issue rather than living implicitly in this PR.

nebi >= 0.10 is required to publish or import these, since asset-layer bundling is what carries config.yaml and the README alongside pixi.toml and pixi.lock. Older versions silently ship only the two lock files.

This does not conflict with ADR-0005's "docgen is the authoritative config-reference generator". That rule scopes to reference material read from the Go config structs; this reads examples/*.yaml as text and never touches Go types, so no second struct-reader is introduced.

Dependencies

The CHANGEME rejection has landed (#583, closing #561), so validate-starters is green: both starters are rejected unedited, the rejection comes from the placeholder gate, and it names exactly the expected field paths.

Publishing still needs #579. pixi lock cannot resolve nebari-infrastructure-core from the prefix.dev channel until the package is published there, so publish is blocked rather than merely degraded - and because the publish job is tag-only it has never executed. The first v* tag is where that path gets exercised for real.

Validation

The starter is asserted through nic rather than through the generator's own field list, which is what keeps the check honest:

• the unedited starter must be rejected, and the error must come from the placeholder gate (placeholder value, emitted only after a successful unmarshal - so this doubles as proof the line-edit surgery left loadable YAML)
• the rejection must name exactly the expected field paths, which nic derives from the parsed config

That second assertion is deliberately a hand-written list in CI. An earlier revision filled the placeholders back in using a sed script the generator emitted from the same list it used to insert them; that could not catch a prefix matching the wrong line, and the "filled config validates" half mostly re-proved pkg/nic/examples_test.go.

The generator requires each declared prefix to match exactly one line - zero means a key was renamed and a real value would ship untouched, more than one means a same-named key appeared at the same indent and blanking both would bury a real value under a plausible-looking placeholder. Failures accumulate and report together.

Placeholder edits keep the trailing comment on the line they replace, so path: CHANGEME # Optional subdirectory within the repository reaches the reader with its hint intact.

[tool.nebi.bundle] include is a strict allowlist (config.yaml, README.md alongside the always-bundled pixi.toml/pixi.lock), which is a stronger answer than the exclude list #560 asked for: a filled config, a kubeconfig or tfstate a user creates later cannot enter a republished bundle by default.

Test plan

• ./scripts/gen-starters.sh dist/starters renders both providers; the aws config keeps its inline comments
• no leftover __TOKEN__ in the rendered output; both pixi.toml files parse
• both starters are rejected unedited, the rejection comes from the placeholder gate, and it names exactly the expected fields (project_name for local; certificate.acme.email, domain, project_name, repository.existing.path, repository.existing.url for aws)
• injecting a duplicate path: key into examples/aws-config.yaml makes the generator fail with matched 2 lines ... want exactly 1 and exit 1
• make starters renders both providers; the aws path: placeholder keeps its inline comment
• generated output is gitignored
• actionlint clean; ./scripts/check-action-pins.sh passes

[marcelovilla]

Thanks for this PR @viniciusdc!

I left some inline comments and I'm leaving a general one here:

I think we could benefit from a small go cmd—similar to what you implemented in cmd/docgen and the placeholder validation—to render the starter config files and the companion pixi.toml file. Using a bash script for this feels fragile. For example, the yaml keys are hardcoded, including their indentation:

placeholder_fields() {
  case "$1" in
    local)
      printf '%s\n' \
        'project_name: '
      ;;
    aws)
      printf '%s\n' \
        'project_name: ' \
        'domain: ' \
        '    email: ' \
        '    url: ' \
        '    path: '
      ;;
  esac
}

While the CI assertions you wrote would catch this, it would only do so in a CI environment. Having this as a small go cmd would make it straightforward to add unit tests to catch this before.

Initially I thought one argument against doing this in go would be one of your comments on the bash script:

# Applied as line edits rather than a YAML round trip, so the inline comments
# that make the examples useful survive into the starter - including the comment
# on a replaced line, which is preserved and reattached (those five lines are
# exactly the ones whose hint the reader needs most).

However, I tested it against examples/aws-config.yaml using goccy/go-yaml, and both comments and blank lines are preserved. when resolving each field as a path (using yaml.PathString), reading the token line and column, and then injecting a placeholder there.

Not necessarily a blocker and could be a follow-up issue if there's any urgency to get this merged soon. Just wanted to raise it here because I think it makes for a more robust approach.

[marcelovilla]

Should this version be static? I'm not super familiar with Pixi workspaces so not sure if having the NIC release here would buy us anything. I know it's already specified below in nebari-infrastructure-core and that nebi diff will still render other differences in the TOML file.

[viniciusdc]

Static is deliberate, and it matches the snippet in #560 (which also has version = "0.1.0"). This is the workspace version - the starter as a thing - not the tool it pins; nic version lives in [dependencies] below and the bundle tag carries it too. Setting it to the nic release would give us three copies of the same string with no consumer reading this one. Leaving as is unless you feel strongly.

[viniciusdc]

Thanks @marcelovilla - agreed on the Go cmd, and thanks for actually testing the comment-preservation bit rather than taking the script comment at face value. That was the one argument I had for staying in shell and you removed it.

Two things that make it more clear-cut than it looks:

goccy/go-yaml is already the parser on both sides of this. #603 replaced yaml.v3 with it, and the placeholder gate from #583 walks its AST - so resolving each field with yaml.PathString is the same library the validation half already uses, not a new dependency.

And it deletes machinery rather than porting it. Three things in the current script only exist because it is doing text surgery: the hardcoded prefixes-with-indentation you quoted, the "each prefix must match exactly one line" guard (with a path, it either resolves or it does not - there is no wrong-line case to defend against), and the sed dance that reattaches a trailing comment to a replaced line. All three go away.

On sequencing: I would rather do it as its own PR than grow this one, and it is not urgent for merge since publishing is blocked on #579 anyway. But I do want it in before the first real tag - nothing is published yet, so there is no bundle compatibility to preserve, and that is the cheapest this change will ever be. It also folds into the ADR that is still owed here, which can then just record that generation is Go and consistent with docgen instead of having to justify a shell exception.

Starting on it now. The four inline comments are answered, and two of them are fixed in aefc233.

[viniciusdc]

Opened #615 with the Go generator, stacked on this branch so the diff there is just the swap.

Output is byte-identical - I diffed both generators' full trees before deleting the script - so this is a fragility change, not a behaviour one. The three things that go away are the ones that only existed to do text surgery: the exactly-one-match guard, the sed that reattached a trailing comment, and the hardcoded indentation. Comments and blank lines still survive, via the token line/column approach you described.

One thing worth reporting back: block scalars need rejecting explicitly. Their token is the |/> indicator rather than the body, so checking the token text for a newline never fires, and an in-place edit would leave the following lines orphaned. Cost me a red test before I worked out why.

[marcelovilla]

Thanks for addressing all the comments @viniciusdc! Approving now