Skip to content

[ENH] Update release process #5807

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

[ENH] Update release process #5807

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
6b1e515
checkpoint
itaismith Nov 3, 2025
dd5517f
Python update
itaismith Nov 4, 2025
74b2b30
[ENH] Improve release process
itaismith Nov 4, 2025
6b1f1e0
JS update
itaismith Nov 4, 2025
67b6d59
checkpoint
itaismith Nov 5, 2025
d311dc1
CLI instructions
itaismith Nov 5, 2025
68ab630
checkpoint
itaismith Nov 5, 2025
79da25e
Add CI for releasing CLI and clients
itaismith Nov 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 13 additions & 18 deletions .github/workflows/release-cli.yml → .github/workflows/_build_release_cli.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,15 @@ name: Release CLI
on:
workflow_dispatch:
inputs:
release_name:
version:
description: "Release name to use (e.g. cli-1.2.3) when dispatching manually"
required: false
push:
tags:
- 'cli_release_[0-9]*.[0-9]*.[0-9]*'
workflow_call:
inputs:
version:
description: "Version to publish"
required: true
type: string

jobs:
build-linux:
Expand Down Expand Up @@ -139,20 +142,12 @@ jobs:
- name: Determine release info
id: release_info
run: |
if [ "${GITHUB_EVENT_NAME}" = "push" ]; then
# The tag is available as refs/tags/cli_release_a.b.c.
TAG=${GITHUB_REF#refs/tags/}
VERSION=${TAG#cli_release_}
echo "release_name=cli-${VERSION}" >> $GITHUB_OUTPUT
echo "tag_name=${TAG}" >> $GITHUB_OUTPUT
else
if [ -z "${{ github.event.inputs.release_name }}" ]; then
echo "::error::Manual dispatch requires a release_name input."
exit 1
fi
echo "release_name=${{ github.event.inputs.release_name }}" >> $GITHUB_OUTPUT
echo "tag_name=${{ github.event.inputs.release_name }}" >> $GITHUB_OUTPUT
if [ -z "${{ github.event.inputs.release_name }}" ]; then
echo "::error::Manual dispatch requires a release_name input."
exit 1
fi
echo "release_name=${{ github.event.inputs.version }}" >> $GITHUB_OUTPUT
echo "tag_name=${{ github.event.inputs.version }}" >> $GITHUB_OUTPUT

- name: Create GitHub Release
id: create_release
Expand All @@ -161,7 +156,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ steps.release_info.outputs.tag_name }}
release_name: ${{ steps.release_info.outputs.release_name }}
release_name: ${{ steps.release_info.outputs.version }}
body: "CLI release."
draft: false
prerelease: false
Expand Down
323 changes: 323 additions & 0 deletions .github/workflows/release-chromadb-all.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,323 @@
name: 📦 Release CLI, Python, and JS clients

on:
push:
tags:
- "cli_*_python_*_js_*"

jobs:
check-tag:
runs-on: blacksmith-4vcpu-ubuntu-2404
outputs:
tag_matches: ${{ steps.check-tag.outputs.tag_matches }}
cli_version: ${{ steps.check-tag.outputs.cli_version }}
python_version: ${{ steps.check-tag.outputs.python_version }}
js_version: ${{ steps.check-tag.outputs.js_version }}
steps:
- name: Check Tag
id: check-tag
run: |
if [[ ${{ github.event.ref }} =~ ^refs/tags/cli_[0-9]+\.[0-9]+\.[0-9]+_python_[0-9]+\.[0-9]+\.[0-9]+_js_[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "tag_matches=true" >> $GITHUB_OUTPUT
# Extract versions from cli_X.Y.Z_python_A.B.C_js_D.E.F format
TAG=$(echo "${{ github.event.ref }}" | sed 's/refs\/tags\///')
CLI_VERSION=$(echo "$TAG" | sed -E 's/^cli_([0-9]+\.[0-9]+\.[0-9]+)_python_[0-9]+\.[0-9]+\.[0-9]+_js_[0-9]+\.[0-9]+\.[0-9]+$/\1/')
PYTHON_VERSION=$(echo "$TAG" | sed -E 's/^cli_[0-9]+\.[0-9]+\.[0-9]+_python_([0-9]+\.[0-9]+\.[0-9]+)_js_[0-9]+\.[0-9]+\.[0-9]+$/\1/')
JS_VERSION=$(echo "$TAG" | sed -E 's/^cli_[0-9]+\.[0-9]+\.[0-9]+_python_[0-9]+\.[0-9]+\.[0-9]+_js_([0-9]+\.[0-9]+\.[0-9]+)$/\1/')
echo "cli_version=$CLI_VERSION" >> $GITHUB_OUTPUT
echo "python_version=$PYTHON_VERSION" >> $GITHUB_OUTPUT
echo "js_version=$JS_VERSION" >> $GITHUB_OUTPUT
else
echo "Tag does not match the release tag pattern (cli_X.Y.Z_python_A.B.C_js_D.E.F), exiting workflow"
echo "tag_matches=false" >> $GITHUB_OUTPUT
Comment on lines +31 to +32
Copy link
Contributor

@propel-code-bot propel-code-bot bot Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BestPractice]

If the tag doesn't match the expected format, this step prints a message but the job succeeds. This allows subsequent jobs to run with empty version variables, which will likely cause them to fail with confusing errors. It's better to fail this job immediately to make it clear why the workflow stopped.

Context for Agents 
[**BestPractice**]

If the tag doesn't match the expected format, this step prints a message but the job succeeds. This allows subsequent jobs to run with empty version variables, which will likely cause them to fail with confusing errors. It's better to fail this job immediately to make it clear why the workflow stopped.

File: .github/workflows/release-chromadb-all.yml
Line: 32

fi
get-python-version:
runs-on: blacksmith-4vcpu-ubuntu-2404
needs: [check-tag]
outputs:
version: ${{ steps.version.outputs.version }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.9'
- name: Install setuptools_scm
run: python -m pip install setuptools_scm
- name: Get Release Version
id: version
run: echo "version=$(python -m setuptools_scm)" >> $GITHUB_OUTPUT
- name: Validate Python version matches tag
if: ${{ needs.check-tag.outputs.tag_matches == 'true' }}
run: |
SETUPTOOLS_VERSION="${{ steps.version.outputs.version }}"
TAG_VERSION="${{ needs.check-tag.outputs.python_version }}"
if [ "$SETUPTOOLS_VERSION" != "$TAG_VERSION" ]; then
echo "ERROR: setuptools_scm version ($SETUPTOOLS_VERSION) does not match tag python version ($TAG_VERSION)"
exit 1
else
echo "✓ setuptools_scm version matches tag python version: $TAG_VERSION"
fi
python-tests-linux:
uses: ./.github/workflows/_python-tests.yml
secrets: inherit
with:
python_versions: '["3.9", "3.10", "3.11", "3.12"]'
property_testing_preset: 'normal'

python-tests-windows:
uses: ./.github/workflows/_python-tests.yml
secrets: inherit
with:
# we only run windows tests on 3.12 because windows runners are expensive
# and we usually don't see failures that are isolated to a specific version
python_versions: '["3.12"]'
property_testing_preset: 'normal'
runner: '8core-32gb-windows-latest'

javascript-client-tests:
name: JavaScript client tests
uses: ./.github/workflows/_javascript-client-tests.yml

rust-tests:
name: Rust tests
uses: ./.github/workflows/_rust-tests.yml
secrets: inherit

go-tests:
name: Go tests
uses: ./.github/workflows/_go-tests.yml
secrets: inherit

release-cli:
name: Release CLI
needs:
- check-tag
- get-python-version
- python-tests-linux
- python-tests-windows
- javascript-client-tests
- rust-tests
- go-tests
uses: ./.github/workflows/_build_release_cli.yml
secrets: inherit
with:
version: cli-${{ needs.check-tag.outputs.cli_version }}

release-js-bindings:
name: Release JS Bindings
needs:
- check-tag
- get-python-version
- python-tests-linux
- python-tests-windows
- javascript-client-tests
- rust-tests
- go-tests
uses: ./.github/workflows/_build_js_bindings.yml
secrets: inherit

release-js-client:
name: Release JS Client
needs:
- check-tag
- release-js-bindings
uses: ./.github/workflows/release-javascript-client.yml
secrets: inherit
with:
tag: js_release_${{ needs.check-tag.outputs.js_version }}

release-docker:
name: Publish to DockerHub and GHCR
needs:
- check-tag
- get-python-version
- python-tests-linux
- python-tests-windows
- javascript-client-tests
- rust-tests
- go-tests
uses: ./.github/workflows/_build_release_container.yml
secrets: inherit
with:
tag: ${{ needs.get-python-version.outputs.version }}
tag_as_latest: ${{ needs.check-tag.outputs.tag_matches == 'true' }}
push: true

release-pypi:
name: Publish to PyPI
needs:
- check-tag
- get-python-version
- python-tests-linux
- python-tests-windows
- javascript-client-tests
- rust-tests
- go-tests
uses: ./.github/workflows/_build_release_pypi.yml
secrets: inherit
with:
publish_to_test_pypi: true
publish_to_pypi: ${{ needs.check-tag.outputs.tag_matches == 'true' }}
version: ${{ needs.get-python-version.outputs.version }}

release-thin-pypi:
name: Publish thin client to PyPI
runs-on: blacksmith-4vcpu-ubuntu-2404
needs:
- check-tag
- python-tests-linux
- python-tests-windows
- javascript-client-tests
- rust-tests
- go-tests
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: ./.github/actions/python
with:
python-version: '3.12'
- name: Build Client
run: ./clients/python/build_python_thin_client.sh
- name: Test Client Package
run: bin/test-package/test-thin-client-package.sh dist/*.tar.gz
- name: Install setuptools_scm
run: python -m pip install setuptools_scm
- name: Publish to Test PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.TEST_PYPI_PYTHON_CLIENT_PUBLISH_KEY }}
repository-url: https://test.pypi.org/legacy/
verbose: 'true'
- name: Publish to PyPI
if: ${{ needs.check-tag.outputs.tag_matches == 'true' }}
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_PYTHON_CLIENT_PUBLISH_KEY }}
verbose: 'true'

release-github:
name: Make GitHub release
runs-on: blacksmith-4vcpu-ubuntu-2404
needs:
- check-tag
- get-python-version
- release-docker
- release-pypi
- release-thin-pypi
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Download artifact
uses: actions/download-artifact@v4
with:
pattern: wheels-*
path: dist
- name: Get current date
id: builddate
run: echo "builddate=$(date +'%Y-%m-%dT%H:%M')" >> $GITHUB_OUTPUT
- name: Release Tagged Version
uses: ncipollo/[email protected]
if: ${{ needs.check-tag.outputs.tag_matches == 'true' }}
with:
body: |
Version: `${{needs.get-python-version.outputs.version}}`
Git ref: `${{github.ref}}`
Build Date: `${{steps.builddate.outputs.builddate}}`
PIP Package: `chroma-${{needs.get-python-version.outputs.version}}.tar.gz`
Github Container Registry Image: `${{ env.GHCR_IMAGE_NAME }}:${{ needs.get-python-version.outputs.version }}`
DockerHub Image: `${{ env.DOCKERHUB_IMAGE_NAME }}:${{ needs.get-python-version.outputs.version }}`
artifacts: "dist/*"
prerelease: false
makeLatest: true
generateReleaseNotes: true
- name: Update Tag
uses: richardsimko/[email protected]
if: ${{ needs.check-tag.outputs.tag_matches != 'true' }}
with:
tag_name: latest
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Release Latest
uses: ncipollo/[email protected]
if: ${{ needs.check-tag.outputs.tag_matches != 'true' }}
with:
tag: "latest"
name: "Latest"
body: |
Version: `${{needs.get-python-version.outputs.version}}`
Git ref: `${{github.ref}}`
Build Date: `${{steps.builddate.outputs.builddate}}`
PIP Package: `chroma-${{needs.get-python-version.outputs.version}}.tar.gz`
Github Container Registry Image: `${{ env.GHCR_IMAGE_NAME }}:${{ needs.get-python-version.outputs.version }}`
DockerHub Image: `${{ env.DOCKERHUB_IMAGE_NAME }}:${{ needs.get-python-version.outputs.version }}`
artifacts: "dist/*"
allowUpdates: true
removeArtifacts: true
prerelease: true
Comment on lines +244 to +267
Copy link
Contributor

@propel-code-bot propel-code-bot bot Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BestPractice]

The Update Tag and Release Latest steps seem designed to create a latest pre-release. However, this workflow is specifically triggered by version tags (e.g., cli_A.B.C_python_D.E.F_js_X.Y.Z). The logic to handle non-matching tags by creating a latest release seems out of place for a versioned release workflow and could lead to unexpected behavior if a malformed tag is pushed. Consider removing these steps to ensure this workflow only handles explicit versioned releases.

Context for Agents 
[**BestPractice**]

The `Update Tag` and `Release Latest` steps seem designed to create a `latest` pre-release. However, this workflow is specifically triggered by version tags (e.g., `cli_A.B.C_python_D.E.F_js_X.Y.Z`). The logic to handle non-matching tags by creating a `latest` release seems out of place for a versioned release workflow and could lead to unexpected behavior if a malformed tag is pushed. Consider removing these steps to ensure this workflow only handles explicit versioned releases.

File: .github/workflows/release-chromadb-all.yml
Line: 267


deploy-staging:
name: Deploy to staging
# depends on release-github because it updates the tag to latest, which is what will get deployed
needs:
- release-github
uses: ./.github/workflows/_deploy.yml
secrets: inherit

release-docs:
name: Deploy docs to Vercel
runs-on: blacksmith-4vcpu-ubuntu-2404
needs:
- check-tag
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: "18.x"
registry-url: "https://registry.npmjs.org"
- name: Install vercel
run: npm install -g vercel
- name: Deploy
run: vercel deploy --token ${{ secrets.VERCEL_TOKEN }} ${{ needs.check-tag.outputs.tag_matches == 'true' && '--prod' || '' }}
env:
VERCEL_ORG_ID: ${{ secrets.VERCEL_ORG_ID }}
VERCEL_PROJECT_ID: ${{ secrets.VERCEL_DOCS_PROJECT_ID }}

notify-slack-on-failure:
name: Notify Slack on ChromaDB Release Failure
if: failure()
needs:
- release-docker
- release-pypi
- release-thin-pypi
- release-github
- deploy-staging
- release-docs
runs-on: blacksmith-2vcpu-ubuntu-2404
steps:
- name: Notify Slack
uses: slackapi/[email protected]
with:
token: ${{ secrets.SLACK_BOT_TOKEN }}
method: chat.postMessage
payload: |
channel: ${{ secrets.SLACK_CHANNEL_ID }}
text: |
:x: *ChromaDB release failure!*
*Workflow:* ${{ github.workflow }}
*Run:* <https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}|View run>
*Ref:* <https://github.com/${{ github.repository }}/tree/${{ github.ref_name }}|${{ github.ref_name }}>
*Author:* ${{ github.actor }}
Loading
Loading