(WIP) SQ:812: Setup Pypi Publishing for divbase-lib and divbase-cli

[RMCrean]

work in progress, creating PR now so can test a GH action, will add more context later.
Pointing to non-main branch so can show actual commit differences

[Copilot]

Pull request overview

This PR sets up PyPI publishing infrastructure for the divbase-lib and divbase-cli packages. It includes package metadata configuration, version management tooling, GitHub Actions workflows for automated publishing, and refactoring to move shared authentication schemas to the library package.

Changes:

• Added GitHub Actions workflows for publishing packages to PyPI and TestPyPI with validation and smoke testing
• Created helper scripts for version management and validation across packages in the monorepo
• Moved authentication schemas from divbase-api to divbase-lib to enable sharing between CLI and API packages
• Updated package metadata (README, classifiers, dependencies) and configured import restrictions via ruff

Reviewed changes

Copilot reviewed 19 out of 22 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
.github/workflows/build-pypi-packages.yaml	Reusable workflow for building and validating packages with smoke tests
.github/workflows/publish-to-pypi.yaml	Workflow to publish packages to PyPI on release
.github/workflows/publish-to-test-pypi.yaml	Workflow to publish packages to TestPyPI for testing
scripts/set_divbase_version.py	Helper script to update version numbers across all packages
scripts/ci/validate_package_versions.py	CI validation script to ensure version consistency
packages/divbase-lib/pyproject.toml	Added package metadata, classifiers, URLs, and import restrictions
packages/divbase-cli/pyproject.toml	Added package metadata, pinned divbase-lib dependency, added dual CLI entry points
packages/divbase-api/pyproject.toml	Updated dependencies and added import restrictions
packages/divbase-lib/src/divbase_lib/api_schemas/auth.py	New file containing authentication schemas moved from divbase-api
packages/divbase-cli/src/divbase_cli/user_auth.py	Updated import to use auth schemas from divbase-lib
packages/divbase-api/src/divbase_api/routes/auth.py	Updated import to use auth schemas from divbase-lib
packages/*/src/*/__init__.py	Version numbers updated to 0.1.0.dev2
ruff.toml	Added flake8-tidy-imports (TID) linting rules
pyproject.toml	Added ruff configuration and import restrictions for root
.python-version	Added Python version file specifying 3.13
.gitignore	Added dist/ directory for built packages
packages/divbase-lib/README.md	New README for the library package
packages/divbase-cli/README.md	New README for the CLI package
docs/development/deployment.md	Comprehensive documentation on publishing to PyPI
uv.lock	Updated dependencies including alembic, boto3, celery, fastapi, and others

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[RMCrean]

This PR is focused on being able to build and publish divbase-cli and divbase-lib to both PyPI (publish-to-pypi.yaml) and TestPyPI (publish-to-test-pypi.yaml) via GH actions.
As there is no need to publish divbase-api to pypi, I have not done that.

Summary

• 3 gh actions workflows added. 2 to publish and 1 that builds the packages and runs a smoke test on the packages (build-pypi-packages.yaml). The 2 actions for publishing use this one.
  We use trusted publishing to publish to pypi. This means we don't need to create/store an API key to publish the packages.
  The split between the building/validation job and the publishing job is recommended to keep the publishing workflow as short as possible (as it limits what actions jobs/steps can access the id-token: write used in the publishing jobs to authenticate with pypi.
• Updated the pyproject.tomls of all packages - bumped versions of all packages and pinned everything at current major version.
• Updated the package metadata for divbase-lib and divbase-cli's pyproject.tomls so they are more "production ready". This metadata is for example displayed on the pypi website: https://pypi.org/project/divbase-cli/
• added documentation on how to do a release of the cli in deployment.md.
• The first published version of divbase-cli did not work because there was a sneaky import from divbase-api present. We didn't see this as our local venvs have divbase-api installed. I've now added some extra linting settings to ruff to prevent this issue happening again (squiggly line in the IDE and pre-commit would fail).

Workflow testing

I've tested both workflows for publishing to pypi and testpypi and have it to a state that I think works nicely. The actions are both now set to trigger on what i think they should be triggered on. So commits to this branch no longer trigger new releases. Happy to turn back on though if you would like to experiment.

To Consider

At the moment I have defined 2 entrypoints for the divbase-cli package. One being divbase-cli (as before) and now divbase.
Perhaps for a user perspective it would be nicer to write just divbase? We should discuss and pick one I think.

Proposed versioning strategy:

For the sake of simplicity I would like to propose that all 3 packages (divbase-lib/cli/api) should have the same version number at all times. So even if at one point we only make a change to the code in the cli or api, we bump all 3 versions for a new release. Happy to discuss, but especially during the early stages of development I think it might just make life a little easier to do it like this.

@brinkdp, happy to give you a walk through on this PR (and docs one if wanted) first :)