feat: use Nix to build OpenSSL 3.1.2

Author: Manuthor

.github/copilot-instructions.md

@@ -1,195 +1,127 @@
-# Cosmian KMS
+# Cosmian KMS — Nix-first build, test, and packaging
 
-Cosmian KMS is a high-performance, open-source FIPS 140-3 compliant Key Management System written in Rust. The repository contains the KMS server (`cosmian_kms_server`) and supporting libraries for cryptographic operations, database management, and various integrations.
+Cosmian KMS is a high-performance, open-source FIPS 140-3 compliant Key Management System written in Rust.
 
-Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
+This repository is maintained to be reproducible, deterministic, and offline-friendly. All build, test, and packaging is orchestrated via a single entrypoint:
 
-## Working Effectively
+- Build and package: `bash .github/scripts/nix.sh package [deb|rpm|dmg]`
+- Run tests: `bash .github/scripts/nix.sh test [all|sqlite|mysql|psql|redis|google_cse|hsm]`
 
-- **Bootstrap and build the repository:**
+Always reference these instructions first; only fall back to ad‑hoc commands when troubleshooting discrepancies.
 
-    - First, initialize git submodules: `git submodule update --recursive --init`
-    - System requires Rust stable toolchain (1.90.0) with rustfmt and clippy components
-    - OpenSSL 3.2.0 is REQUIRED (not 3.0.13+) for proper FIPS compliance and static linking
-    - OpenSSL must be installed to `/usr/local/openssl` using `.github/reusable_scripts/get_openssl_binaries.sh`
-    - Build process follows CI workflow: `bash .github/scripts/cargo_build.sh`
-    - Environment variables required: `OPENSSL_DIR=/usr/local/openssl`, `DEBUG_OR_RELEASE=debug|release`
-    - For non-FIPS builds: `FEATURES=non-fips`
-    - The CLI binary `cosmian` IS built in this repository and included in build artifacts
+## Why Nix?
 
-- **UI and Packaging:**
+- Pinned nixpkgs for hermetic, reproducible environments
+- Native deterministic verification in the derivation (installCheckPhase)
+- Pinned Rust toolchain (1.90.0) from Nix — no rustup downloads
+- Pinned OpenSSL 3.1.2 with static linking (no dynamic OpenSSL at runtime)
+- One-command packaging for DEB/RPM/DMG with smoke tests
 
-    - UI is built on Ubuntu distributions using `bash .github/scripts/build_ui.sh`
-    - UI files are located in `crate/server/ui` directory
-    - Release builds create Debian packages via `cargo deb -p cosmian_kms_server`
-    - RPM packages created via `cargo generate-rpm -p crate/server`
-    - Packages support both FIPS and non-FIPS variants
+## Quick start
 
-- **Testing and validation:**
+```bash
+# Build default packages for your platform (Linux → deb+rpm, macOS → dmg)
+bash .github/scripts/nix.sh package
 
-    - Multi-database testing: sqlite, postgresql, mysql, redis-findex
-    - Database environment variables: `KMS_POSTGRES_URL=postgresql://kms:[email protected]:5432/kms`, `KMS_MYSQL_URL=mysql://kms:kms@localhost:3306/kms`, `KMS_SQLITE_PATH=data/shared`
-    - MySQL tests are currently disabled (skipped in CI)
-    - Redis-findex tests skipped in FIPS mode (not supported)
-    - Debug builds only test sqlite; release builds test all enabled databases
-    - macOS runners only support sqlite tests (no docker containers)
-    - HSM testing on Ubuntu with Utimaco: `HSM_USER_PASSWORD="12345678" cargo test -p utimaco_pkcs11_loader --features utimaco`
-    - Logging control: `RUST_LOG="cosmian_kms_cli=error,cosmian_kms_server=error,cosmian_kmip=error,test_kms_server=error"`
-    - Test execution: `cargo test --workspace --lib $RELEASE $FEATURES -- --nocapture $SKIP_SERVICES_TESTS`
+# Build a specific format/variant
+bash .github/scripts/nix.sh --variant fips package deb
+bash .github/scripts/nix.sh --variant non-fips package rpm
 
-- **Build artifacts and binaries:**
+# Build and test on SQLite only
+bash .github/scripts/nix.sh test sqlite
 
-    - Primary binaries: `cosmian`, `cosmian_kms`, `cosmian_findex_server`
-    - Binary locations: `target/$DEBUG_OR_RELEASE/` (e.g., `target/debug/`)
-    - Release builds include benchmarks: `cargo bench $FEATURES --no-run`
-    - Static linking verified (no dynamic OpenSSL dependencies): `ldd cosmian_kms | grep ssl` should fail
-    - Version verification: `cosmian_kms --info` must show OpenSSL 3.2.0
-    - Binary tests: `cargo test --workspace --bins $RELEASE $FEATURES`
+# Run tests (defaults to 'all' - run 'docker compose up -d' first for DB backends)
+bash .github/scripts/nix.sh test
+```
 
-- **Run the KMS server:**
+Artifacts are placed under `result-deb-<variant>/`, `result-rpm-<variant>/`, and `result-dmg-<variant>/`.
 
-    - ALWAYS build first using the build script above
-    - Debug mode: `./target/debug/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data`
-    - Release mode: `./target/release/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data`
-    - Server listens on <http://0.0.0.0:9998> by default
-    - Supported databases: sqlite, postgresql, mysql, redis-findex (redis-findex not available in FIPS mode)
+## Determinism and native hash enforcement
 
-- **Docker usage:**
-    - Development with services: `docker compose up -d` (starts postgresql, mysql, redis)
-    - Production: `docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms:latest`
-    - Pre-built images include UI at <http://localhost:9998/ui>
-    - Local Docker builds use the same OpenSSL setup as CI
+The KMS server derivation computes the SHA-256 of the output binary and compares it to `nix/expected-hashes/{fips,non-fips}.sha256` during `installCheckPhase`. Any mismatch fails the build. Packaging scripts also enforce the expected hash when reusing prebuilt results to ensure drift is caught early.
 
-## Validation
+To update an expected hash after a legitimate change:
 
-- **CRITICAL**: Always manually test server functionality after making changes by starting the server and verifying it responds to HTTP requests
-- Test server startup: Start server with `--database-type sqlite --sqlite-path /tmp/test-db`
-- Test API responses: `curl -s -X POST -H "Content-Type: application/json" -d '{}' http://localhost:9998/kmip/2_1` should return KMIP validation error (confirms server is working)
-- Test server version: `./target/release/cosmian_kms --version` should show version 5.11.1
-- OpenSSL validation: `./target/release/cosmian_kms --info` should show OpenSSL 3.2.0
-- Static linking check: `ldd ./target/release/cosmian_kms | grep ssl` should return empty (no dynamic OpenSSL)
-- Always run `cargo fmt --check` before committing (takes 3 seconds)
-- Clippy requires installation: `rustup component add clippy`
+```bash
+nix-build -A kms-server-fips -o result-server-fips
+sha256sum result-server-fips/bin/cosmian_kms | cut -d' ' -f1 > nix/expected-hashes/fips.sha256
+# repeat for non-fips
+```
 
-## Common tasks
+## Offline packaging
 
-The following are outputs from frequently run commands. Reference them instead of viewing, searching, or running bash commands to save time.
+The first run can prewarm the Nix store (pinned nixpkgs, tools) and the Cargo registry cache. Subsequent runs can be fully offline:
 
-### Repo root structure
+- Nix: store prewarmed and reused; builds run with empty `substituters`
+- Cargo: registry and crate sources cached in `target/cargo-offline-home` and packaging runs with `CARGO_NET_OFFLINE=true`
+- OpenSSL: `resources/tarballs/openssl-3.1.2.tar.gz` must exist locally (script fetches it once if online)
 
-```text
-.cargo/                  # Cargo configuration
-.github/                 # CI/CD workflows and scripts
-  scripts/               # Build scripts (cargo_build.sh, build_ui.sh)
-  reusable_scripts/      # OpenSSL setup scripts
-crate/                   # Rust workspace crates
-  server/                # KMS server binary crate
-    ui/                  # Web UI files (built by build_ui.sh)
-  cli/                   # CLI binary crate (cosmian)
-  crypto/                # Cryptographic operations
-  kmip/                  # KMIP protocol implementation
-  client_utils/          # Client utilities
-  kms_client/            # KMS client library
-  access/                # Access control
-  interfaces/            # Database and HSM interfaces
-  server_database/       # Database management
-  hsm/                   # HSM integrations (proteccio, utimaco, softhsm2)
-documentation/           # Project documentation
-docker-compose.yml       # Development services (postgres, mysql, redis)
-Dockerfile              # Container build
-README.md               # Project documentation
-Cargo.toml              # Workspace configuration
-rust-toolchain.toml     # Rust toolchain: 1.90.0
-```
+Verification: disconnect network, then re-run `bash .github/scripts/nix.sh package deb` — it should succeed and produce the same artifact hash.
 
-### Key build commands and timing
+## Testing
 
 ```bash
-# Full CI build process (includes UI, packaging, multi-database tests)
-git submodule update --recursive --init
-export OPENSSL_DIR=/usr/local/openssl
-export DEBUG_OR_RELEASE=debug  # or release
-export FEATURES=non-fips       # optional, for non-FIPS builds
-
-# OpenSSL setup (required first)
-sudo mkdir -p /usr/local/openssl/ssl /usr/local/openssl/lib64/ossl-modules
-sudo chown -R $USER /usr/local/openssl
-bash .github/reusable_scripts/get_openssl_binaries.sh
-bash .github/scripts/cargo_build.sh
+# Typical flows
+bash .github/scripts/nix.sh test              # all tests supported on your OS
+bash .github/scripts/nix.sh test sqlite       # sqlite-only (macOS/Linux)
+bash .github/scripts/nix.sh test psql         # requires local PostgreSQL
+bash .github/scripts/nix.sh test redis        # non-FIPS only
+
+# HSM tests (Linux only)
+bash .github/scripts/nix.sh test hsm          # softhsm2 + utimaco + proteccio
+bash .github/scripts/nix.sh test hsm softhsm2 # single backend
+```
 
+Environment variables for DB tests:
 
-# UI build (Ubuntu only)
-bash .github/scripts/build_ui.sh
+- `KMS_POSTGRES_URL=postgresql://kms:[email protected]:5432/kms`
+- `KMS_MYSQL_URL=mysql://kms:kms@localhost:3306/kms`
+- `KMS_SQLITE_PATH=data/shared`
 
-# Individual builds (after OpenSSL setup)
-cargo build --features non-fips
-cargo build --release --features non-fips
+Notes:
 
-# Multi-database testing
-export KMS_TEST_DB=sqlite  # or postgresql, mysql, redis-findex
-cargo test --workspace --lib --features non-fips
+- MySQL tests are currently disabled in CI
+- Redis-findex tests are skipped in FIPS mode
+- On macOS, only sqlite tests run (no DB containers)
 
-# HSM testing (Ubuntu only)
-bash .github/reusable_scripts/test_utimaco.sh
-HSM_USER_PASSWORD="12345678" cargo test -p utimaco_pkcs11_loader --features utimaco
+## Validation and smoke tests
 
-# Packaging (release builds only)
-cargo install cargo-deb cargo-generate-rpm
-cargo deb -p cosmian_kms_server
-cargo generate-rpm -p crate/server
+Packaging runs include a smoke test that extracts the artifact and runs `cosmian_kms --info` to verify OpenSSL 3.1.2 and static linkage. You can also run the server manually (after building or unpacking a package):
 
-# Format check (3 seconds)
-cargo fmt --check
+```bash
+./cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data
 ```
 
-### Server startup and validation
+Basic API probe:
 
 ```bash
-# Start server (debug)
-./target/debug/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data
-
-# Start server (release)
-./target/release/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data
-
-# Test server is responding
 curl -s -X POST -H "Content-Type: application/json" -d '{}' http://localhost:9998/kmip/2_1
-# Expected response: "Invalid Request: missing field `tag` at line 1 column 2"
+```
 
-# Check version and OpenSSL
-./target/release/cosmian_kms --version
-# Expected: "cosmian_kms_server 5.11.1"
+Expected response is a KMIP validation error, confirming the server is alive.
 
-./target/release/cosmian_kms --info
-# Expected: Output containing "OpenSSL 3.2.0"
+## Repository layout (high level)
 
-# Verify static linking (should return empty)
-ldd ./target/release/cosmian_kms | grep ssl
+```text
+.github/                # Orchestrator scripts (nix.sh), CI, and helpers
+nix/                    # Nix derivations, scripts, expected hashes
+crate/                  # Rust workspace crates (server, cli, crypto, …)
+pkg/                    # Packaging metadata (deb/rpm service files, configs)
+resources/tarballs/     # OpenSSL 3.1.2 tarball (local copy for offline)
+result-*/               # Symlinks to build/package results
 ```
 
-### Docker quick start
+## Tips
+
+- Format/lints: run `cargo fmt --check` and clippy inside the Nix environment if needed
+- If repeated packaging triggers rebuilds, set `NO_PREWARM=1` for faster reuse-only runs
+- If a package fails due to hash mismatch, update the relevant file in `nix/expected-hashes/` only after reviewing the change
+
+## Docker
 
 ```bash
-# Pull and run pre-built image (includes UI)
 docker pull ghcr.io/cosmian/kms:latest
 docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms:latest
-
-# Development with services
-docker compose up -d
-
-# Access UI
-curl http://localhost:9998/ui
-# Expected: HTML content with KMS web interface
 ```
 
-## Important notes
-
-- **OpenSSL Version**: OpenSSL 3.2.0 is mandatory, not 3.0.13+. The build verifies this specific version.
-- **Static Linking**: All binaries must be statically linked with OpenSSL. CI verifies no dynamic OpenSSL dependencies.
-- **Build Artifacts**: Three primary binaries are built: `cosmian`, `cosmian_kms`, `cosmian_findex_server`
-- **Database Testing**: Only sqlite works in debug mode and on macOS. Full database testing requires release builds.
-- **FIPS vs non-FIPS**: Redis-findex database support is not available in FIPS mode
-- **UI Building**: UI is only built on Ubuntu distributions and requires separate build script
-- **Packaging**: Debian and RPM packages are created as part of release builds with proper FIPS/non-FIPS variants
-- **HSM Support**: Utimaco HSM testing is included but only runs on Ubuntu with specific setup
-- **MySQL**: MySQL database tests are currently disabled in CI
-- **Workspace**: Build from workspace root using cargo_build.sh script, not individual crate directories
+Images include the UI at `http://localhost:9998/ui`.

.github/reusable_scripts

@@ -9,8 +9,30 @@
 # Exit on error, print commands
 set -ex
 
-if [ -n "$FEATURES" ]; then
-  CARGO_FEATURES="--features $FEATURES"
+# Args: --variant fips|non-fips (default: fips)
+VARIANT="fips"
+while [ $# -gt 0 ]; do
+  case "$1" in
+  -v | --variant)
+    VARIANT="${2:-}"
+    shift 2 || true
+    ;;
+  *) shift ;; # ignore
+  esac
+done
+
+case "$VARIANT" in
+fips | non-fips) : ;;
+*)
+  echo "Error: --variant must be 'fips' or 'non-fips'" >&2
+  exit 1
+  ;;
+esac
+
+if [ "$VARIANT" = "non-fips" ]; then
+  CARGO_FEATURES=(--features non-fips)
+else
+  CARGO_FEATURES=()
 fi
 
 # Install nodejs from nodesource if npm is not installed
@@ -37,7 +59,7 @@ cargo install wasm-pack
 # Build WASM component
 cd crate/wasm
 # shellcheck disable=SC2086
-wasm-pack build --target web --release $CARGO_FEATURES
+wasm-pack build --target web --release "${CARGO_FEATURES[@]}"
 
 # Copy WASM artifacts to UI directory
 WASM_DIR="../../ui/src/wasm/"
@@ -56,7 +78,10 @@ npm audit
 # Deploy built UI to root
 cd .. # current path: ./
 
-DEST_DIR="crate/server/ui${CARGO_FEATURES:+_non_fips}"
+DEST_DIR="crate/server/ui"
+if [ "$VARIANT" = "non-fips" ]; then
+  DEST_DIR="crate/server/ui_non_fips"
+fi
 rm -rf "$DEST_DIR"
 mkdir -p "$DEST_DIR"
 cp -R ui/dist "$DEST_DIR"

.github/scripts/build_packages.sh

@@ -9,8 +9,8 @@
 # Exit on error, print commands
 set -ex
 
-bash ./.github/scripts/build_ui.sh
+bash ./.github/scripts/build_ui.sh --variant fips
 git add crate/server/ui
 
-FEATURES=non-fips bash ./.github/scripts/build_ui.sh
+bash ./.github/scripts/build_ui.sh --variant non-fips
 git add crate/server/ui_non_fips