Nix: Reproducible Package Management

.github/copilot-instructions.md

@@ -1,196 +1,99 @@
-# Cosmian KMS
+# Cosmian KMS — Build and Test Guide
 
-Cosmian KMS is a high-performance, open-source FIPS 140-3 compliant Key Management System written in Rust. The repository contains the KMS server (`cosmian_kms_server`) and supporting libraries for cryptographic operations, database management, and various integrations.
+Cosmian KMS is a high-performance, open-source FIPS 140-3 compliant Key Management System written in Rust.
 
-Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
+## Quick start
 
-## Working Effectively
-
-- **Bootstrap and build the repository:**
-
-    - First, initialize git submodules: `git submodule update --recursive --init`
-    - System requires Rust stable toolchain (1.90.0) with rustfmt and clippy components
-    - OpenSSL 3.2.0 is REQUIRED (not 3.0.13+) for proper FIPS compliance and static linking
-    - OpenSSL must be installed to `/usr/local/openssl` using `.github/reusable_scripts/get_openssl_binaries.sh`
-    - Build process follows CI workflow: `bash .github/scripts/cargo_build.sh`
-    - Environment variables required: `OPENSSL_DIR=/usr/local/openssl`, `DEBUG_OR_RELEASE=debug|release`
-    - For non-FIPS builds: `FEATURES=non-fips`
-    - The CLI binary `cosmian` IS built in this repository and included in build artifacts
+```bash
+# Build the project
+cargo build --release
 
-- **UI and Packaging:**
+# Build with specific features
+cargo build --release --features fips
 
-    - UI is built on Ubuntu distributions using `bash .github/scripts/build_ui.sh`
-    - UI files are located in `crate/server/ui` directory
-    - Release builds create Debian packages via `cargo deb -p cosmian_kms_server`
-    - RPM packages created via `cargo generate-rpm -p crate/server`
-    - Packages support both FIPS and non-FIPS variants
+# Run tests
+cargo test
 
-- **Testing and validation:**
+# Run tests with specific features
+cargo test --features fips
+```
 
-    - Multi-database testing: sqlite, postgresql, mysql, redis-findex
-    - Database environment variables: `KMS_POSTGRES_URL=postgresql://kms:[email protected]:5432/kms`, `KMS_MYSQL_URL=mysql://kms:kms@localhost:3306/kms`, `KMS_SQLITE_PATH=data/shared`
-    - MySQL tests are currently disabled (skipped in CI)
-    - Redis-findex tests skipped in FIPS mode (not supported)
-    - Debug builds only test sqlite; release builds test all enabled databases
-    - macOS runners only support sqlite tests (no docker containers)
-    - HSM testing on Ubuntu with Utimaco: `HSM_USER_PASSWORD="12345678" cargo test -p utimaco_pkcs11_loader --features utimaco`
-    - Logging control: `RUST_LOG="cosmian_kms_cli=error,cosmian_kms_server=error,cosmian_kmip=error,test_kms_server=error"`
-    - Test execution: `cargo test --workspace --lib $RELEASE $FEATURES -- --nocapture $SKIP_SERVICES_TESTS`
+## Testing
 
-- **Build artifacts and binaries:**
+```bash
+# Run all tests
+cargo test
 
-    - Primary binaries: `cosmian`, `cosmian_kms`, `cosmian_findex_server`
-    - Binary locations: `target/$DEBUG_OR_RELEASE/` (e.g., `target/debug/`)
-    - Release builds include benchmarks: `cargo bench $FEATURES --no-run`
-    - Static linking verified (no dynamic OpenSSL dependencies): `ldd cosmian_kms | grep ssl` should fail
-    - Version verification: `cosmian_kms --info` must show OpenSSL 3.2.0
-    - Binary tests: `cargo test --workspace --bins $RELEASE $FEATURES`
+# Run tests with specific features
+cargo test --features fips
 
-- **Run the KMS server:**
+# Run tests for a specific package
+cargo test -p cosmian_kms_server
+cargo test -p cosmian_kms_cli
 
-    - ALWAYS build first using the build script above
-    - Debug mode: `./target/debug/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data`
-    - Release mode: `./target/release/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data`
-    - Server listens on <http://0.0.0.0:9998> by default
-    - Supported databases: sqlite, postgresql, mysql, redis-findex (redis-findex not available in FIPS mode)
+# Run specific test suites
+cargo test sqlite       # SQLite tests
+cargo test postgres     # PostgreSQL tests (requires local PostgreSQL)
+cargo test redis        # Redis tests
+```
 
-- **Docker usage:**
-    - Development with services: `docker compose up -d` (starts postgresql, mysql, redis)
-    - Production: `docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms:latest`
-    - Pre-built images include UI at <http://localhost:9998/ui>
-    - Local Docker builds use the same OpenSSL setup as CI
+Environment variables for DB tests:
 
-## Validation
+- `KMS_POSTGRES_URL=postgresql://kms:[email protected]:5432/kms`
+- `KMS_MYSQL_URL=mysql://kms:kms@localhost:3306/kms`
+- `KMS_SQLITE_PATH=data/shared`
 
-- **CRITICAL**: Always manually test server functionality after making changes by starting the server and verifying it responds to HTTP requests
-- Test server startup: Start server with `--database-type sqlite --sqlite-path /tmp/test-db`
-- Test API responses: `curl -s -X POST -H "Content-Type: application/json" -d '{}' http://localhost:9998/kmip/2_1` should return KMIP validation error (confirms server is working)
-- Test server version: `./target/release/cosmian_kms --version` should show version 5.12.0
-- OpenSSL validation: `./target/release/cosmian_kms --info` should show OpenSSL 3.2.0
-- Static linking check: `ldd ./target/release/cosmian_kms | grep ssl` should return empty (no dynamic OpenSSL)
-- Always run `cargo fmt --check` before committing (takes 3 seconds)
-- Clippy requires installation: `rustup component add clippy`
+Notes:
 
-## Common tasks
+- MySQL tests are currently disabled in CI
+- Redis-findex tests are skipped in FIPS mode
+- Start database backends with `docker compose up -d` before running DB tests
 
-The following are outputs from frequently run commands. Reference them instead of viewing, searching, or running bash commands to save time.
+## Running the server
 
-### Repo root structure
+After building, you can run the server manually:
 
-```text
-.cargo/                  # Cargo configuration
-.github/                 # CI/CD workflows and scripts
-  scripts/               # Build scripts (cargo_build.sh, build_ui.sh)
-  reusable_scripts/      # OpenSSL setup scripts
-crate/                   # Rust workspace crates
-  server/                # KMS server binary crate
-    ui/                  # Web UI files (built by build_ui.sh)
-  cli/                   # CLI binary crate (cosmian)
-  crypto/                # Cryptographic operations
-  kmip/                  # KMIP protocol implementation
-  client_utils/          # Client utilities
-  kms_client/            # KMS client library
-  access/                # Access control
-  interfaces/            # Database and HSM interfaces
-  server_database/       # Database management
-  hsm/                   # HSM integrations (proteccio, utimaco, softhsm2)
-documentation/           # Project documentation
-docker-compose.yml       # Development services (postgres, mysql, redis)
-Dockerfile              # Container build
-README.md               # Project documentation
-Cargo.toml              # Workspace configuration
-rust-toolchain.toml     # Rust toolchain: 1.90.0
+```bash
+cargo run --release --bin cosmian_kms -- --database-type sqlite --sqlite-path /tmp/kms-data
 ```
 
-### Key build commands and timing
+Or run the compiled binary directly:
 
 ```bash
-# Full CI build process (includes UI, packaging, multi-database tests)
-git submodule update --recursive --init
-export OPENSSL_DIR=/usr/local/openssl
-export DEBUG_OR_RELEASE=debug  # or release
-export FEATURES=non-fips       # optional, for non-FIPS builds
-
-# OpenSSL setup (required first)
-sudo mkdir -p /usr/local/openssl/ssl /usr/local/openssl/lib64/ossl-modules
-sudo chown -R $USER /usr/local/openssl
-bash .github/reusable_scripts/get_openssl_binaries.sh
-bash .github/scripts/cargo_build.sh
-
-
-# UI build (Ubuntu only)
-bash .github/scripts/build_ui.sh
-
-# Individual builds (after OpenSSL setup)
-cargo build --features non-fips
-cargo build --release --features non-fips
-
-# Multi-database testing
-export KMS_TEST_DB=sqlite  # or postgresql, mysql, redis-findex
-cargo test --workspace --lib --features non-fips
-
-# HSM testing (Ubuntu only)
-bash .github/reusable_scripts/test_utimaco.sh
-HSM_USER_PASSWORD="12345678" cargo test -p utimaco_pkcs11_loader --features utimaco
-
-# Packaging (release builds only)
-cargo install cargo-deb cargo-generate-rpm
-cargo deb -p cosmian_kms_server
-cargo generate-rpm -p crate/server
-
-# Format check (3 seconds)
-cargo fmt --check
+./target/release/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data
 ```
 
-### Server startup and validation
+Basic API probe:
 
 ```bash
-# Start server (debug)
-./target/debug/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data
-
-# Start server (release)
-./target/release/cosmian_kms --database-type sqlite --sqlite-path /tmp/kms-data
-
-# Test server is responding
 curl -s -X POST -H "Content-Type: application/json" -d '{}' http://localhost:9998/kmip/2_1
-# Expected response: "Invalid Request: missing field `tag` at line 1 column 2"
+```
 
-# Check version and OpenSSL
-./target/release/cosmian_kms --version
-# Expected: "cosmian_kms_server 5.12.0"
+Expected response is a KMIP validation error, confirming the server is alive.
 
-./target/release/cosmian_kms --info
-# Expected: Output containing "OpenSSL 3.2.0"
+## Repository layout (high level)
 
-# Verify static linking (should return empty)
-ldd ./target/release/cosmian_kms | grep ssl
+```text
+.github/                # CI workflows and scripts
+crate/                  # Rust workspace crates (server, cli, crypto, …)
+pkg/                    # Packaging metadata (deb/rpm service files, configs)
+documentation/          # Documentation and guides
+resources/              # Configuration files and resources
+test_data/              # Test fixtures and data
+ui/                     # Web UI source
 ```
 
-### Docker quick start
+## Tips
+
+- Format/lints: run `cargo fmt --check` and `cargo clippy` to check code style
+- Use `cargo build --release` for optimized builds
+- Run `cargo test` frequently to ensure changes don't break functionality
+
+## Docker
 
 ```bash
-# Pull and run pre-built image (includes UI)
 docker pull ghcr.io/cosmian/kms:latest
 docker run -p 9998:9998 --name kms ghcr.io/cosmian/kms:latest
-
-# Development with services
-docker compose up -d
-
-# Access UI
-curl http://localhost:9998/ui
-# Expected: HTML content with KMS web interface
 ```
 
-## Important notes
-
-- **OpenSSL Version**: OpenSSL 3.2.0 is mandatory, not 3.0.13+. The build verifies this specific version.
-- **Static Linking**: All binaries must be statically linked with OpenSSL. CI verifies no dynamic OpenSSL dependencies.
-- **Build Artifacts**: Three primary binaries are built: `cosmian`, `cosmian_kms`, `cosmian_findex_server`
-- **Database Testing**: Only sqlite works in debug mode and on macOS. Full database testing requires release builds.
-- **FIPS vs non-FIPS**: Redis-findex database support is not available in FIPS mode
-- **UI Building**: UI is only built on Ubuntu distributions and requires separate build script
-- **Packaging**: Debian and RPM packages are created as part of release builds with proper FIPS/non-FIPS variants
-- **HSM Support**: Utimaco HSM testing is included but only runs on Ubuntu with specific setup
-- **MySQL**: MySQL database tests are currently disabled in CI
-- **Workspace**: Build from workspace root using cargo_build.sh script, not individual crate directories
-- **HashMaps**: Verify that when creating HashMaps, use `HashMap::with_capacity(n)` to avoid unnecessary reallocations instead of `HashMap::new()`.
+Images include the UI at `http://localhost:9998/ui`.