Zephyrite

Zephyrite is a modern key-value database designed for speed, reliability, and scalability. It combines the performance of in-memory operations with the durability of persistent storage.

Please note: This project is currently in active development and may undergo significant changes. The API and features are subject to change as we refine the design and implementation. Contributions are welcome!

Zephyrite

✨ Current Features

Fast In-Memory Storage: High-performance key-value operations with metadata tracking
Persistent Storage with WAL: Write-Ahead Logging ensures durability and crash recovery
Crash Recovery: Automatic restoration of data from WAL on server restart
HTTP REST API: Fully functional interface for all CRUD operations
Comprehensive Validation: Robust key and value validation with security checks
Structured Logging: Detailed tracing and observability
Error Handling: Comprehensive error responses with proper HTTP status codes
Metadata Tracking: Automatic timestamps and size tracking for stored values
Flexible Storage Options: Choose between in-memory or persistent storage modes

🚀 Quick Start

Prerequisites

Rust 1.85+ (Edition 2024 support)
Just (task runner) - cargo install just
cargo-nextest (enhanced test runner) - just install-nextest
Node.js (optional) - For automated commit tools and validation

Installation

# Clone the repository
git clone https://github.com/yourusername/zephyrite
cd zephyrite

# Install nextest (enhanced test runner)
just install-nextest

# Build the project
just build

Optional: Install additional testing tools for coverage and watch mode:

just install-test-tools  # Installs nextest, cargo-llvm-cov, and cargo-watch

Running the Server

# Start the server with in-memory storage (default port 8080)
just run

# Start with persistent storage and crash recovery
cargo run -- --persistent

# Start with custom WAL file path
cargo run -- --wal-file ./data/my-database.wal

# Start with custom configuration
cargo run -- --port 3000 --log-level debug --persistent --memory-capacity 10000

API Usage Examples

Once the server is running, you can interact with it using curl:

Health Check:

curl -X GET http://localhost:8080/health
# Response: {"status":"ok","version":"0.1.0","service":"Zephyrite"}

Store a Key-Value Pair:

curl -X PUT http://localhost:8080/keys/user:john \
  -H "Content-Type: application/json" \
  -d '{"value": "{\"name\":\"John Doe\",\"age\":30}"}'
# Response: 201 Created (for new keys) or 200 OK (for updates)

Retrieve a Value:

curl -X GET http://localhost:8080/keys/user:john
# Response: {
#   "key": "user:john",
#   "value": "{\"name\":\"John Doe\",\"age\":30}",
#   "found": true,
#   "size": 28,
#   "created_at": "2025-06-22T10:30:14.050Z",
#   "updated_at": "2025-06-22T10:30:14.050Z"
# }

List All Keys:

curl -X GET http://localhost:8080/keys
# Response: {"keys":["user:john","config:app"],"count":2}

Delete a Key:

curl -X DELETE http://localhost:8080/keys/user:john
# Response: 204 No Content (if key existed) or 404 Not Found

Error Handling:

curl -X GET http://localhost:8080/keys/nonexistent
# Response: {"error":"key_not_found","message":"Key 'nonexistent' not found"}

🛠️ Development

Commands

# Development workflow
just dev      # Format + lint + test (comprehensive)
just run      # Start the server

# Testing (powered by nextest)
just test     # Run tests with nextest (local-dev profile)
just test-fast # Quick test run with fail-fast behavior
just test-all  # Run all tests including doctests
just test-ci   # CI profile with retries and JUnit output

# Code quality
just fmt      # Format code
just lint     # Run linting

# Additional testing commands (run 'just test-help' for full list)
just test-unit        # Unit tests only
just test-integration # Integration tests only
just test-storage     # Storage-specific tests
just test-coverage    # Generate HTML coverage report
just test-watch       # Watch mode for continuous testing
just test-profiles    # Show available nextest profiles

API Endpoints

Method	Endpoint	Description	Status
`GET`	`/health`	Health check	✅ Done
`PUT`	`/keys/{key}`	Store a key-value pair	✅ Done
`GET`	`/keys/{key}`	Retrieve a value	✅ Done
`DELETE`	`/keys/{key}`	Delete a key	✅ Done
`GET`	`/keys`	List all keys	✅ Done

Request/Response Format

Store a value:

PUT /keys/mykey
Content-Type: application/json

{
  "value": "myvalue"
}

Response:

201 Created - New key created
200 OK - Existing key updated
400 Bad Request - Invalid key or value

Retrieve a value:

GET /keys/mykey

Response:

{
  "key": "mykey",
  "value": "myvalue",
  "found": true,
  "size": 7,
  "created_at": "2025-06-22T10:30:14.050Z",
  "updated_at": "2025-06-22T10:30:14.050Z"
}

List keys:

GET /keys

Response:

{
  "keys": ["key1", "key2", "key3"],
  "count": 3
}

Delete a key:

DELETE /keys/mykey

Response:

204 No Content - Key successfully deleted
404 Not Found - Key does not exist
400 Bad Request - Invalid key format

Error Response Format:

{
  "error": "error_code",
  "message": "Human-readable error description"
}

🧪 Testing

Zephyrite uses cargo-nextest for enhanced test execution with better performance, improved output formatting, and advanced CI/CD integration.

Quick Testing

# Run all tests (recommended for development)
just test

# Quick iteration with fail-fast behavior
just test-fast

# Run all tests including doctests
just test-all

# Get help with all testing commands
just test-help

Targeted Testing

# Run only unit tests (excludes integration tests)
just test-unit

# Run only integration tests (HTTP API)
just test-integration

# Run storage-specific tests
just test-storage

Advanced Testing

# Generate HTML coverage report (opens in browser)
just test-coverage

# Watch mode - auto-rerun tests on file changes
just test-watch

# CI mode with retries and JUnit output
just test-ci

# Show available nextest profiles
just test-profiles

Nextest Profiles

Zephyrite includes several optimized test profiles:

local-dev: Default profile for development with verbose output
fast: Quick profile with fail-fast behavior for rapid iteration
integration-only: Runs integration tests single-threaded to avoid port conflicts
ci: CI profile with automatic retries and JUnit XML output
coverage: Optimized for code coverage collection

Test Organization

The test suite includes:

Unit Tests: Storage engine, validation, utilities, and WAL operations
Integration Tests: HTTP API endpoints, error scenarios, and end-to-end workflows
Documentation Tests: Code examples in documentation and API contracts
Storage Tests: In-memory and persistent storage with crash recovery scenarios

Test Features

Parallel Execution: Tests run in separate processes for better isolation
Smart Scheduling: Longer tests start first to optimize total runtime
Port Management: Integration tests run single-threaded to prevent conflicts
Automatic Retries: Flaky tests are automatically retried in CI mode
JUnit Output: XML reports for CI/CD integration
Coverage Reports: HTML coverage reports with detailed metrics

🔧 Configuration

Zephyrite supports comprehensive command-line configuration:

Basic Options

# Default: port 8080, in-memory storage, info logging
just run

# Custom port
cargo run -- --port 9090

# Debug logging
cargo run -- --log-level debug

# Combined options
cargo run -- --port 3000 --log-level trace

Available log levels: trace, debug, info, warn, error

Persistent Storage & Crash Recovery

# Enable persistent storage with default WAL file (zephyrite.wal)
cargo run -- --persistent

# Custom WAL file location
cargo run -- --wal-file ./data/database.wal

# Set initial memory capacity (entries)
cargo run -- --persistent --memory-capacity 50000

# Disable WAL checksums (faster writes, less safe)
cargo run -- --persistent --no-checksums

# Full configuration example
cargo run -- \
  --port 8080 \
  --log-level info \
  --wal-file ./data/prod.wal \
  --memory-capacity 100000

Crash Recovery Behavior

When using persistent storage (--persistent or --wal-file):

On Startup: Zephyrite automatically reads the WAL file and replays all operations
During Operation: All write operations (PUT, DELETE, CLEAR) are logged to WAL before execution
On Crash: Data is preserved in the WAL and will be recovered on next startup
Checksum Verification: WAL entries are verified for integrity (can be disabled with --no-checksums)

The recovery process is automatic and requires no manual intervention.

🗺️ Development Roadmap

Phase 1: Foundation

Phase 2: Persistence (WIP)

Phase 3: Distribution (Planned)

Node discovery
Data replication
Consistent hashing
Cluster management

Phase 4: Advanced Features (Planned)

Consensus protocol (Raft)
Transactions
Performance optimizations
Metrics and monitoring

📚 Documentation

Nextest Setup - Comprehensive testing setup and configuration
Key Validation Rules - Detailed key validation specifications
Conventional Commits Setup - Detailed commit message guide
API Documentation (complete - see above)
Development Guide (coming soon)

🤝 Contributing

We welcome contributions! Please see our development setup:

Install prerequisites: Rust 1.85+, Just, and nextest (just install-nextest)
Optional: Install Node.js for enhanced commit tools and additional testing tools (just install-test-tools)
Clone and build: git clone ... && just build
Setup conventional commits: just setup-git
Run tests: just test (or just test-fast for quick iteration)
Follow our workflow: just dev before committing (runs format + lint + comprehensive tests)
Use conventional commits: See just commit-examples for format

Testing workflow: Use just test-help to see all available testing commands. We recommend just test-fast during development and just test-all before committing.

📄 License

Zephyrite is unlicensed software at the moment, meaning it is free to use without restrictions. However, contributions are welcome under the Contributing Guidelines. Please note that contributions may be subject to future licensing changes.

🙋 Support

Issues: GitHub Issues
Discussions: GitHub Discussions

Name		Name	Last commit message	Last commit date
Latest commit History 67 Commits
.cargo		.cargo
.github		.github
.vscode		.vscode
dagger		dagger
docs		docs
src		src
tests		tests
.commitlintrc.json		.commitlintrc.json
.gitignore		.gitignore
.gitmessage		.gitmessage
.rustfmt.toml		.rustfmt.toml
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
README.md		README.md
SECURITY.md		SECURITY.md
justfile		justfile
package-lock.json		package-lock.json
package.json		package.json

minikin/zephyrite

Folders and files

Latest commit

History

Repository files navigation

Zephyrite

✨ Current Features

🚀 Quick Start

Prerequisites

Installation

Running the Server

API Usage Examples

🛠️ Development

Commands

API Endpoints

Request/Response Format

🧪 Testing

Quick Testing

Targeted Testing

Advanced Testing

Nextest Profiles

Test Organization

Test Features

🔧 Configuration

Basic Options

Persistent Storage & Crash Recovery

Crash Recovery Behavior

🗺️ Development Roadmap

Phase 1: Foundation

Phase 2: Persistence (WIP)

Phase 3: Distribution (Planned)

Phase 4: Advanced Features (Planned)

📚 Documentation

🤝 Contributing

📄 License

🙋 Support

About

Topics

Resources

Code of conduct

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Uh oh!

Contributors 4

Uh oh!

Languages