Skip to content

HACKING.md

Latest commit

 

History

History
164 lines (108 loc) · 5.73 KB

HACKING.md

File metadata and controls

164 lines (108 loc) · 5.73 KB

MicroCeph Hacking Guide

Wait! No-one's "hacking" anything. The aim of this document is to enable new developers/users get familiarised with the build tools and MicroCeph codebase so that they can build and contribute to the MicroCeph Codebase.

Table of Contents

  1. ⚡️ Introduction to snaps
  2. 🧰 Tools
  3. 📖 References
  4. 🏭 Build Guide
  5. 👍 Unit-Testing
  6. DB Schema Updates

⚡️ Introduction to snaps

A snap is a bundle of an app and its dependencies that works without modification across Linux distributions.

Apart from being a self-contained artifact, being a snap enables MicroCeph to be isolated from host and provide a clean and consistent building, installing and cleanup experience.

The microceph snap packages all the required ceph-binaries, dqlite and a small management daemon (microcephd) which ties all of this together. Using the light-weight distributed dqlite layer, MicroCeph enables orchestration of a ceph cluster in a centralised and easy to use manner.

🧰 Tools

Snaps are built and published using another snap called snapcraft. It uses lxd to pull dependencies and build an artifact completely isolated from the host system. This makes it easier for developers to work on MicroCeph without polluting their host system with unwanted dependencies. You can install snapcraft and lxd using snap tool.

sudo snap install snapcraft --classic
sudo snap install lxd

Note

For a detailed how-to-use snapcraft tool guide, check-out Snap-Tutorials

📖 References

The MicroCeph codebase resides inside the microceph sub-directory of the repo. This subdir is neatly organised into parts which make up the whole of MicroCeph. Below is a brief description of what can be found in each of those parts for reference.

Sub Directories

  1. api

    Contains files related to the internal REST APIs which the microceph client uses to communicate to the microceph daemon. It also contains a types subdir which has necessary data structures for the APIs.

  2. ceph

    Contains files directly related to ceph orchestration. This includes code for ceph cluster configuration, service orchestration etc.

  3. client

    Contains REST client code which is used by microceph CLI for interacting with microceph daemon.

  4. cmd

    Contains microceph CLI commands written with Cobra Commands

  5. common

    Contains common code

  6. database

    Contains DQlite schema and migration definitions along with generated mappers for DB interfacing.

  7. mocks

    Contains mock definitions for unit testing

🏭 Build Guide

Building MicroCeph is as easy as a snap!

# v for verbose output of the build process.
snapcraft -v
...
Creating snap package
...
Created snap package microceph_0+git.ac1da26_amd64.snap

The newly created .snap artifact can then be installed as

# Dangerous flag for locally built snap
sudo snap install --dangerous microceph_*.snap
# Locally built snaps do no auto-connect the available plugs on install, they can be connected manually using;
sudo snap connect microceph:block-devices
sudo snap connect microceph:hardware-observe
sudo snap connect microceph:mount-observe
sudo snap connect microceph:load-rbd
sudo snap connect microceph:microceph-support
sudo snap connect microceph:network-bind
sudo snap connect microceph:process-control
sudo snap connect microceph:dm-crypt
sudo snap restart microceph.daemon

👍 Unit-Testing

The MicroCeph Makefile has targets for running unit tests and lint checks. However, you will need the following packages or tool to run them locally.

# Add general requirements
sudo apt install gcc make shellcheck

# Add libdqlite-dev, required for building microceph
# you may need a specific version such as libdqlite1.17-dev
sudo add-apt-repository ppa:dqlite/dev -y
sudo apt install -y libdqlite-dev

# Install go and export the binary to PATH
sudo snap install go --classic
export PATH=$PATH:$HOME/go/bin

Once you install the prerequisite, you can run unit tests and lint checks as follows:

cd microceph

# Run unit tests
make check-unit

# Run static checks
make check-static

DB Schema Updates

MicroCeph uses lxd-generate to auto-generate database mapper code (SQL statements and Go CRUD functions) from struct definitions. When adding a new database entity or modifying //go:generate directives, you need to regenerate the corresponding .mapper.go file.

Installing lxd-generate

Install the generator matching the version of github.com/canonical/lxd in go.mod:

# Check the pinned version
grep 'canonical/lxd ' go.mod

# Install it (replace the version hash as needed)
go install github.com/canonical/lxd/lxd/db/generate@v0.0.0-20241106165613-4aab50ec18c3

# The binary installs as "generate" — create a symlink with the expected name
ln -s ~/go/bin/generate ~/go/bin/lxd-generate

# Ensure ~/go/bin is in your PATH
export PATH="$HOME/go/bin:$PATH"

Running the generator

Target the specific file:

cd microceph
go generate ./database/host_tag.go

This reads the //go:generate mapper directives in the source file and produces the corresponding .mapper.go file with SQL statements and Go functions.