Add musl support #16
Open
Add musl support #16
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* feat(devices): add overlayfs implementation for macOS Add an overlay filesystem implementation that combines multiple layers into a single logical filesystem, following OCI image specification's layer filesystem changeset format. This implementation: - Supports multiple read-only lower layers and one writable upper layer - Uses OCI-style whiteout files (.wh. prefix) to mark deleted files - Uses OCI-style opaque directory markers (.wh..wh..opq) to mask directories - Includes comprehensive test suite for layer operations - Adds intaglio and tempfile dependencies for symbol interning and testing The implementation provides a foundation for container image support on macOS by allowing multiple filesystem layers to be combined into a single view. * refactor(overlayfs): improve lookup logic and file organization - Split overlayfs.rs into separate fs.rs and tests.rs modules - Simplify layer root handling by replacing path_to_inode_map with layer_roots - Improve lookup logic to handle whiteouts and opaque directories correctly - Add comprehensive test cases for complex directory structures - Add Debug derive for MultikeyBTreeMap and other structs - Remove unused helper methods and simplify path handling - Add helper functions for volume path construction - Improve error handling consistency The main architectural change is moving from a path-based lookup system to a layer-based traversal system that better handles overlay filesystem semantics like whiteouts and opaque directories. * feat(overlayfs): implement setattr and copyup functionality Adds support for modifying file attributes and copying files from lower layers to the upper layer in the macOS overlayfs implementation. Key changes include: - Add setattr support for changing file permissions, ownership, size and timestamps - Implement copyup functionality to promote files to upper layer when modified - Add extended attribute support for storing overlayfs-specific metadata - Add comprehensive tests for operations - Refactor stat operations to handle overlayfs permission overrides - Make inode data fields public within crate for testing * feat(overlayfs): implement unlink operation and add tests Implements the unlink operation for the overlayfs filesystem, allowing files to be deleted from any layer. Key changes include: - Add get_top_layer_idx helper method to get index of top layer - Implement do_unlink to handle file deletion and whiteout creation - Refactor do_forget to handle inode reference counting - Add dev_ino_and_name_to_vol_whiteout_path helper for whiteout files - Add comprehensive test suite for unlink functionality including: - Basic file deletion - Whiteout creation for lower layer files - Multi-layer scenarios - Error cases - Complex directory structures * feat(overlayfs): implement rmdir and fix lookup path handling - Add do_rmdir() implementation for directory removal across layers - Fix path_inodes handling in lookup_segment_by_segment - Add whiteout file creation for removed directories - Fix path handling in do_copyup to skip root inode - Add comprehensive test coverage for rmdir functionality * feat(overlayfs): implement symlink support for OverlayFs and add tests Implement symlink support for the OverlayFs filesystem and add comprehensive test coverage for OverlayFs. Key changes include: - Add do_symlink() implementation in OverlayFs - Add symlink tests covering basic functionality, nested directories, multiple layers, and error cases - Clean up error handling pattern in do_mkdir() * feat(overlayfs): implement rename operation and add tests This commit implements the rename operation for the overlayfs filesystem on macOS and adds extensive test coverage. Key changes include: - Implement `do_rename` method to handle file/directory renaming - Add helper method `create_whiteout_for_lower` to handle whiteout creation - Support LINUX_RENAME_WHITEOUT and LINUX_RENAME_EXCHANGE flags - Add comprehensive tests for rename functionality including: - Basic rename operations - Whiteout handling - Multi-layer scenarios - Complex directory structures - Error cases * feat(overlayfs): implement hard link support for overlayfs Adds support for creating hard links in the overlayfs filesystem implementation. Key changes include: - Implement do_link() method to handle hard link creation - Add comprehensive test suite for link functionality including: - Basic link creation - Links across multiple layers - Error handling - Nested directory scenarios - Whiteout handling * feat(overlayfs): Implement open/release and improve copy-up operations This commit adds several key improvements to the overlayfs implementation: - Implements open() and release() operations for files and directories - Adds CachePolicy enum to control FUSE caching behavior - Improves copy-up operation by: - Using clonefile() for COW semantics when available on macOS - Falling back to regular copy when clonefile fails - Extracting copy logic into separate helper method - Adds comprehensive tests for open/release and copy-up operations - Fixes init_handle value to start at 0 - Renames do_copyup to copy_up for consistency The changes improve performance by using COW semantics where possible and add proper file handle management. The new CachePolicy gives users control over how aggressively the FUSE client caches file data. * feat(overlayfs): Implement read functionality and add comprehensive tests Implement the read method for the OverlayFS filesystem, allowing files to be read from any layer in the overlay stack. The implementation handles: - Basic file reading with offset and size controls - Reading through copied-up files - Proper handling of whiteout files and opaque directories - Special handling for the init binary when not in EFI mode Add extensive test coverage including: - Basic read functionality - Reading with offsets - Partial reads - Reading through nested directories - Proper handling of whiteouts and opaque directories - Reading after copy-up operations - Error cases with invalid handles * feat(overlayfs): Implement write functionality and add tests Implements the write method for the overlayfs filesystem, allowing files to be written to and modified. The implementation includes: - Basic write functionality with offset support - Copy-up behavior when writing to files in lower layers - Proper handling of whiteouts and opaque directories - Support for partial writes and multiple write operations Also adds comprehensive test coverage for the write functionality, including: - Basic write operations - Writing with offsets - Partial writes - Copy-up behavior - Invalid handle handling - Multiple sequential writes - Writing to files in nested directories - Interaction with whiteouts and opaque directories Additionally: - Renames NotReallyZeroCopyWriter to TestContainer - Implements ZeroCopyReader for TestContainer to support write tests - Removes commented out test_open_root_directory * feat(overlayfs): implement flush, fsync and opendir operations for macos overlayfs Implement several filesystem operations for the macOS overlayfs: - Add flush operation that emulates fd close behavior using dup/close - Add fsync operation to synchronize file contents - Implement opendir by reusing existing do_open with O_DIRECTORY flag - Add comprehensive test suite for opendir functionality The tests cover: - Basic directory opening - Handling non-existent directories - Whiteout directory behavior - Copy-up scenarios - Multiple open/release cycles - Various open flags combinations * feat(overlayfs): Skip copy-up for read-only file operations Previously, the overlayfs implementation would copy files to the top layer even for read-only operations. This was unnecessary overhead since files only need to be in the top layer when they are being modified. This change optimizes the behavior by only performing copy-up when write access is requested. The implementation now checks the O_ACCMODE flags to determine if the file needs to be copied to the top layer. Added test cases to verify: - Opening files read-only keeps them in the bottom layer - Opening directories read-only keeps them in the bottom layer * feat(overlayfs): implement extended attributes support Add support for extended attributes (xattrs) in the overlayfs implementation: - Implement setxattr, getxattr, listxattr and removexattr operations - Add xattr configuration flag to enable/disable xattr support - Handle copy-up operations when setting xattrs on files in lower layers - Add comprehensive test coverage for xattr operations including: - Basic set/get/list/remove operations - Copy-up behavior for lower layer files - Error handling for invalid operations - Proper handling when xattr support is disabled * feat(virtio-fs): implement create, mknod and access ops for macOS overlayfs - Add create() operation to support creating new files with permissions - Add mknod() operation for special files (as regular files on macOS) - Add access() operation with Unix permission checking logic - Reorganize tests into separate modules The implementation handles file ownership, permissions, security contexts, and parent directory copy-up when needed. * feat(overlayfs): implement fs operations for macOS overlayfs Implements the following filesystem operations for macOS overlayfs: - fallocate: Preallocates space for files - lseek: Repositions file offset with special handling for SEEK_DATA/SEEK_HOLE - setupmapping: Sets up DAX memory mapping between guest and host - removemapping: Removes DAX memory mappings Also fixes comparison of libc function return values to check for < 0 instead of != 0 to properly handle errors.
* feat(fs): expose overlayfs API for macOS Add public API for using OverlayFS functionality in libkrun on macOS by exposing the implementation through a new FsImplConfig enum. This allows clients to configure either passthrough or overlayfs filesystem modes. Key changes: - Add FsImplConfig enum to select between Passthrough and Overlayfs modes - Add FsImplShare enum to handle different sharing configurations - Refactor Fs implementation to delegate operations to selected backend - Update Config struct to include layers configuration - Clean up and reorganize filesystem server code - Add comprehensive test coverage for overlayfs operations The implementation maintains the existing passthrough functionality while adding the ability to configure overlayfs mode with multiple read-only layers and a writable top layer. * feat(overlayfs): add special handling for init.krun lookup Add special case handling in the lookup method to return a predefined entry when looking up "init.krun" file. This allows exposing the init binary with specific permissions (755) and size based on the included binary data. * fix(overlayfs): improve init.krun inode handling and documentation - Set init.krun inode ID dynamically using next_inode counter - Reorder init.krun inode check before handle data lookup in read() - Add documentation about default values for config options - Clarify proc_sfd_rawfd usage in sandboxing scenarios - Update comments to specifically reference init.krun * refactor(fs): remove debug println statements from overlayfs
* feat(net): Implement IP filtering for TSI backend
This commit introduces IP-based filtering capabilities for the default TSI
(Transparent Socket Impersonation) network backend in libkrun.
A new C API function, `krun_set_tsi_scope`, allows users to configure:
- An optional static IP address for the guest within the host network
namespace. If specified, the guest can only bind/listen on this IP.
- An optional subnet (in CIDR notation) defining the allowed
communication group when scope 1 is used.
- A reachability scope (0-3) controlling network access:
- 0: Deny all IP communication.
- 1: Allow communication only within the specified `subnet`.
- 2: Allow communication only with public (non-private) IPs.
- 3: Allow communication with any IP.
The filtering logic is implemented in `src/devices/src/virtio/vsock/ip_filter.rs`
and integrated into the `VsockMuxer`. It checks destination IPs for connect/sendto
operations and bind IPs for listen operations against the configured rules.
If an operation is denied, an appropriate error (ECONNREFUSED or EACCES)
is sent back to the guest via a vsock control message.
This feature enhances security by allowing finer-grained control over the
network connectivity of krun virtual machines when using the TSI backend.
Changes include:
- Added `krun_set_tsi_scope` to `libkrun.h` and implementation in `lib.rs`.
- Added `ipnetwork` dependency to relevant Cargo.toml files.
- Created `ip_filter.rs` module for filtering logic.
- Updated `Vsock` device, `VsockMuxer`, and `VsockDeviceConfig` to handle
IP, subnet, and scope configuration.
- Integrated filtering checks into `VsockMuxer`'s packet processing methods.
- Added helper functions in `VsockMuxer` to send error responses to the guest.
* refactor(net): rename reach to scope in header file
…sandbox#10) When using scope 1 (Group) for IP filtering, subnet specification is now optional. If no subnet is provided, all connections will be blocked, matching scope 0 behavior. This provides more flexibility in network configuration while maintaining security. - Update IpFilterConfig to make subnet optional for scope 1 - Modify is_valid() to accept scope 1 without subnet - Update documentation in libkrun.h to clarify scope behaviors - Improve warning message specificity in VsockMuxer creation
…x#11) * feat(overlayfs): complete Linux overlayfs implementation Move overlayfs implementation to Linux-specific directory and add comprehensive Linux filesystem support. This change introduces: Core Features: - Implement path traversal with lookup_layer_by_layer and lookup_segment_by_segment - Add copy_up functionality for promoting files from lower to upper layers - Support FICLONE ioctl for efficient Copy-on-Write operations - Implement Linux-specific file operations (fallocate, lseek, copyfilerange) - Add memory mapping support via setupmapping and removemapping Linux Integration: - Handle Linux-specific file descriptors and system calls - Support Linux xattr operations and mount point management - Implement proper device ID handling for Linux filesystems - Add CoW optimizations using Linux-specific filesystem features * chore: cleanup workspace and test configuration - Remove ignore/debug from workspace members - Add #[cfg(test)] annotation to overlayfs test helper module - Remove redundant newline in devices/Cargo.toml * fix(overlayfs): fix linux overlayfs for tests
Add mount -a command to process /etc/fstab entries, enabling virtiofs shares
…sandbox#13) Combine scope 2 (Public) and scope 3 (Any) to have the same behavior, both allowing binding to any IP address. This simplifies the logic by removing the public IP check since both scopes now permit unrestricted IP binding.
Replace system("/bin/mount -a") with direct mounting of virtiofs entries from
/etc/fstab. This provides better error handling and eliminates dependency on
mount(8) binary.
- Add mkdir_p() helper for recursive directory creation
- Add is_mounted() to check existing mount points
- Add clean_opts() to handle mount options
- Add mount_fstab_virtiofs() to process fstab entries
- Only mount virtiofs entries, leaving other filesystems unmounted
|
Great! Reviewing... |
appcypher
requested changes
May 28, 2025
Comment on lines
+5
to
+13
| [workspace.dependencies] | ||
| libc = { git = "https://github.com/frc4533-lincoln/libc.git", branch = "main", features = ["extra_traits"] } | ||
| vmm-sys-util = "0.12.1" | ||
| kvm-bindings = { version = "0.11", features = ["fam-wrappers"] } | ||
| kvm-ioctls = "0.21" | ||
| vm-memory = { version = "0.16", features = ["backend-mmap"] } | ||
| tokio = { version = "1", features = ["rt", "sync"] } | ||
| serde = { version = "1", features = ["derive"] } | ||
|
|
There was a problem hiding this comment.
This is a great contribution! Sorry for the delay. I don't have an Alpine machine to test this on.
I have a few things I would like addressed.
-
With this fork, I plan to stay close to upstream repo as much as possible, maybe even contribute the changes at some point. I understand you are trying to consolidate the use of these libs across multiple crates but I will suggest leaving them be to reduce future rebase issues when pulling upstream changes.
-
For the libc changes, I'm thinking we should replace renameat2 with a
libc::syscall(libc::SYS_renameat2, ...)under musl. There are a few places where a similar thing is done. -
I think you should raise a PR on libc. https://github.com/rust-lang/libc/pulls?q=is%3Apr+author%3Afrc4533-lincoln+
|
I had to rebase develop to bring in some changes. It should be straightforward to rebase: # Make sure the upstream remote exists
git remote add upstream https://github.com/microsandbox/libkrun.git
# Fetch the rewritten upstream branch
git fetch upstream develop
# Locate the OLD base rev (where your dev branch split)
OLD_BASE=$(git merge-base develop upstream/develop)
# Rebase your commits onto the new upstream history
git rebase --onto upstream/develop "$OLD_BASE" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.