[RFC] Introduce Asynchronous Filesystem in NGO

[liqinggd]

• Feature Name: Introduce Asynchronous Filesystem
• Start Date: 2022-06-15

Summary

We will add the newly designed and implemented AsyncFS into NGO. The AsyncFS uses Rust asynchronous programming skills, making it fast and concurrent.

Motivation

In NGO, we have already implemented many features using async programming, but the FS part is still synchronized, which makes the file I/O very slow. To improve the performance of file I/O, we design and implement a new FS. The APIs of AsyncFS are all asynchronous.

High-level Design

We newly design the hierarchy of the FS in Occlum as follows. The Async VFS, Async SFS, and Page Cache are three new crates.

The Async-VFS is the abstraction of FS and provides some facilities for OS to perform file operations.

The Async-SFS is the core implementation of FS, it organizes raw blocks into files and dirs and provides all the inode operations for files.

The Page Cache is a memory cache for block devices, it caches the blocks in the middle of FS and block-device to accelerate file I/O.

Detail-level explanation

Async-VFS

It provides the AsyncFileSystem, AsyncInode, AsyncFileHandle, and Dentry.

AsyncFileSystem is a trait for Async FS. The Async FS should implement this trait.

AsyncInode is a trait of FS Inode in Async FS. The Inode inside one Async FS should implement its methods.

AsyncFileHandle is the representation of one opened file in libos. If Apps open a file, it will create the AsyncFileHandle, then insert it into the file table.

Dentry is used to speed up the file lookup operation, and provides the method to get the absolute path of one inode.

Async-SFS

The Async-SFS is rewritten from https://github.com/occlum/sefs/tree/unionfs/rcore-fs-sfs, so async-sfs inherits some parameters and code from it.

We have mostly redesigned and rewritten the implementation of SFS to satisfy the requirements of Rust async programming, better performance, correctness, and stability. Meanwhile, we also adopt some design ideas from EXT2 FS.

The Block Layout of FS

Async-SFS uses the Block-Device (a trait) as the underlying data storage and sends async IO requests to it.

All the metadata and data of FS are organized into logic blocks. The FS reads or writes data at block-level I/O. It provides inode-level operation to upper FS users.

The Block Indexing of Inode

Every Inode must store the information to locate the data block on the device through file offset. The three-level indexing mechanism is supported now, i.e., direct block, indirect block, and double indirect block.

The Directory Entry

A directory is a specially formatted file containing file records, It is like a linked list of directory entry structures.

Performance

To improve the performance of file I/O, Async-SFS adds Page Cache as the feature in Cargo.toml.

Async-SFS stores both data and metadata into the page cache to accelerate block I/O operations. Thanks to the page cache, the result of the benchmark (e.g., FIO and Filebench) is significantly better than SEFS.

Current Limitation

• The maximum size of the file inside Async-SFS is 4GB because the type of file size is u32, and the inode just uses double indirect blocks.

• The maximum size of FS is 16TB because the type of BlockId is u32. The size is 4KB * 2 ^ 32 = 16TB.

Page Cache

Please refer to the issue: #267