[RFC] Extend the SGX untrusted allocator to support untrusted device memory

[guzongmin]

• Feature Name: untrusted device memory
• Start Date: 2022-08-22

Summary

Occlum gets the untrusted memory range and registers the untrusted memory range into the SGX untrusted allocator.

Motivation

When supporting the untrusted device, we could get some untrusted memory that is shared with the device. The SGX untrusted allocator could help to manage this area. The application may do mprotect or munmap with the virtual address inside the untrusted area.

Guide-level explanation

The existing sgx-untrusted-alloc crate (https://github.com/occlum/ngo/tree/master/src/libos/crates/sgx-untrusted-alloc) could be extended to support the new feature.

The new interfaces are required:

1. insert a committed untrusted memory range into the allocator
2. remove a range of memory from the allocator (If the range is added by the insert interface)

Below is one example:
When an application performs munmap with an untrusted memory address

1. First, check whether the address is inside the enclave or an outside address.
2. If the address is an outside address, query the Untrusted Allocator to confirm it is a legal address managed by the allocator
3. Perform the munmap with the address
4. If the area is mapped with an untrusted device, remove the range from the allocator

[guzongmin]

Another comment about the untrusted memory allocator is:

1. The allocator could be more flexible to manage big memory blocks and small ones. For example (https://github.com/rcore-os/buddy_system_allocator)
2. The allocator could be wrapped with thread_local to enhance the small block's allocation performance if the usage required.