diff --git a/eeps/dynamic-libraries.md b/eeps/dynamic-libraries.md new file mode 100644 index 0000000..4ccd26d --- /dev/null +++ b/eeps/dynamic-libraries.md @@ -0,0 +1,71 @@ +--- +Title: Dynamic Libraries +Author: interacsion +Status: Draft +Type: Informational +Created: 2025-05-11 +--- + +# Preface + +*Dynamic Libraries* (also referred to as *Shared Libraries*, *Shared Objects* or just *Libraries*) are files containing executable code (*Executables*) that can be loaded into a process at runtime. This is generally done in one of 2 ways: + +### 1. *Dynamic Linking* + +An *Executable* (either a *Binary*, which contains an entry point, or a *Library*) can be compiled against a Dynamic Library. When such Binary is run it will first invoke the *Dynamic Linker* (also referred to as *Dynamic Loader* or *ELF Interpreter*) which will in turn recursively load all required Dynamic Libraries into process memory. + +### 2. *Dynamic Loading* + +A Dynamic Library can also be loaded into process memory by calling `dlopen` (or `dlmopen`, more on that later). The Executable does not need to be compiled against the Library, and can conditionally load different Libraries. + +## Dynamic Libraries in Nix + +On traditional Linux distributions, Dynamic Libraries are usually linked to by their *`SONAME`* (The filename if the Shared Library). The Dynamic Linker then searches a predefined set of system directories to find it. + +Nix, on the other hand, takes a different approach; since it doesn't install packages into standard directories, it links Dynamic Libraries by their full filepath, and this is where the problem starts. + +# Problem Statement + +Most Dynamic Libraries assume that the Library and the Executable that uses it have access to the same instance of a Dynamic Library they both depend on. + +The most obvious example is a Library which allocates memory and expects the User to free it. This requires that both the Library and the User share the same instance of, usually, glibc. + +The way the Dynamic linker accommodates this is as following: Before a Library is loaded it checks if a Library with the same `SONAME` is already loaded into the process, disregarding the full filepath, if specified. if so, it substitutes it for the one that is already loaded. + +This works fine on traditional Linux distributions, but **actively undermines** the purity guarantees Nix tries to make. For example, if a Binary is linked to a certain version of glibc and to a Library that's linked to a different version of glibc, the first one that gets loaded will trump the other. + +Nixpkgs manages to mostly avoid this problem due to its monorepo nature; there's a single attribute set containing all the packages and as long as a Binary only links to dependencies from a single Nixpkgs revision, no conflict can occur. + +Ekapkgs, on the other hand, would have to manage this more carefully due to its polyrepo structure. For example, a binary from `pkgs` could depend on one `corepkgs` revision and on a library from `ocaml-pkgs` which depends on a different `corepkgs` revision, which would attempt to load two different glibc versions. + +## System Libraries + +A small subset of Dynamic Libraries must be installed as part of the system; most notable are graphics drivers (which are loaded using `libvulkan`, `libglvnd` or `libgbm`), but GPGPU and sound systems probably fall into this category as well. + +Since these Libraries are installed separately than the applications using them, they cannot be expected to link to the same version of glibc as the application. This is a problem that actually manifests in NixOS (Citation Needed). + +# Non-Solutions + +### 1. Patching the Dynamic Linker to load multiple versions of the same Library + +This is not a solution since *Most Dynamic Libraries assume that the Library and the Executable that uses it have access to the same instance of a Dynamic Library they both depend on.* + +# Proposed Solution + +The following is a concept of a 3-part system which should ultimately resolve all the aforementioned problems. + +### 1. Dependency tree analysis + +At either package evaluation or build time, we would analyze the tree of *Linked* Dynamic Libraries and ensure only one version of each Library is present. + +### 2. Runtime Loading validation + +We would have to patch glibc and make `dlopen` error out when a different version of a Library is already loaded, to prevent accidental impure behavior. + +### 3. System Library isolation + +System Libraries would have to run in a separate *link-map list* than the rest of the process, which would allow them to link to a different glibc version. This can be done in 2 ways: + +1. We can patch Libraries like `libvulkan` to use `dlmopen` instead of `dlopen` when loading drivers. + +2. We can write shims for Libraries like `libvulkan` that would load the real Library using `dlmopen`.