Support C API dynamic loading using dlsym()

[kevinthornberry]

Currently the C wrapper classes link directly to the underlying C functions, meaning the library must be present at build-time. Add support for lazy-loading the C functions at runtime using dlsym(), enabled with a new --c-wrapper-use-dlsym=true generator argument.

Example djinni (namespace test):

Foo = interface {
    bar(n: i32);
    baz(s: string): bool;
}

C wrapper functions before:

    void bar(int32_t n)  {
        test_Foo_bar(_ref, n);
    }

    bool baz(djinni_string_ref s)  {
        return test_Foo_baz(_ref, s);
    }

C wrapper functions after:

    void bar(int32_t n)  {
        _loadFuncs()->test_Foo_bar(_ref, n);
    }

    bool baz(djinni_string_ref s)  {
        return _loadFuncs()->test_Foo_baz(_ref, s);
    }

private:
    struct Funcs {
        decltype(&test_Foo_bar) test_Foo_bar;
        decltype(&test_Foo_baz) test_Foo_baz;
    };

    static inline Funcs* _loadFuncs() {
      auto loadAndAssert = [](const char* funcName) {
        auto* ptr = DJINNI_C_LOAD_SYM_FUNC(DJINNI_C_LIB_HANDLE_test_Foo, funcName);
        assert(ptr && "Failed to load djinni C functions for class test::c_wrappers::Foo");
        return ptr;
      };
      static Funcs funcs {
        reinterpret_cast<decltype(&test_Foo_bar)>(loadAndAssert("test_Foo_bar")),
        reinterpret_cast<decltype(&test_Foo_baz)>(loadAndAssert("test_Foo_baz")),
      };
      return &funcs;
    }

DJINNI_C_LOAD_SYM_FUNC(libHandle, functionName) can be defined by the platform to provide a symbol loader function similar to dlsym(). If it is not explicitly defined dlsym() will be used by default.
DJINNI_C_LIB_HANDLE_test_Foo can be defined to reference a specific handle opened with DJINNI_C_LOAD_SYM_FUNC(). Also DJINNI_C_DEFAULT_LIB_HANDLE can be defined as a global default (individual wrapper classes can still use their own definitions overriding the default). If neither of these is defined RTLD_DEFAULT will be used as the handle.

If the --c-wrapper-use-dlsym=true argument is not supplied, the generated code is identical to the current version.

[li-feng-sc]

I wonder if you could add an abstraction layer on top of dlopen/dlsym. This way the generated code won't be tied to platforms.

[kevinthornberry]

Hi @li-feng-sc I have updated the PR to add an abstraction around dlsym() so a platform can provide their own symbol loader implementation.

[li-feng-sc]

@kevinthornberry I mean something like this:

class DynamicLibrary {
public:
    using Handle = void*;

    DynamicLibrary() = default;

    explicit DynamicLibrary(const char* path) {
        open(path);
    }

    ~DynamicLibrary() {
        close();
    }

    DynamicLibrary(const DynamicLibrary&) = delete;
    DynamicLibrary& operator=(const DynamicLibrary&) = delete;

    DynamicLibrary(DynamicLibrary&& other) noexcept
        : handle_(std::exchange(other.handle_, nullptr)) {}

    DynamicLibrary& operator=(DynamicLibrary&& other) noexcept {
        if (this != &other) {
            close();
            handle_ = std::exchange(other.handle_, nullptr);
        }
        return *this;
    }

    void open(const char* path) {
        close();
        handle_ = ::dlopen(path, RTLD_NOW);
    }

    void close() noexcept {
        if (handle_) {
            ::dlclose(handle_);
            handle_ = nullptr;
        }
    }

    bool isOpen() const noexcept {
        return handle_ != nullptr;
    }

    template <typename T>
    T tryGet(const char* symbol) const noexcept {
        return isOpen() ? reinterpret_cast<T>(::dlsym(handle_, symbol)) : nullptr;
    }

private:
    Handle handle_ = nullptr;
};

And usage:

DynamicLibrary lib("libm.so.6");
auto cos_fn = lib.tryGet<double(*)(double)>("cos");
double v = cos_fn(0.5);

With this library utility, the codegen can simply include a header file and emit portable code.