Embedded LLD Phase 1: Build Infrastructure + CRT Objects

[SeanTAllen]

Scope

This phase establishes build infrastructure for embedded LLD linking in ponyc. It does NOT change how ponyc links user programs — the legacy system() path remains the only active code path.

What this phase delivers:

• LLD libraries built as part of the vendored LLVM and linked into ponyc
• compiler-rt CRT objects compiled for the host architecture
• CRT objects compiled during cross-compilation builds
• genexe.c and program.c converted to C++ for future LLD API access
• New structured API for library arguments (consumed by Phase 2)

What this phase does NOT do:

• Call any LLD functions
• Change how ponyc links user programs
• Add new CLI options (--sysroot is Phase 2)

Prerequisites

All already in place:

• LLVM submodule at lib/llvm/src/ (hash 2078da43e25a4623)
• LLD source at lib/llvm/src/lld/ (present in monorepo checkout)
• compiler-rt CRT sources at lib/llvm/src/compiler-rt/lib/builtins/crtbegin.c and crtend.c

Steps

Step 1: Add LLD to the LLVM build

File: lib/CMakeLists.txt

Add before add_subdirectory(llvm/src/llvm) (before line 147):

set(LLVM_ENABLE_PROJECTS "lld" CACHE STRING "ponyc: build LLD as an LLVM sub-project")

LLVM's cmake processes LLVM_ENABLE_PROJECTS by iterating the list and calling add_subdirectory("${CMAKE_CURRENT_SOURCE_DIR}/../${proj}" "${proj}") for each entry. From the LLVM source root at lib/llvm/src/llvm/, ../lld resolves to lib/llvm/src/lld/, which exists and contains a CMakeLists.txt.

LLD respects the existing LLVM_ENABLE_ZLIB OFF and LLVM_ENABLE_ZSTD OFF settings — it builds without compression support, which is fine (compressed debug sections are uncommon and not needed for correctness).

The build produces these static libraries in build/libs/lib/:

• liblldCommon.a — shared utilities
• liblldELF.a — ELF linker (Linux, BSD)
• liblldMachO.a — Mach-O linker (macOS)
• liblldCOFF.a — COFF linker (Windows)
• liblldWasm.a — WebAssembly linker
• liblldMinGW.a — MinGW-style linker

LLD headers install to build/libs/include/lld/. LLD cmake config installs to build/libs/lib/cmake/lld/.

CI impact: Changes to lib/CMakeLists.txt invalidate the libs cache (keyed on hashFiles('lib/CMakeLists.txt', ...)). The first CI run after this change will rebuild libs from scratch, including LLD. Subsequent runs cache normally.

Step 2: Link LLD libraries into ponyc

File: CMakeLists.txt (top-level)

After the existing find_package(LLVM ...) on line 27, add:

find_package(LLD REQUIRED CONFIG PATHS "build/libs/lib/cmake/lld" "build/libs/lib64/cmake/lld" NO_DEFAULT_PATH)

The lib64 path mirrors the existing pattern used by find_package(LLVM ...), find_package(GTest ...), and find_package(benchmark ...) on the same lines.

After the llvm_map_components_to_libnames call (line 216), define LLD libraries:

set(PONYC_LLD_LIBS lldELF lldMachO lldCOFF lldWasm lldMinGW lldCommon)

lldCommon must be last — the other LLD libraries depend on it. Link order matters for static archives.

File: src/ponyc/CMakeLists.txt

Line 77 is outside the MSVC/non-MSVC conditional (which ends at line 74) and applies to all platforms. Change it to include LLD:

target_link_libraries(ponyc PRIVATE ${PONYC_LLVM_LIBS} ${PONYC_LLD_LIBS})

No changes needed to the MSVC block (lines 18-27) — line 77 already covers MSVC.

File: test/libponyc/CMakeLists.txt

Change line 135:

target_link_libraries(libponyc.tests PRIVATE ${PONYC_LLVM_LIBS} ${PONYC_LLD_LIBS})

File: benchmark/libponyc/CMakeLists.txt

Change line 51:

target_link_libraries(libponyc.benchmarks PRIVATE ${PONYC_LLVM_LIBS} ${PONYC_LLD_LIBS})

This is the only benchmark target that links ${PONYC_LLVM_LIBS} (benchmark/libponyrt does not).

Binary size note: Since no code references LLD symbols yet, the static linker won't pull in any LLD object files from the archives. The LLD_HAS_DRIVER macro (Step 6) expands to a forward declaration only — not a symbol reference. Binary size increase in Phase 1 should be zero. When Phase 2 actually calls LLD driver functions, only the referenced driver's code gets pulled in.

Step 3: Update libponyc-standalone to include LLD

The standalone library bundles all LLVM libraries into a single archive. It needs to also include LLD.

File: src/libponyc/CMakeLists.txt

Linux (around line 217): The find command currently globs libLLVM*.a. Add a second command for LLD right after it:

COMMAND find ${PROJECT_SOURCE_DIR}/../../build/libs/ -name "liblld*.a" | xargs -I % -n 1 echo 'addlib %' >> standalone.mri

macOS (around line 150): Add a glob for LLD archives and include them in the libtool command:

file(GLOB_RECURSE LLD_OBJS "${PROJECT_SOURCE_DIR}/../../build/libs/lib/liblld*.a")

Then include ${LLD_OBJS} in the libtool -static -o command alongside ${LLVM_OBJS}.

FreeBSD (around line 169): Add the same find ... liblld*.a line after the libLLVM*.a line.

MSVC (around line 132): Add LLD .lib files:

file(GLOB_RECURSE LLD_OBJS "${PROJECT_SOURCE_DIR}/../../build/libs/lib/lld*.lib")

Include ${LLD_OBJS} in the lib.exe command alongside ${LLVM_OBJS}.

Note: OpenBSD and DragonFly have # TODO stubs (lines 188-191) and don't currently build standalone libraries. This plan doesn't change that — LLD additions for those platforms would come when their standalone builds are implemented.

Step 4: Build compiler-rt CRT objects

Rather than adding compiler-rt as an LLVM sub-project (which drags in its complex cmake infrastructure — CompilerRTUtils, BuiltinTests, filter_available_targets, etc.), compile the two CRT source files directly. They are small, self-contained C files that only need a standard C compiler.

New file: src/crt/CMakeLists.txt

cmake_minimum_required(VERSION 3.21 FATAL_ERROR)

project(ponyc_crt LANGUAGES C)

set(CRT_SRC_DIR ${CMAKE_SOURCE_DIR}/lib/llvm/src/compiler-rt/lib/builtins)
set(CRT_COMMON_FLAGS -DCRT_HAS_INITFINI_ARRAY -DEH_USE_FRAME_REGISTRY)

# PIE/shared variants (for dynamic linking — the default)
add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/crtbeginS.o
    COMMAND ${CMAKE_C_COMPILER} -c ${PONY_PIC_FLAG} ${CRT_COMMON_FLAGS}
        -o ${CMAKE_CURRENT_BINARY_DIR}/crtbeginS.o
        ${CRT_SRC_DIR}/crtbegin.c
    DEPENDS ${CRT_SRC_DIR}/crtbegin.c
    COMMENT "Compiling crtbeginS.o (PIC)")

add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/crtendS.o
    COMMAND ${CMAKE_C_COMPILER} -c ${PONY_PIC_FLAG} ${CRT_COMMON_FLAGS}
        -o ${CMAKE_CURRENT_BINARY_DIR}/crtendS.o
        ${CRT_SRC_DIR}/crtend.c
    DEPENDS ${CRT_SRC_DIR}/crtend.c
    COMMENT "Compiling crtendS.o (PIC)")

# Static variants (for --static linking)
add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/crtbeginT.o
    COMMAND ${CMAKE_C_COMPILER} -c ${CRT_COMMON_FLAGS}
        -o ${CMAKE_CURRENT_BINARY_DIR}/crtbeginT.o
        ${CRT_SRC_DIR}/crtbegin.c
    DEPENDS ${CRT_SRC_DIR}/crtbegin.c
    COMMENT "Compiling crtbeginT.o (static)")

add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/crtend.o
    COMMAND ${CMAKE_C_COMPILER} -c ${CRT_COMMON_FLAGS}
        -o ${CMAKE_CURRENT_BINARY_DIR}/crtend.o
        ${CRT_SRC_DIR}/crtend.c
    DEPENDS ${CRT_SRC_DIR}/crtend.c
    COMMENT "Compiling crtend.o (static)")

add_custom_target(crt_objects ALL
    DEPENDS
        ${CMAKE_CURRENT_BINARY_DIR}/crtbeginS.o
        ${CMAKE_CURRENT_BINARY_DIR}/crtendS.o
        ${CMAKE_CURRENT_BINARY_DIR}/crtbeginT.o
        ${CMAKE_CURRENT_BINARY_DIR}/crtend.o)

# Copy to output directory alongside libponyrt.
# Uses the same per-config pattern as src/libponyrt/CMakeLists.txt and
# src/libponyc/CMakeLists.txt for the POST_BUILD copy commands.
set(CRT_FILES
    ${CMAKE_CURRENT_BINARY_DIR}/crtbeginS.o
    ${CMAKE_CURRENT_BINARY_DIR}/crtendS.o
    ${CMAKE_CURRENT_BINARY_DIR}/crtbeginT.o
    ${CMAKE_CURRENT_BINARY_DIR}/crtend.o)

foreach(CRT_FILE ${CRT_FILES})
    add_custom_command(TARGET crt_objects POST_BUILD
        COMMAND $<$<CONFIG:Debug>:${CMAKE_COMMAND}> ARGS -E copy ${CRT_FILE} ${CMAKE_BINARY_DIR}/../debug${PONY_OUTPUT_SUFFIX}/
        COMMAND $<$<CONFIG:Release>:${CMAKE_COMMAND}> ARGS -E copy ${CRT_FILE} ${CMAKE_BINARY_DIR}/../release${PONY_OUTPUT_SUFFIX}/
        COMMAND $<$<CONFIG:RelWithDebInfo>:${CMAKE_COMMAND}> ARGS -E copy ${CRT_FILE} ${CMAKE_BINARY_DIR}/../relwithdebinfo${PONY_OUTPUT_SUFFIX}/
        COMMAND $<$<CONFIG:MinSizeRel>:${CMAKE_COMMAND}> ARGS -E copy ${CRT_FILE} ${CMAKE_BINARY_DIR}/../minsizerel${PONY_OUTPUT_SUFFIX}/
    )
endforeach()

Compile flags explained:

• CRT_HAS_INITFINI_ARRAY: Use .init_array/.fini_array sections (standard on all modern Linux). Without this, crtbegin.c falls back to legacy .ctors/.dtors sections.
• EH_USE_FRAME_REGISTRY: Include .eh_frame section and frame registration code for exception handling support.
• PONY_PIC_FLAG (for S variants): Position-independent code, needed for PIE executables and shared libraries. ponyc defaults to PIE.

CRT variant naming follows GCC conventions:

File	PIC	Use case
crtbeginS.o	Yes	PIE executables, shared libraries (default)
crtendS.o	Yes	PIE executables, shared libraries (default)
crtbeginT.o	No	Fully static executables (--static)
crtend.o	No	Fully static executables (--static)

File: CMakeLists.txt (top-level)

Add the CRT subdirectory, guarded to only build on Linux (ELF targets that need CRT objects):

if(PONY_CROSS_LIBPONYRT)
    add_subdirectory(src/libponyrt)
    if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
        add_subdirectory(src/crt)
    endif()
else()
    # ... existing subdirectories ...
    if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
        add_subdirectory(src/crt)
    endif()
endif()

macOS doesn't use CRT objects (uses -lSystem instead). Windows uses MSVC CRT. BSD CRT objects are handled in Phase 6.

For cross-compilation, CMAKE_SYSTEM_NAME is set to Linux by the cross-libponyrt Makefile target (-DCMAKE_SYSTEM_NAME=Linux), so the CRT objects are compiled with the cross-compiler automatically.

File: Makefile

Update the cross-libponyrt target (line 220) to also build CRT objects:

cross-libponyrt:
	$(SILENT)mkdir -p $(crossBuildDir)
	$(SILENT)cd '$(crossBuildDir)' && env CC=$(CC) CXX=$(CXX) cmake ...
	$(SILENT)cd '$(crossBuildDir)' && env CC=$(CC) CXX=$(CXX) cmake --build '$(crossBuildDir)' --config $(config) --target libponyrt crt_objects -- $(build_flags)

The change is adding crt_objects to the --target list. Since crt_objects only exists when CMAKE_SYSTEM_NAME is Linux (which cross-libponyrt always sets), this is safe.

Step 5: Ship CRT objects in the install layout

File: Makefile

In the install target (after line 327, following the existing libponyrt-pic copy), add:

$(SILENT)if [ -f $(outDir)/crtbeginS.o ]; then cp $(outDir)/crtbeginS.o $(ponydir)/lib/$(arch); fi
$(SILENT)if [ -f $(outDir)/crtendS.o ]; then cp $(outDir)/crtendS.o $(ponydir)/lib/$(arch); fi
$(SILENT)if [ -f $(outDir)/crtbeginT.o ]; then cp $(outDir)/crtbeginT.o $(ponydir)/lib/$(arch); fi
$(SILENT)if [ -f $(outDir)/crtend.o ]; then cp $(outDir)/crtend.o $(ponydir)/lib/$(arch); fi

The if [ -f ... ] guard matches the existing pattern for optional files (libponyrt-pic, libponyc, etc.) and ensures non-Linux installs don't fail.

After this, the installed directory layout on Linux is:

lib/<arch>/
    libponyrt.a
    libponyrt-pic.a     (existing)
    crtbeginS.o         (new)
    crtendS.o           (new)
    crtbeginT.o         (new)
    crtend.o            (new)
    libponyc.a          (optional, existing)
    libponyc-standalone.a (optional, existing)

Step 6: Convert genexe.c to C++

Rename: src/libponyc/codegen/genexe.c → src/libponyc/codegen/genexe.cc

The existing PONY_EXTERN_C_BEGIN/PONY_EXTERN_C_END guards in genexe.h ensure proper C linkage for genexe() and gen_main(). Callers in C files (e.g., codegen.c) continue to work — they see C-linkage declarations via the header, and the C++ implementation emits C-linkage symbols.

This is the same pattern already used by genopt.cc, gendebug.cc, and host.cc in the same directory.

On MSVC, all .c files are already compiled as C++ (lines 116-117 of src/libponyc/CMakeLists.txt), so the code is already C++-compatible.

Add LLD header includes at the top of genexe.cc:

#include <lld/Common/Driver.h>

// Declare LLD drivers we'll use in later phases.
// These are declarations only — no LLD code is called in Phase 1.
LLD_HAS_DRIVER(elf)
LLD_HAS_DRIVER(macho)
LLD_HAS_DRIVER(coff)
LLD_HAS_DRIVER(wasm)

The LLD_HAS_DRIVER macro expands to a namespace lld { namespace elf { bool link(...); } } forward declaration. No symbols are referenced, so no LLD object files are pulled into the binary.

Open question for Sean: A reviewer suggested deferring the #include <lld/Common/Driver.h> and LLD_HAS_DRIVER macros to Phase 2, arguing they're unused code. My reasoning for including them now: they validate that LLD headers are correctly installed and includable as part of the build. Without them, a header installation problem wouldn't surface until Phase 2. The tradeoff is unused includes in production code for one phase.

File: src/libponyc/CMakeLists.txt

Change line 28 from codegen/genexe.c to codegen/genexe.cc.

Step 7: Convert program.c to C++

Rename: src/libponyc/pkg/program.c → src/libponyc/pkg/program.cc

Same approach as Step 6. program.h already has PONY_EXTERN_C_BEGIN/PONY_EXTERN_C_END.

File: src/libponyc/CMakeLists.txt

Change line 75 from pkg/program.c to pkg/program.cc.

Step 8: Add embedded library args API

The existing program_lib_build_args() builds a flat string with shell-specific formatting (-Wl, prefixes, "..." quoting, -l/.lib affixes). Embedded LLD needs structured data: separate path and library lists without formatting.

The raw data already exists in program_t (data->libpaths and data->libs), but paths from opt->package_search_paths (CLI -p and PONYPATH) are only merged during program_lib_build_args. Also, stored values are wrapped in "..." by quoted_locator().

File: src/libponyc/pkg/program.h

Add inside the PONY_EXTERN_C_BEGIN/PONY_EXTERN_C_END block:

/** Build library arguments for embedded LLD linking.
 * Collects all library paths (source-defined and CLI/PONYPATH) and library
 * names into structured arrays without shell-specific formatting.
 * Mutually exclusive with program_lib_build_args() — call one or the other.
 */
void program_lib_build_args_embedded(ast_t* program, pass_opt_t* opt);

/** Return the number of library search paths.
 * Only valid after program_lib_build_args_embedded(). */
size_t program_lib_path_count(ast_t* program);

/** Return a library search path by index (unquoted, no -L prefix). */
const char* program_lib_path_at(ast_t* program, size_t index);

/** Return the number of user-specified libraries.
 * Only valid after program_lib_build_args_embedded(). */
size_t program_lib_count(ast_t* program);

/** Return a library name by index (unquoted, no -l prefix or .lib suffix). */
const char* program_lib_at(ast_t* program, size_t index);

File: src/libponyc/pkg/program.cc

Add fields to program_t:

typedef struct program_t
{
  // ... existing fields ...

  // Embedded LLD support (set by program_lib_build_args_embedded)
  const char** embedded_paths;
  size_t embedded_path_count;
  const char** embedded_libs;
  size_t embedded_lib_count;
} program_t;

Initialize the new fields in program_create() (all to NULL/0).

Implement program_lib_build_args_embedded():

1. Assert data->lib_args == NULL AND data->embedded_paths == NULL (ensures neither build function has been called yet)
2. Count paths from data->libpaths + opt->package_search_paths
3. Allocate embedded_paths array, populate with unquoted path strings
4. Count libs from data->libs
5. Allocate embedded_libs array, populate with unquoted library name strings

Mutual exclusion between the two build functions relies on embedded_paths != NULL (set by the embedded path) and lib_args != NULL (set by the legacy path). Each function asserts both are NULL on entry:

• program_lib_build_args_embedded(): assert data->lib_args == NULL && data->embedded_paths == NULL
• program_lib_build_args(): add pony_assert(data->embedded_paths == NULL) at line 191, alongside the existing pony_assert(data->lib_args == NULL)

No sentinel value is needed — the arrays themselves serve as the indicator. program_free() is unaffected because it already guards lib_args with a NULL check, and the new arrays get their own NULL-guarded free calls.

"Unquoting" strips the "..." wrapper added by quoted_locator(): skip the first byte, copy up to (but not including) the last byte, null-terminate. Use stringtab for the result so callers don't need to manage memory.

Implement the four accessor functions as simple index-into-array operations with bounds assertions.

Update program_free() to free the embedded_paths and embedded_libs arrays (allocated via ponyint_pool_alloc_size).

Serialization: program_t has program_serialise_trace, program_serialise, and program_deserialise functions. The new embedded fields are transient — they're only populated during the linking phase, never during compilation when serialization occurs. They will always be NULL/0 at serialization time. Nevertheless, update the serialization functions:

• program_serialise: write the new fields as NULL/0 (they're already zero, but be explicit)
• program_deserialise: initialize the new fields to NULL/0
• program_serialise_trace: no action needed for NULL pointers

Design Decisions

CRT build: direct compilation vs. compiler-rt cmake

Decision: Compile crtbegin.c and crtend.c directly, bypassing compiler-rt's cmake.

Rationale: compiler-rt's cmake infrastructure is complex — it requires CompilerRTUtils, BuiltinTests, filter_available_targets, and extensive platform detection. We need exactly two small C files. Direct compilation with explicit flags is simpler, more maintainable, and produces identical output. The source files are stable LLVM project code from the vendored submodule.

LLD drivers to link

Decision: List all LLD drivers in PONYC_LLD_LIBS on all platforms.

Rationale: Since no LLD symbols are referenced in Phase 1, the static linker won't pull in any LLD code — binary size increase is zero. When later phases reference specific drivers, only those drivers' code is pulled in. Listing all drivers now avoids revisiting the CMakeLists in each subsequent phase.

CRT platforms

Decision: Only build CRT objects on Linux (guarded by CMAKE_SYSTEM_NAME STREQUAL "Linux").

Rationale: macOS doesn't use CRT objects. Windows uses MSVC CRT. BSD CRT is deferred to Phase 6. Linux is the primary target for Phases 2-3.

compiler-rt licensing

Non-issue: compiler-rt is under the Apache 2.0 license with LLVM Exceptions — the same license as the already-vendored LLVM. No new license concerns.

Testing

Build verification

make libs build_flags=-j8          # Verify LLD builds alongside LLVM
make configure                     # Verify find_package(LLD) succeeds
make build                         # Verify ponyc links with LLD libraries

Regression testing

make test-core                     # libponyrt tests, libponyc tests, full-program tests
make test-stdlib-release           # Full stdlib test suite

All tests must pass with zero changes to test code — this phase doesn't alter behavior.

CRT object verification

# Verify CRT objects exist in output directory
ls build/release/crt{begin,end}*.o

# Verify they're valid ELF relocatable objects
file build/release/crtbeginS.o
# Expected: ELF 64-bit LSB relocatable, x86-64, ...

# Verify key symbols
nm build/release/crtbeginS.o | grep __dso_handle
nm build/release/crtbeginS.o | grep __do_init
nm build/release/crtendS.o | grep __EH_FRAME_LIST_END__

Cross-compilation verification

# Verify cross-compiled CRT objects are produced
make cross-libponyrt arch=rv64gc CC=riscv64-linux-gnu-gcc-10 config=debug cross_cflags="-march=rv64gc -mabi=lp64d"
file build/rv64gc/debug/crtbeginS.o
# Expected: ELF 64-bit LSB relocatable, UCB RISC-V, ...

Install verification

make install prefix=/tmp/ponyc-test
ls /tmp/ponyc-test/lib/pony/*/lib/native/crt*.o

Binary size measurement

# Before this change
ls -la build/release/ponyc | awk '{print $5}'

# After this change (should be same or negligibly larger)
ls -la build/release/ponyc | awk '{print $5}'

Files Modified

File	Change
lib/CMakeLists.txt	Add LLVM_ENABLE_PROJECTS "lld"
CMakeLists.txt	Add find_package(LLD), define PONYC_LLD_LIBS, add src/crt subdirectory
src/ponyc/CMakeLists.txt	Link ${PONYC_LLD_LIBS}
test/libponyc/CMakeLists.txt	Link ${PONYC_LLD_LIBS}
benchmark/libponyc/CMakeLists.txt	Link ${PONYC_LLD_LIBS}
src/libponyc/CMakeLists.txt	Rename source files (.c → .cc), update standalone archive to include LLD
src/crt/CMakeLists.txt	New — CRT object compilation
src/libponyc/codegen/genexe.c → .cc	Rename, add LLD header includes and driver declarations
src/libponyc/pkg/program.c → .cc	Rename, add embedded args API implementation, update serialization
src/libponyc/pkg/program.h	Add embedded args function declarations
Makefile	Add CRT objects to install target, update cross-libponyrt to build CRT

[SeanTAllen]

Implementation PR: #4945