[WebAssembly] Generate a call to __wasm_apply_global_tls_relocs in __wasm_init_memory

[Arshia001]

Motivation

We recently implemented the WebAssembly exception handling proposal in Wasmer 6.0. As a result, we can now take advantage of clang's support for compiling SjLj and C++ exceptions to WASM EH. This PR fixes a wasm-ld issue that breaks the use of C++ exception handling in WASI(X) modules.

Note: I use WASI(X) to mean either wasi preview 1 or WASIX modules.

Error details

When compiling C++ code that uses exceptions, clang generates a GOT.data.internal.__wasm_lpad_context global, which points to the wasm landing pad context that's shared between compiler code and libunwind. This global is initialized in the __wasm_apply_global_tls_relocs function.

TLS initialization happens in two separate places; for the "main thread", __wasm_init_memory runs as the (start) function of the WASM module, initializing all memory segments (including TLS), while also initializing the main thread's __tls_base to the space reserved for it by the compiler, and signalling this fact to other threads via an atomic. Other threads need to run __wasm_init_tls after getting their respective __tls_base global initialized externally.

As it stands, __wasm_apply_global_tls_relocs is only called through __wasm_init_tls, meaning if code doesn't call __wasm_init_tls, any globals that are initialized in __wasm_apply_global_tls_relocs do not get initialized. This is the case for the main thread.

It is important to note that exception handling code generated by the compiler uses GOT.data.internal.__wasm_lpad_context, while the code in _Unwind_CallPersonality goes through __tls_base + offset directly. Because GOT.data.internal.__wasm_lpad_context is not initialized in the main thread, the compiler and _Unwind_CallPersonality do not agree on where the landing pad context is stored. This results in scan_eh_tab not getting the correct LSDA pointer. Exception handling is then completely broken; the catch-all block runs for every exception due to a lack of any type information at runtime.

This PR allows a call to __wasm_apply_global_tls_relocs to be generated in __wasm_init_memory if needed, which should fix the value of GOT.data.internal.__wasm_lpad_context in modules' main threads. Interestingly, through all of our recent work on dynamic linking and PIC modules, we never encountered __wasm_apply_global_tls_relocs, and I don't know if it's used for anything besides GOT.data.internal.__wasm_lpad_context.

But how does emscripten work if this is broken?

Good question! Emscripten calls __wasm_init_tls redundantly for main threads, and thus initializes the TLS area twice. This has no observable effect besides being slower, and does indeed fix C++ exception handling.

This is a workaround that we can use in WASIX as well. However, as far as I understand, the current behavior is wasm-ld is broken, since __wasm_init_memory and __wasm_init_tls should behave similarly with respect to TLS initialization, but feel free to disagree with me here.

[github-actions]

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

[llvmbot]

@llvm/pr-subscribers-lld

@llvm/pr-subscribers-lld-wasm

Author: None (Arshia001)

Changes

Motivation

We recently implemented the WebAssembly exception handling proposal in Wasmer 6.0. As a result, we can now take advantage of clang's support for compiling SjLj and C++ exceptions to WASM EH. This PR fixes a wasm-ld issue that breaks the use of C++ exception handling in WASI(X) modules.

Note: I use WASI(X) to mean either wasi preview 1 or WASIX modules.

Error details

When compiling C++ code that uses exceptions, clang generates a GOT.data.internal.__wasm_lpad_context global, which points to the wasm landing pad context that's shared between compiler code and libunwind. This global is initialized in the __wasm_apply_global_tls_relocs function.

TLS initialization happens in two separate places; for the "main thread", __wasm_init_memory runs as the (start) function of the WASM module, initializing all memory segments (including TLS), while also initializing the main thread's __tls_base to the space reserved for it by the compiler, and signalling this fact to other threads via an atomic. Other threads need to run __wasm_init_tls after getting their respective __tls_base global initialized externally.

As it stands, __wasm_apply_global_tls_relocs is only called through __wasm_init_tls, meaning if code doesn't call __wasm_init_tls, any globals that are initialized in __wasm_apply_global_tls_relocs do not get initialized. This is the case for the main thread.

It is important to note that exception handling code generated by the compiler uses GOT.data.internal.__wasm_lpad_context, while the code in _Unwind_CallPersonality goes through __tls_base + offset directly. Because GOT.data.internal.__wasm_lpad_context is not initialized in the main thread, the compiler and _Unwind_CallPersonality do not agree on where the landing pad context is stored. This results in scan_eh_tab not getting the correct LSDA pointer. Exception handling is then completely broken; the catch-all block runs for every exception due to a lack of any type information at runtime.

This PR allows a call to __wasm_apply_global_tls_relocs to be generated in __wasm_init_memory if needed, which should fix the value of GOT.data.internal.__wasm_lpad_context in modules' main threads. Interestingly, through all of our recent work on dynamic linking and PIC modules, we never encountered __wasm_apply_global_tls_relocs, and I don't know if it's used for anything besides GOT.data.internal.__wasm_lpad_context.

But how does emscripten work if this is broken?

Good question! Emscripten calls __wasm_init_tls redundantly for main threads, and thus initializes the TLS area twice. This has no observable effect besides being slower, and does indeed fix C++ exception handling.

This is a workaround that we can use in WASIX as well. However, as far as I understand, the current behavior is wasm-ld is broken, since __wasm_init_memory and __wasm_init_tls should behave similarly with respect to TLS initialization, but feel free to disagree with me here.

---

Full diff: https://github.com/llvm/llvm-project/pull/149832.diff

1 Files Affected:

• (modified) lld/wasm/Writer.cpp (+9)

diff --git a/lld/wasm/Writer.cpp b/lld/wasm/Writer.cpp
index b704677d36c93..3cd6a73fb1a31 100644
--- a/lld/wasm/Writer.cpp
+++ b/lld/wasm/Writer.cpp
@@ -1366,6 +1366,15 @@ void Writer::createInitMemoryFunction() {
           writeUleb128(os, s->index, "segment index immediate");
           writeU8(os, 0, "memory index immediate");
         }
+
+        // After initializing the TLS segment, we also need to apply TLS
+        // relocations in the same way __wasm_init_tls does.
+        if (ctx.arg.sharedMemory && s->isTLS() &&
+            ctx.sym.applyGlobalTLSRelocs) {
+          writeU8(os, WASM_OPCODE_CALL, "CALL");
+          writeUleb128(os, ctx.sym.applyGlobalTLSRelocs->getFunctionIndex(),
+                      "function index");
+        }
       }
     }
 

[sbc100]

According to the comment for createApplyGlobalTLSRelocationsFunction it cannot be called during the start function: (

llvm-project/lld/wasm/Writer.cpp

Lines 1520 to 1523 in 6fa759b

	// Similar to createApplyGlobalRelocationsFunction but for
	// TLS symbols. This cannot be run during the start function
	// but must be delayed until __wasm_init_tls is called.
	void Writer::createApplyGlobalTLSRelocationsFunction() {

.

I don't remember exactly why this is...

[sbc100]

ctx.arg.sharedMemory is probably redundant here since without it applyGlobalTLSRelocs would never be created.

[Arshia001]

Fixed.

[Arshia001]

@sbc100 thanks for the review!

it cannot be called during the start function:

I can't think of anything, except the fact that it needs __tls_base to be initialized before it can work. Maybe, at some point, __tls_base wasn't initialized in __wasm_init_memory, and nobody updated that comment after this behavior changed? I'll dig into the commit history to see if I can figure this out.

In the meantime, do you have other suggestions on how to fix this? I suppose making __wasm_apply_global_tls_relocs public would at least allow it to be called from the WASIX module's _start function.

[Arshia001]

This is the change that introduced the comment: https://github.com/llvm/llvm-project/blob/ef8c9135efcb3847fc0e5bbdb55eae18751090df/lld/wasm/Writer.cpp

Looking over the code, it seems that back then, __tls_base wasn't being initialized during __wasm_memory_init:

llvm-project/lld/wasm/Writer.cpp

Lines 1116 to 1138 in ef8c913

	writeU8(os, WASM_OPCODE_END, "end $init");
	for (const OutputSegment *s : segments) {
	if (needsPassiveInitialization(s)) {
	// destination address
	writePtrConst(os, s->startVA, is64, "destination address");
	if (config->isPic) {
	writeU8(os, WASM_OPCODE_GLOBAL_GET, "GLOBAL_GET");
	writeUleb128(os, WasmSym::memoryBase->getGlobalIndex(),
	"memory_base");
	writeU8(os, is64 ? WASM_OPCODE_I64_ADD : WASM_OPCODE_I32_ADD,
	"i32.add");
	}
	// source segment offset
	writeI32Const(os, 0, "segment offset");
	// memory region size
	writeI32Const(os, s->size, "memory region size");
	// memory.init instruction
	writeU8(os, WASM_OPCODE_MISC_PREFIX, "bulk-memory prefix");
	writeUleb128(os, WASM_OPCODE_MEMORY_INIT, "memory.init");
	writeUleb128(os, s->index, "segment index immediate");
	writeU8(os, 0, "memory index immediate");
	}
	}

Around a year later, static allocation of the TLS section was added in:

llvm-project/lld/wasm/Writer.cpp

Lines 1190 to 1208 in 74f9841

	// When we initialize the TLS segment we also set the `__tls_base`
	// global. This allows the runtime to use this static copy of the
	// TLS data for the first/main thread.
	if (config->sharedMemory && s->isTLS()) {
	if (config->isPic) {
	// Cache the result of the addionion in local 0
	writeU8(os, WASM_OPCODE_LOCAL_TEE, "local.tee");
	writeUleb128(os, 1, "local 1");
	} else {
	writePtrConst(os, s->startVA, is64, "destination address");
	}
	writeU8(os, WASM_OPCODE_GLOBAL_SET, "GLOBAL_SET");
	writeUleb128(os, WasmSym::tlsBase->getGlobalIndex(),
	"__tls_base");
	if (config->isPic) {
	writeU8(os, WASM_OPCODE_LOCAL_GET, "local.tee");
	writeUleb128(os, 1, "local 1");
	}
	}

But __wasm_apply_global_tls_relocs probably flew under the radar and the comment was never removed. I assume the correct thing to do here would be to remove that comment as well. What do you think, @sbc100?