Add text format specification for Linking.md

[feedab1e]

This PR proposes a vendor-neutral syntax for describing relocation information in WAT.

Unlike the syntax described in WebAssembly/wabt#2649, this proposal is intended to express everything that the binary format can, including whatever the current proposed implementation in WABT does not support. Of note here is the fact that @sym annotations can appear multiple times per declaration, the inclusion of COMDATs, segment infos, and the inclusion of labels and addends.

[sbc100]

Seems like a good idea to me. Good to see the annotation proposal being used in innovative ways like this!

[sbc100]

If we think retain is a better terminology then we should probably propose renaming WASM_SYM_NO_STRIP rather than diverging.

[feedab1e]

Yeah, that's sound. I just thought that those names are from LLVM's source, so that rename should be done in sync with LLVM

[sbc100]

For comparison I wonder if you could post a simple hello_world.o object file in both .s assembly format and the .wat format. We should probably almost mention that LLVM uses its own .s format somewhere here too, but that can be separate PR if you like.

[feedab1e]

For comparison I wonder if you could post a simple hello_world.o object file in both .s assembly format and the .wat format. We should probably almost mention that LLVM uses its own .s format somewhere here too, but that can be separate PR if you like.

I can do that for WAT, but since I don't really know the structure of .s, and there seems to be no documentation for it, I don't think that I can do it with .s.

[feedab1e]

On second thought, I can just use clang -S for .s and clang -c wasm2wat for .wat, so I can do that, yeah

[feedab1e]

@sbc100 My "hello world" examples turned out to be more than a full screen of code when using .s. I can send the files here, but I don't think it would be good to put that output into the document.

[sbc100]

@sbc100 My "hello world" examples turned out to be more than a full screen of code when using .s. I can send the files here, but I don't think it would be good to put that output into the document.

Can you just include it here as quoted text? Along with the equivalent wat for comparison?

[sbc100]

@sbc100 My "hello world" examples turned out to be more than a full screen of code when using .s. I can send the files here, but I don't think it would be good to put that output into the document.

Agreed, Im mostly curious to see the two side by side here in the comments for comparison/discussion.

[feedab1e]

Oh, sure, then, for the source file test.cpp, when compiled with clang -S -O3, yields test.s, and when compiled clang -c -O3, and than run through wasm2wat -r --enable-annotations (provided wabt from my fork), yields test.wat.

(Unfortunately, github forbids me from posting .s and .wat, so .txt it is)

[sbc100]

Oh, sure, then, for the source file test.cpp, when compiled with clang -S -O3, yields test.s, and when compiled clang -c -O3, and than run through wasm2wat -r --enable-annotations (provided wabt from my fork), yields test.wat.

(Unfortunately, github forbids me from posting .s and .wat, so .txt it is)

Oh nice, the wat format looks much more readable than I expected it to be.

[feedab1e]

One thing I wanted to mention here is that there is a limitation in the text format, that code section relocations that do not make sense for instruction operands may not be expressed in the text format. So for example i32.load instruction with a R_WASM_TYPE_INDEX_LEB could occur in a perfectly valid object file, from binary format's PoV, yet would be inexpressible in the text format. Do we want to disallow such relocations in the binary format too?
(AFAIK this would also be a problem for .s)

[feedab1e]

Also the same issue exists for labels, which can only point between instructions in the code segment, or into the data area of a data segment in the data section

[sbc100]

One thing I wanted to mention here is that there is a limitation in the text format, that code section relocations that do not make sense for instruction operands may not be expressed in the text format. So for example i32.load instruction with a R_WASM_TYPE_INDEX_LEB could occur in a perfectly valid object file, from binary format's PoV, yet would be inexpressible in the text format. Do we want to disallow such relocations in the binary format too? (AFAIK this would also be a problem for .s)

Yes it seems perfectly reasonable for an object file validator to declare such relocations as invalid based on the instructions they are part of.

However, the linker itself (wasm-ld) will blindly accept such things, and likely produce invalid output as a result too. The linker explictly does not parse the code section but instead blindly applies relocation. The same goes for relocation that don't point to a correct spot in the instruction stream, e.g. a relocation could in theory point at the i32.load opcode itself, rather than its operand. This would technically be an invalid object files, but the linker is blind to that.

[feedab1e]

Would you be fine with me adding into the doc that such relocations are invalid for the purposes of validation, then?
And same for R_WASM_SECTION_OFFSET_I32 addends.
Also, do we want to specify that section entries in relocation tables may only point to the data section, the code section, or custom sections?

[sbc100]

Would you be fine with me adding into the doc that such relocations are invalid for the purposes of validation, then?

Sure that makes sense to me. Relocation that don't point to a valid spot in the instruction stream are certainly invalid. It might be worth also noting that wasm-ld does not do validation of the code section though, so bad inputs can result in bad outputs.

And same for R_WASM_SECTION_OFFSET_I32 addends. Also, do we want to specify that section entries in relocation tables may only point to the data section, the code section, or custom sections?

Sounds reasonable yes, we could always expand the list, but for now those are the only sections that are copied by the linker from input files into the output file, so they are the only section for which relocations make sense. I think for GC types we maybe want to one day include the type section somehow, but we are long way from that.

[feedab1e]

Added the validation rules, please take a look

[sbc100]

Hm.. I'm not sure about this actually.

When you asked about documenting a limitation on sections I thought you were referring to the fact that relocations can only apply to certain section types.

"Which sections can have relocations within them" is a different concept to "which sections can be referred to by WASM_SYMBOL_TYPE_SECTION symbols".

I believe that WASM_SYMBOL_TYPE_SECTION symbols are only used by debug info, but my memory is a little foggy here.

Looking at the code it actually looks like this symbols might only be valid for custom sections: https://github.com/llvm/llvm-project/blob/38372df53fd7f6c8bd8c46bf720b676e12f481d9/lld/wasm/InputFiles.cpp#L697-L705.

Which would make sense if these only used in debug info since all debug info is stored in custom sections.

[feedab1e]

I don't believe it can work like that, though, since R_WASM_SECTION_OFFSET_I32 relocations reference a section symbol, and for that to work as DWARF code addresses, the symbol that relocation references would have to reference the CODE section, while the relocation itself would have to target a place in the debug section.

[feedab1e]

No, actually, that can absolutely work if WASM_*_OFFSET_* relocations actually resolve to offsets form the file start, like DWARF actually expects. I do think the current spec is not very clear on this and someone from LLVM should take a look at what actually happens there and adjust https://github.com/WebAssembly/tool-conventions/blob/main/Linking.md#processing-relocations accordingly.

[feedab1e]

Updated the text format to reflect that section offset relocations may only reference custom sections for now.

[feedab1e]

Thanks for this! I'd second the idea of including examples directly in this document if you're up for it. Would it be possible to show the C/C++ source you're working with plus the raw text output? Maybe not the entire file in all cases but it'd be convenient to see example usage of all of these dierctives within the context of actual *.wat.

I think a simple hello world in C should be enough since that will include a function relocation for printf and a data relocation for the hello world string. No need to do anything more fancy in the basic example.

As per current primary symbol rules, the relocation annotation for printf would actually be entirely elided, so for a simple example I'd actually suggest the more complex code I posted above. IMO it is a good showcase for symbols, relocations in code section, data symbols, data imports, and relocations in data sections.

[sbc100]

As per current primary symbol rules, the relocation annotation for printf would actually be entirely elided

I see so there would still be a reloc entry in the binary but not the the test format.

When relocation entries are implicit like this how does the binary writer know to produce them? Would there be special flag to the wat2wasm program that says something like --generate-object-file / --add-implicit-relocs ? Or would it be somehow automatic? i.e. what would stop the wat2wasm program generating relocs in normal non-object-file cases?

[feedab1e]

When relocation entries are implicit like this how does the binary writer know to produce them? Would there be special flag to the wat2wasm program that says something like --generate-object-file / --add-implicit-relocs ? Or would it be somehow automatic? i.e. what would stop the wat2wasm program generating relocs in normal non-object-file cases?

Yes, that flag already exists in wat2wasm trunk, it's -r, or --relocatable

[feedab1e]

The caveat here is that currently wat2wasm can only produce implicit relocs. The plan for this proposal is to mostly preserve old -r behaviour while additionally allowing explicit relocs where they are actually required.

[sbc100]

When relocation entries are implicit like this how does the binary writer know to produce them? Would there be special flag to the wat2wasm program that says something like --generate-object-file / --add-implicit-relocs ? Or would it be somehow automatic? i.e. what would stop the wat2wasm program generating relocs in normal non-object-file cases?

Yes, that flag already exists in wat2wasm trunk, it's -r, or --relocatable

Do you think this a good design (its something I just threw together back in the day)?

I wonder if "explicit relocs everywhere" might not be better? Otherwise won't all the tools that do text to binary conversion will need some kind of extra flag to see if implicit relocs are enabled or not?

Or can we magically enable implicit relocs whenever we see any kind linking annotation in the wat file? Could there exist a wat file with zero explict annotations? How would the tooling know to create a linking section or not in that case?

[feedab1e]

When relocation entries are implicit like this how does the binary writer know to produce them? Would there be special flag to the wat2wasm program that says something like --generate-object-file / --add-implicit-relocs ? Or would it be somehow automatic? i.e. what would stop the wat2wasm program generating relocs in normal non-object-file cases?

Yes, that flag already exists in wat2wasm trunk, it's -r, or --relocatable

Do you think this a good design (its something I just threw together back in the day)?

Well, it does make the code prettier, and it does allow to link against all existing WAT modules with no source changes, I see that as a benefit.

I wonder if "explicit relocs everywhere" might not be better? Otherwise won't all the tools that do text to binary conversion will need some kind of extra flag to see if implicit relocs are enabled or not?

Or can we magically enable implicit relocs whenever we see any kind linking annotation in the wat file? Could there exist a wat file with zero explict annotations? How would the tooling know to create a linking section or not in that case?

Well, if we really do want that, I suppose we could dispatch based on whether the features section is present. But that wouldn't be reliable in any case, since tooling that isn't relocation-aware would skip unknown annotations and silently yield a simple module instead of an object file.

[alexcrichton]

I wonder if "explicit relocs everywhere" might not be better? Otherwise won't all the tools that do text to binary conversion will need some kind of extra flag to see if implicit relocs are enabled or not?

I may not fully be following the context here, but my hope is that wasm-tools print wouldn't need any sort of special flags to print @reloc annotations from a *.o file. Similarly I wouldn't want to need to pass extra flags to wasm-tools parse to generate reloc.* and linking sections when generating a *.o file.

[feedab1e]

I wonder if "explicit relocs everywhere" might not be better? Otherwise won't all the tools that do text to binary conversion will need some kind of extra flag to see if implicit relocs are enabled or not?

I may not fully be following the context here, but my hope is that wasm-tools print wouldn't need any sort of special flags to print @reloc annotations from a *.o file. Similarly I wouldn't want to need to pass extra flags to wasm-tools parse to generate reloc.* and linking sections when generating a *.o file.

So, @sbc100 is basically asking if it would be possible at all to assemble a file form WAT without emitting relocation metadata. For disassembly it'd be easy, since it's always obvious if a file is an object file based on the presence of the relevant custom sections. But for assembly with current syntax, any valid text module can also be an object file now.

[sbc100]

I wonder if "explicit relocs everywhere" might not be better? Otherwise won't all the tools that do text to binary conversion will need some kind of extra flag to see if implicit relocs are enabled or not?

I may not fully be following the context here, but my hope is that wasm-tools print wouldn't need any sort of special flags to print @reloc annotations from a *.o file. Similarly I wouldn't want to need to pass extra flags to wasm-tools parse to generate reloc.* and linking sections when generating a *.o file.

So, @sbc100 is basically asking if it would be possible at all to assemble a file form WAT without emitting relocation metadata. For disassembly it'd be easy, since it's always obvious if a file is an object file based on the presence of the relevant custom sections. But for assembly with current syntax, any valid text module can also be an object file now.

I suppose we could make a rule such as "If at least one explicit linking annotation exists in the wat file then the whole file is assumed to be relocatable, and implicit relocations will be injected/generated in all possible locations"

Should we go with "at least on explicit annotation" or should we maybe have some kind of top level annotation that expresses "this is a relocatable object file"?

[sbc100]

Something like (@linkable) at the top level? Or (@relocatable)?

[feedab1e]

Looking at prior art, if I am to run wat2wasm --enable-code-metadata --enable-annotations on a file with no code metadata annotations, do the sections still appear?

[sbc100]

Looking at prior art, if I am to run wat2wasm --enable-code-metadata --enable-annotations on a file with no code metadata annotations, do the sections still appear?

I think they would appear in the binary if and only if they appear in the source (wat).

The idea of creating implicit annotations (i.e. annotation that don't exist in the wat file at all) is the tricky thing here. I'm not sure its good idea to that route.

[feedab1e]

Something like (@linkable) at the top level? Or (@relocatable)?

Well, I can make a description for target-features, require that to be present for the binary to be relocatable

[alexcrichton]

Personally I would advocate for linking and reloc.* sections are only generated if the input has @sym or @reloc by default. I'd prefer to add more flags to wasm-tools parse-the-CLI (the text-to-binary conversion) as that would also proliferate to all API users of the programmatic functionality as well. To me it also feels less magic to have annotations-per-item as opposed to generating them automatically

[feedab1e]

Personally I would advocate for linking and reloc.* sections are only generated if the input has @sym or @reloc by default. I'd prefer to add more flags to wasm-tools parse-the-CLI (the text-to-binary conversion) as that would also proliferate to all API users of the programmatic functionality as well. To me it also feels less magic to have annotations-per-item as opposed to generating them automatically

I would assume, then, that a reasonable implementation of that would always generate the sections internally, but then strip them during output, unless either a flag or either @reloc or @sym is specified, right? If so, that seems fine by me.

[alexcrichton]

My assumption is that the presence of @reloc and @sym causes stuff to be generated internally, but otherwise nothing happens. For example call $foo wouldn't generate any relocations by default (I realize this is against the -r flag y'all seem to be describing with wabt). So I was naively thinking that nothing would need stripping, only things would need to be preserved if they were present.

[feedab1e]

By "default" do you mean "if no linking annotations are present in the file" (i.e. not creating an object file) or do you mean just silently not generating a relocation there?

[feedab1e]

I guess my actual question is how would you diagnose the following:

(module
  (func (param) (result))
  (func (@sym) (param) (result)))

It is either

1. "missing (@sym)" on the first declaration (all WebAssembly entities are required to have at least one symbol)
2. "(@sym) is not allowed" on the second declaration (upon seeing the first declaration, we decided that we aren't actually processing an object file)

Option 1 would require either restarting the parse, or remembering all of the declaration locations, and then issuing an error for every one that had no annotations.

[sbc100]

I guess my actual question is how would you diagnose the following:

(module
  (func (param) (result))
  (func (@sym) (param) (result)))

It is either

1. "missing (@sym)" on the first declaration (all WebAssembly entities are required to have at least one symbol)
2. "(@sym) is not allowed" on the second declaration (upon seeing the first declaration, we decided that we aren't actually processing an object file)

Option 1 would require either restarting the parse, or remembering all of the declaration locations, and then issuing an error for every one that had no annotations.

How about we require @relocatable on the module itself?

[alexcrichton]

Could handling func-with-@sym plus func-without-@sym be part of the validation of the linking section? Where validation requries that if any entity has @sym then they all have @sym? And then dealing with the text format of invalid modules is expected to return an error of some kind?

[feedab1e]

Could handling func-with-@sym plus func-without-@sym be part of the validation of the linking section? Where validation requries that if any entity has @sym then they all have @sym? And then dealing with the text format of invalid modules is expected to return an error of some kind?

Sure, but how would you implement those diagnostics? The parser state is gone by the time you do object file validation, as well as source locations for the declarations.

[feedab1e]

I guess my actual question is how would you diagnose the following:

(module
  (func (param) (result))
  (func (@sym) (param) (result)))

It is either

1. "missing (@sym)" on the first declaration (all WebAssembly entities are required to have at least one symbol)
2. "(@sym) is not allowed" on the second declaration (upon seeing the first declaration, we decided that we aren't actually processing an object file)

Option 1 would require either restarting the parse, or remembering all of the declaration locations, and then issuing an error for every one that had no annotations.

How about we require @relocatable on the module itself?

That would work, too. Actually, then, I think it would be better to have an annotation describing all WAT features that are enabled for the module.
Something like:

(module (@wat-features linking code-metadata dwarf))

For object file linking specifically, another option is to say something like

(module (@target-features +overlong-leb -multimemory))

to trigger object file generation, since those make sense only for linking, AFAIK

[feedab1e]

@sbc100 @alexcrichton Hi, do you have any more comments? if not, I’d like to merge this soon, since my GCC codegen backend can already compile programs, but it needs spec to be reviewable

[sbc100]

Don't we still need to resolve the issue of how wat reader knows if it should produce a relocatable output at all?

My current preference is for a global annotation on the top level module, but I don't see that reflected in the current PR.

I'm also still not sure about the implicit relocation for things like call $foo .. does that currently generate an implicit relocation? I still think it might be good to remove all implicit relocation for v1 of the spec, although the addition of the the level annotation on the module object makes me feel a little better about it.

I'm curious if other folks have opinions about implicit relocations. @dschuff maybe? @sunfishcode @tlively ? I guess it kind of matches assembly formats where call foo is an implicit relocation.

[sbc100]

Should we Wasm consistently? So Wasm entity instead of `WebAssembly entity?

[feedab1e]

Yeah, I can do that

[sbc100]

Same here? Should we just use Wasm?

[sbc100]

Is there any reason not to reflect the entire list of relocation types like they are listed in the binary format and/or in llvm: https://github.com/llvm/llvm-project/blob/main/llvm/include/llvm/BinaryFormat/WasmRelocs.def

i.e. why create this new concept of a base type + a modifier that doesn't exist elsewhere yet? Why not just use type=R_WASM_EVENT_INDEX_XX in the text format? This would also make the format redundant since its also part of the name of the relocation type.

Maybe this new method/format/modifier concept could be added more globally later once the initial version of the text format is added? But for v1 it seems like it would make sense to simply mirror the existing binary format enum.

[feedab1e]

This was covered extensively in #258 (comment), and @alexcrichton expressed support for it here, but in short, that way there wouldn't be an option to elide parts of the relocation annotation (i.e. defaulting and predefinig wouldn't work), so all relocations would be incredibly verbose (for example, call $foo would become call $foo (@reloc type=R_WASM_FUNC_INDEX_LEB) for no reason).

[sbc100]

I think I don't see how specifying the full relocation type (e.g using R_WASM_FUNC_INDEX_LEB when a reloc is present) would prevent the whole relocation from being implicit / elided.

This seem like two orthogonal decisions, but I get that I must be missing something:

1. Do we implicitly generate reloc entries for things like call instruction?
2. When reloc is specified explicitly do we use the existing enum, of something new/different

I'm also not sure that reducing verbosity needs to be the highest priority since the plan is for this format to be mostly machine read and machine written, right?

[feedab1e]

I think I don't see how specifying the full relocation type (e.g using R_WASM_FUNC_INDEX_LEB when a reloc is present) would prevent the whole relocation from being implicit / elided.

This seem like two orthogonal decisions, but I get that I must be missing something:

1. Do we implicitly generate reloc entries for things like call instruction?
2. When reloc is specified explicitly do we use the existing enum, of something new/different

Apart form fully elidable relocations, other types of relocations exist, like in memory (i32.load (@reloc $mem_sym)) and in constants (i32.const (@reloc data $mem_sym)) where a relocation is not entirely elided but is greatly abbreviated form the complete relocation type. Apart form that, specifying the complete relocation type would expose relocation type names (which are for now an implementation detail of LLVM) to the wider text format.

I'm also not sure that reducing verbosity needs to be the highest priority since the plan is for this format to be mostly machine read and machine written, right?

Well, it needs to be human-readable, too, since it's a text format and humans are expected to read that too, like they usually read assembly, and likewise human-writable.

[sbc100]

Apart form that, specifying the complete relocation type would expose relocation type names (which are for now an implementation detail of LLVM) to the wider text format.

The relocation type names are not indented to be LLVM specific. The list of 20 relocation types, along with their ffull names, are listed above in this very document.

This is designed to mirror the ELF relocation types that are defined in the ELF header and not specific to either LLVM or GCC but are using in both place.

I think it might be a good idea to reflect this precisely in text for, so we can avoid having two different ways to specify things.

[feedab1e]

Apart form that, specifying the complete relocation type would expose relocation type names (which are for now an implementation detail of LLVM) to the wider text format.

The relocation type names are not indented to be LLVM specific. The list of 20 relocation types, along with their ffull names, are listed above in this very document.

This is designed to mirror the ELF relocation types that are defined in the ELF header and not specific to either LLVM or GCC but are using in both place.

Sure, but those names currently still don't matter much and can be changed, if we require those names there they suddenly start to matter and can no longer be changed.

I think it might be a good idea to reflect this precisely in text for, so we can avoid having two different ways to specify things.

These aren't really two ways yet, since the names in relocation types don't really matter currently, so the only "stable" way for naming relocations would be in the text format. However, stabilizing those names as fused now would be harmful, since then for v2 when I am to reintroduce the composite names, this "two ways to specify the same thing" argument would become very real, dooming the format to verbose relocations forever.

[sbc100]

Personally I don't think (@reloc ...) or (@sym ...) should be elidable nor should there be a top-level annotation if each relocation/symbol is individually annotated. I think it'd be ok for (@reloc $foo) to automatically infer the kind of relocation depending on context, but I'd personally prefer that if something else were inserted that it's the full LLVM-derived name to avoid ambiguity. Like @sbc100 I'm not personally too worried about verbosity here. The assembly for println!("hi!") is way more verbose than the line of code, and IMO that's just kind of how assembly goes.

I'm OK with eliding relocations, if and only if we have a top level annotation.

I'd also be OK without a top level annotation if we can make all relocs explicit.

@alexcrichton, can I ask, what is your objection to the top level annotation? Don't you think it would be nice to be able to, at a glace, distinguish relocation wat files without having to visually scan the whole wat file a @sym or @Reloc?

[alexcrichton]

To me it's more of a preference of per-relocation/symbol annotations over a top-level annotation. I agree that either system would work alright, and the main concern that I can think of is printing an wasm object file (binary-to-text) where it feels more natural to print @reloc or @sym-per-entry as opposed to verifying that everything has a relocation or symbol and then not printing anything. For a pure text-to-binary use case I'd agree that the top-level annotation is nicer to have.

I'd naively expect that with @sym and @reloc would be frequent enough in a file that it wouldn't take much of a visual scan myself, but I'm not personally too concerned about that myself.

I should also be clear that I'm happy to be overruled here. IMO text-format design is something that's worth bikeshedding but not endlessly, so I wouldn't want to hold up anything on my own behalf too much

[feedab1e]

I’m still not sure there’s much value in either making all symbols/relocs explicit or having a toplevel annotation, since neither actually guarantee that an object file is produced. So the only effect of such an annotation would be to artificially restrict files that can be relocatable. Currently, any valid WASM module as well as any valid WAT module can be transformed into an object file and back by merely attaching or stripping the linking information. It's a nice property and I would like to keep it, especially since there is nothing stopping us from it.

If we really really want something (advisory) that says which features are intended to be used with which WAT files, then perhaps we should have a general annotation that applies to all features, not just linking.

That annotation, if decided upon, could in principle augment assembler flags same way as -DMACRO/#define MACRO works in C and C++ compilers

[feedab1e]

There's also a separate concern of how explicit relocation annotations will fit in with other metadata that isn't aware of linking, like code metadata. Currently it's easy, code metadata info needs a funcidx, so it grabs the primary symbol for that funcidx, and places a relocation of that symbol, no annotations required.

If explicit relocations are mandated, then the standardized syntax for code metadata would simply no longer work, just like every annotation-based metadata feature that isn't explicitly aware of relocations.

[feedab1e]

I'm also still not sure about the implicit relocation for things like call $foo .. does that currently generate an implicit relocation? I still think it might be good to remove all implicit relocation for v1 of the spec, although the addition of the the level annotation on the module object makes me feel a little better about it.

As I explained here, I do think that making all relocations explicit would be a very bad idea, since that would just make the text format excessively cluttered for no gain at all, since those relocations are mandatory and their properties can be trivially inferred.

[sunfishcode]

As I explained here, I do think that making all relocations explicit would be a very bad idea, since that would just make the text format excessively cluttered for no gain at all, since those relocations are mandatory and their properties can be trivially inferred.

This sounds reasonable to me.