Support for emitting data that allows reconstructing original stack trace frames (including inlined frames)

[mkustermann]

The current support for source maps does not allow decoding stack traces in a way that preserves inlined frames. This is problematic as a stack trace from production may not make too much sense if important frames are missing.

At the minimum one would want to have a way to decode production stacks offline with a tool.
Better would be if this also worked nicely in the browser (especially for a world where compile-to-js languages start using compile-to-wasm-gc and may want to do that not only for production but also development use case)

There's a few options:

• Make use of the fact that source maps is a json format which allows emitting extra (i.e. non-specified) information in it (this is the approach of dart2js).
• Rely on DWARF instead of source maps. Currently binaryen disables some optimizations when dwarf mode is enabled. It's unclear if these optimizations have to be disabled if we only care about correct inlined frames in dwarf or they are disabled due to supporting other dwarf features.
  => See also Support for all optimization passes with DWARF output? #4814
• Have a separate, generic mechanism that allows attaching metadata to things in the wasm file (instructions, globals, ...) and ensure the wasm2wasm optimizer preserves this information.

/cc @biggs0125 @osa1

[kripken]

cc @aheejin @dschuff

[dschuff]

Just to link up some issues, there is related discussion in emscripten-core/emscripten#20462 and emscripten-core/emscripten#25116. In the latter I speculated that we might be able to do some partial reconstruction in the absence of this information (and according to this issue, Sentry is doing something similar) but I agree that we should do better if we can because good symbolization for fully optimized production binaries is an important use case.

In addition to the Dart extension for source maps, I also know of something similar from Bloomberg (that page has links to previous work in the area as well).
There is also a proposal at TC39 called Scopes that looks like it's actually fairly far along in the standards process, so maybe it's worth taking a closer look at that.

It might also be reasonable to rely on DWARF. IIRC (@kripken can correct me if I'm wrong) Binaryen reconstructs DWARF line tables from scratch rather than rewriting them; I don't think it does the same for inlining records but maybe it could. It seems like it could be feasible to have a mode where we allow full optimizations with DWARF and don't try to preserve e.g. variable information but do create/update inlining records. Unclear whether that would be more work in Binaryen than implementing a new source map proposal, although it does seem like it would be more work for producers other than LLVM that likely support source maps but not DWARF. Edit: I forgot I filed emscripten-core/emscripten#25239 about this in Emscripten.

Option 3 could potentially be used in combination with 1 or 2. It seems we'd likely need to represent this source/inlining information in Binaryen IR somehow, and maybe we'd want a way to encode that in the text or binary format just for our own testing purposes, in addition to or instead of a JSON or DWARF encoding.

[kripken]

I hadn't heard of Scopes - that sounds promising! It looks like it has what we want. And while we don't necessarily need this to work in devtools (it can be processed on the commandline, for reconstructing original stack traces from the wild) but it would still be very nice if it were, so it's great that it is on the standards track.

In addition to the external format, we need Binaryen IR changes as you said @dschuff . Perhaps something like a vector of scopes for each debug location, and for the inlining pass to update it. That would take some work, but not a huge amount I hope.