Skip to content

Unbundle static libraries with device code#5

Merged
pvelesko merged 1 commit intochipStar-llvm-17from
rdcLink
Apr 19, 2025
Merged

Unbundle static libraries with device code#5
pvelesko merged 1 commit intochipStar-llvm-17from
rdcLink

Conversation

@pvelesko
Copy link
Copy Markdown

No description provided.

This fixed the issue of rdc linking static libraries with device code
@pvelesko pvelesko changed the base branch from chipStar-llvm-18 to chipStar-llvm-17 April 14, 2025 15:22
@colleeneb
Copy link
Copy Markdown

Thanks! This worked for the test case I had.

I see HIPAMD.cpp does this via a routine AddStaticDeviceLibs. Is that too specific to AMDGCN to reuse here?

@pvelesko
Copy link
Copy Markdown
Author

pvelesko commented Apr 15, 2025 via email

Copy link
Copy Markdown

@colleeneb colleeneb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nothing is blocked urgently by this right now (that I know of). The code that resulted in the original issue can use shared library builds to get around it. But understood, this will at least get static libs with device to work.

@pvelesko pvelesko merged commit 2b2831c into chipStar-llvm-17 Apr 19, 2025
2 checks passed
@pvelesko pvelesko deleted the rdcLink branch April 19, 2025 07:07
pvelesko pushed a commit that referenced this pull request May 5, 2025
… collection (llvm#136795)

Fix a [test
failure](llvm#136236 (comment))
in llvm#136236, apply a minor renaming of statistics, and remerge. See
details below.

# Changes in llvm#136236

Currently, `DebuggerStats::ReportStatistics()` calls
`Module::GetSymtab(/*can_create=*/false)`, but then the latter calls
`SymbolFile::GetSymtab()`. This will load symbols if haven't yet. See
stacktrace below.

The problem is that `DebuggerStats::ReportStatistics` should be
read-only. This is especially important because it reports stats for
symtab parsing/indexing time, which could be affected by the reporting
itself if it's not read-only.

This patch fixes this problem by adding an optional parameter
`SymbolFile::GetSymtab(bool can_create = true)` and receiving the
`false` value passed down from `Module::GetSymtab(/*can_create=*/false)`
when the call is initiated from `DebuggerStats::ReportStatistics()`.

---

Notes about the following stacktrace:
1. This can be reproduced. Create a helloworld program on **macOS** with
dSYM, add `settings set target.preload-symbols false` to `~/.lldbinit`,
do `lldb a.out`, then `statistics dump`.
2. `ObjectFile::GetSymtab` has `llvm::call_once`. So the fact that it
called into `ObjectFileMachO::ParseSymtab` means that the symbol table
is actually being parsed.

```
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = step over
    frame #0: 0x0000000124c4d5a0 LLDB`ObjectFileMachO::ParseSymtab(this=0x0000000111504e40, symtab=0x0000600000a05e00) at ObjectFileMachO.cpp:2259:44
  * frame #1: 0x0000000124fc50a0 LLDB`lldb_private::ObjectFile::GetSymtab()::$_0::operator()(this=0x000000016d35c858) const at ObjectFile.cpp:761:9
    frame #5: 0x0000000124fc4e68 LLDB`void std::__1::__call_once_proxy[abi:v160006]<std::__1::tuple<lldb_private::ObjectFile::GetSymtab()::$_0&&>>(__vp=0x000000016d35c7f0) at mutex:652:5
    frame llvm#6: 0x0000000198afb99c libc++.1.dylib`std::__1::__call_once(unsigned long volatile&, void*, void (*)(void*)) + 196
    frame llvm#7: 0x0000000124fc4dd0 LLDB`void std::__1::call_once[abi:v160006]<lldb_private::ObjectFile::GetSymtab()::$_0>(__flag=0x0000600003920080, __func=0x000000016d35c858) at mutex:670:9
    frame llvm#8: 0x0000000124fc3cb0 LLDB`void llvm::call_once<lldb_private::ObjectFile::GetSymtab()::$_0>(flag=0x0000600003920080, F=0x000000016d35c858) at Threading.h:88:5
    frame llvm#9: 0x0000000124fc2bc4 LLDB`lldb_private::ObjectFile::GetSymtab(this=0x0000000111504e40) at ObjectFile.cpp:755:5
    frame llvm#10: 0x0000000124fe0a28 LLDB`lldb_private::SymbolFileCommon::GetSymtab(this=0x0000000104865200) at SymbolFile.cpp:158:39
    frame llvm#11: 0x0000000124d8fedc LLDB`lldb_private::Module::GetSymtab(this=0x00000001113041a8, can_create=false) at Module.cpp:1027:21
    frame llvm#12: 0x0000000125125bdc LLDB`lldb_private::DebuggerStats::ReportStatistics(debugger=0x000000014284d400, target=0x0000000115808200, options=0x000000014195d6d1) at Statistics.cpp:329:30
    frame llvm#13: 0x0000000125672978 LLDB`CommandObjectStatsDump::DoExecute(this=0x000000014195d540, command=0x000000016d35d820, result=0x000000016d35e150) at CommandObjectStats.cpp:144:18
    frame llvm#14: 0x0000000124f29b40 LLDB`lldb_private::CommandObjectParsed::Execute(this=0x000000014195d540, args_string="", result=0x000000016d35e150) at CommandObject.cpp:832:9
    frame llvm#15: 0x0000000124efbd70 LLDB`lldb_private::CommandInterpreter::HandleCommand(this=0x0000000141b22f30, command_line="statistics dump", lazy_add_to_history=eLazyBoolCalculate, result=0x000000016d35e150, force_repeat_command=false) at CommandInterpreter.cpp:2134:14
    frame llvm#16: 0x0000000124f007f4 LLDB`lldb_private::CommandInterpreter::IOHandlerInputComplete(this=0x0000000141b22f30, io_handler=0x00000001419b2aa8, line="statistics dump") at CommandInterpreter.cpp:3251:3
    frame llvm#17: 0x0000000124d7b5ec LLDB`lldb_private::IOHandlerEditline::Run(this=0x00000001419b2aa8) at IOHandler.cpp:588:22
    frame llvm#18: 0x0000000124d1e8fc LLDB`lldb_private::Debugger::RunIOHandlers(this=0x000000014284d400) at Debugger.cpp:1225:16
    frame llvm#19: 0x0000000124f01f74 LLDB`lldb_private::CommandInterpreter::RunCommandInterpreter(this=0x0000000141b22f30, options=0x000000016d35e63c) at CommandInterpreter.cpp:3543:16
    frame llvm#20: 0x0000000122840294 LLDB`lldb::SBDebugger::RunCommandInterpreter(this=0x000000016d35ebd8, auto_handle_events=true, spawn_thread=false) at SBDebugger.cpp:1212:42
    frame llvm#21: 0x0000000102aa6d28 lldb`Driver::MainLoop(this=0x000000016d35ebb8) at Driver.cpp:621:18
    frame llvm#22: 0x0000000102aa75b0 lldb`main(argc=1, argv=0x000000016d35f548) at Driver.cpp:829:26
    frame llvm#23: 0x0000000198858274 dyld`start + 2840
```

# Changes in this PR top of the above

Fix a [test
failure](llvm#136236 (comment))
in `TestStats.py`. The original version of the added test checks that
all modules have symbol count zero when `target.preload-symbols ==
false`. The test failed on macOS. Due to various reasons, on macOS,
symbols can be loaded for dylibs even with that setting, but not for the
main module. For now, the fix of the test is to limit the assertion to
only the main module. The test now passes on macOS. In the future, when
we have a way to control a specific list of plug-ins to be loaded, there
may be a configuration that this test can use to assert that all modules
have symbol count zero.

Apply a minor renaming of statistics, per the
[suggestion](llvm#136226 (comment))
in llvm#136226 after merge.
pvelesko pushed a commit that referenced this pull request May 5, 2025
…mbolConjured" (llvm#137304)

Reverts llvm#128251

ASAN bots reported some errors:
https://lab.llvm.org/buildbot/#/builders/55/builds/10398
Reverting for investigation.

```
Failed Tests (6):
  Clang :: Analysis/loop-widening-ignore-static-methods.cpp
  Clang :: Analysis/loop-widening-notes.cpp
  Clang :: Analysis/loop-widening-preserve-reference-type.cpp
  Clang :: Analysis/loop-widening.c
  Clang :: Analysis/loop-widening.cpp
  Clang :: Analysis/this-pointer.cpp
Testing Time: 411.55s
Total Discovered Tests: 118563
  Skipped          :     33 (0.03%)
  Unsupported      :   2015 (1.70%)
  Passed           : 116291 (98.08%)
  Expectedly Failed:    218 (0.18%)
  Failed           :      6 (0.01%)
FAILED: CMakeFiles/check-all /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/CMakeFiles/check-all 
cd /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan && /usr/bin/python3 /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/./bin/llvm-lit -sv --param USE_Z3_SOLVER=0 /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/utils/mlgo-utils /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/lld/test /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/mlir/test /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/clang/test /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/utils/lit /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/test
ninja: build stopped: subcommand failed.
```

```
/home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/bin/clang -cc1 -internal-isystem /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/lib/clang/21/include -nostdsysteminc -analyze -analyzer-constraints=range -setup-static-analyzer -analyzer-checker=core,unix.Malloc,debug.ExprInspection -analyzer-max-loop 4 -analyzer-config widen-loops=true -verify -analyzer-config eagerly-assume=false /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/test/Analysis/loop-widening.c # RUN: at line 1
+ /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/bin/clang -cc1 -internal-isystem /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/lib/clang/21/include -nostdsysteminc -analyze -analyzer-constraints=range -setup-static-analyzer -analyzer-checker=core,unix.Malloc,debug.ExprInspection -analyzer-max-loop 4 -analyzer-config widen-loops=true -verify -analyzer-config eagerly-assume=false /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/test/Analysis/loop-widening.c
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/bin/clang -cc1 -internal-isystem /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/lib/clang/21/include -nostdsysteminc -analyze -analyzer-constraints=range -setup-static-analyzer -analyzer-checker=core,unix.Malloc,debug.ExprInspection -analyzer-max-loop 4 -analyzer-config widen-loops=true -verify -analyzer-config eagerly-assume=false /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/test/Analysis/loop-widening.c
1.	<eof> parser at end of file
2.	While analyzing stack: 
	#0 Calling nested_loop_inner_widen
 #0 0x0000c894cca289cc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/lib/Support/Unix/Signals.inc:804:13
 #1 0x0000c894cca23324 llvm::sys::RunSignalHandlers() /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/lib/Support/Signals.cpp:106:18
 #2 0x0000c894cca29bbc SignalHandler(int, siginfo_t*, void*) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3
 #3 0x0000f6898da4a8f8 (linux-vdso.so.1+0x8f8)
 #4 0x0000f6898d377608 (/lib/aarch64-linux-gnu/libc.so.6+0x87608)
 #5 0x0000f6898d32cb3c raise (/lib/aarch64-linux-gnu/libc.so.6+0x3cb3c)
 llvm#6 0x0000f6898d317e00 abort (/lib/aarch64-linux-gnu/libc.so.6+0x27e00)
 llvm#7 0x0000c894c5e77fec __sanitizer::Atexit(void (*)()) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:168:10
 llvm#8 0x0000c894c5e76680 __sanitizer::Die() /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_termination.cpp:52:5
 llvm#9 0x0000c894c5e69650 Unlock /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/../sanitizer_common/sanitizer_mutex.h:250:16
llvm#10 0x0000c894c5e69650 ~GenericScopedLock /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/../sanitizer_common/sanitizer_mutex.h:386:51
llvm#11 0x0000c894c5e69650 __hwasan::ScopedReport::~ScopedReport() /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/hwasan_report.cpp:54:5
llvm#12 0x0000c894c5e68de0 __hwasan::(anonymous namespace)::BaseReport::~BaseReport() /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/hwasan_report.cpp:476:7
llvm#13 0x0000c894c5e66b74 __hwasan::ReportTagMismatch(__sanitizer::StackTrace*, unsigned long, unsigned long, bool, bool, unsigned long*) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/hwasan_report.cpp:1091:1
llvm#14 0x0000c894c5e52cf8 Destroy /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/../sanitizer_common/sanitizer_common.h:532:31
llvm#15 0x0000c894c5e52cf8 ~InternalMmapVector /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/../sanitizer_common/sanitizer_common.h:642:56
llvm#16 0x0000c894c5e52cf8 __hwasan::HandleTagMismatch(__hwasan::AccessInfo, unsigned long, unsigned long, void*, unsigned long*) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/hwasan.cpp:245:1
llvm#17 0x0000c894c5e551c8 __hwasan_tag_mismatch4 /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/hwasan/hwasan.cpp:764:1
llvm#18 0x0000c894c5e6a2f8 __interception::InterceptFunction(char const*, unsigned long*, unsigned long, unsigned long) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/compiler-rt/lib/interception/interception_linux.cpp:60:0
llvm#19 0x0000c894d166f664 getBlock /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/include/clang/StaticAnalyzer/Core/PathSensitive/CoreEngine.h:217:45
llvm#20 0x0000c894d166f664 getCFGElementRef /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/include/clang/StaticAnalyzer/Core/PathSensitive/ExprEngine.h:230:59
llvm#21 0x0000c894d166f664 clang::ento::ExprEngine::processCFGBlockEntrance(clang::BlockEdge const&, clang::ento::NodeBuilderWithSinks&, clang::ento::ExplodedNode*) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Core/ExprEngine.cpp:2570:45
llvm#22 0x0000c894d15f3a1c hasGeneratedNodes /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/include/clang/StaticAnalyzer/Core/PathSensitive/CoreEngine.h:333:37
llvm#23 0x0000c894d15f3a1c clang::ento::CoreEngine::HandleBlockEdge(clang::BlockEdge const&, clang::ento::ExplodedNode*) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:319:20
llvm#24 0x0000c894d15f2c34 clang::ento::CoreEngine::dispatchWorkItem(clang::ento::ExplodedNode*, clang::ProgramPoint, clang::ento::WorkListUnit const&) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:220:7
llvm#25 0x0000c894d15f2398 operator-> /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/libcxx_install_hwasan/include/c++/v1/__memory/unique_ptr.h:267:101
llvm#26 0x0000c894d15f2398 clang::ento::CoreEngine::ExecuteWorkList(clang::LocationContext const*, unsigned int, llvm::IntrusiveRefCntPtr<clang::ento::ProgramState const>)::$_0::operator()(unsigned int) const /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:140:12
llvm#27 0x0000c894d15f14b4 clang::ento::CoreEngine::ExecuteWorkList(clang::LocationContext const*, unsigned int, llvm::IntrusiveRefCntPtr<clang::ento::ProgramState const>) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp:165:7
llvm#28 0x0000c894d0ebb9dc release /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/include/llvm/ADT/IntrusiveRefCntPtr.h:232:9
llvm#29 0x0000c894d0ebb9dc ~IntrusiveRefCntPtr /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/include/llvm/ADT/IntrusiveRefCntPtr.h:196:27
llvm#30 0x0000c894d0ebb9dc ExecuteWorkList /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/include/clang/StaticAnalyzer/Core/PathSensitive/ExprEngine.h:192:5
llvm#31 0x0000c894d0ebb9dc RunPathSensitiveChecks /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:772:7
llvm#32 0x0000c894d0ebb9dc (anonymous namespace)::AnalysisConsumer::HandleCode(clang::Decl*, unsigned int, clang::ento::ExprEngine::InliningModes, llvm::DenseSet<clang::Decl const*, llvm::DenseMapInfo<clang::Decl const*, void>>*) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:741:5
llvm#33 0x0000c894d0eb6ee4 begin /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/include/llvm/ADT/DenseMap.h:0:0
llvm#34 0x0000c894d0eb6ee4 begin /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/include/llvm/ADT/DenseSet.h:187:45
llvm#35 0x0000c894d0eb6ee4 HandleDeclsCallGraph /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:516:29
llvm#36 0x0000c894d0eb6ee4 runAnalysisOnTranslationUnit /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:584:5
llvm#37 0x0000c894d0eb6ee4 (anonymous namespace)::AnalysisConsumer::HandleTranslationUnit(clang::ASTContext&) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp:647:3
llvm#38 0x0000c894d18a7a38 clang::ParseAST(clang::Sema&, bool, bool) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Parse/ParseAST.cpp:0:13
llvm#39 0x0000c894ce81ed70 clang::FrontendAction::Execute() /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1231:10
llvm#40 0x0000c894ce6f2144 getPtr /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/include/llvm/Support/Error.h:278:42
llvm#41 0x0000c894ce6f2144 operator bool /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/llvm/include/llvm/Support/Error.h:241:16
llvm#42 0x0000c894ce6f2144 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:1058:23
llvm#43 0x0000c894cea718cc operator-> /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/libcxx_install_hwasan/include/c++/v1/__memory/shared_ptr.h:635:12
llvm#44 0x0000c894cea718cc getFrontendOpts /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/include/clang/Frontend/CompilerInstance.h:307:12
llvm#45 0x0000c894cea718cc clang::ExecuteCompilerInvocation(clang::CompilerInstance*) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:301:14
llvm#46 0x0000c894c5e9cf28 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/tools/driver/cc1_main.cpp:294:15
llvm#47 0x0000c894c5e92a9c ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/tools/driver/driver.cpp:223:12
llvm#48 0x0000c894c5e902ac clang_main(int, char**, llvm::ToolContext const&) /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/tools/driver/driver.cpp:0:12
llvm#49 0x0000c894c5eb2e34 main /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/clang/tools/driver/clang-driver.cpp:17:3
llvm#50 0x0000f6898d3184c4 (/lib/aarch64-linux-gnu/libc.so.6+0x284c4)
llvm#51 0x0000f6898d318598 __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x28598)
llvm#52 0x0000c894c5e52a30 _start (/home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/bin/clang+0x6512a30)
/home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/tools/clang/test/Analysis/Output/loop-widening.c.script: line 2: 2870204 Aborted                 /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/bin/clang -cc1 -internal-isystem /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm_build_hwasan/lib/clang/21/include -nostdsysteminc -analyze -analyzer-constraints=range -setup-static-analyzer -analyzer-checker=core,unix.Malloc,debug.ExprInspection -analyzer-max-loop 4 -analyzer-config widen-loops=true -verify -analyzer-config eagerly-assume=false /home/b/sanitizer-aarch64-linux-bootstrap-hwasan/build/llvm-project/clang/test/Analysis/loop-widening.c
```
pvelesko pushed a commit that referenced this pull request May 5, 2025
`clang-repl --cuda` was previously crashing with a segmentation fault,
instead of reporting a clean error
```
(base) anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl --cuda
#0 0x0000000111da4fbc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x150fbc)
#1 0x0000000111da31dc llvm::sys::RunSignalHandlers() (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x14f1dc)
#2 0x0000000111da5628 SignalHandler(int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x151628)
#3 0x000000019b242de4 (/usr/lib/system/libsystem_platform.dylib+0x180482de4)
#4 0x0000000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem>, llvm::Error&, std::__1::list<clang::PartialTranslationUnit, std::__1::allocator<clang::PartialTranslationUnit>> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0)
#5 0x0000000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem>, llvm::Error&, std::__1::list<clang::PartialTranslationUnit, std::__1::allocator<clang::PartialTranslationUnit>> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0)
llvm#6 0x0000000107f6bac8 clang::Interpreter::createWithCUDA(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x2173ac8)
llvm#7 0x000000010206f8a8 main (/opt/local/libexec/llvm-20/bin/clang-repl+0x1000038a8)
llvm#8 0x000000019ae8c274 
Segmentation fault: 11
```


The underlying issue was that the `DeviceCompilerInstance` (used for
device-side CUDA compilation) was never initialized with a `Sema`, which
is required before constructing the `IncrementalCUDADeviceParser`.


https://github.com/llvm/llvm-project/blob/89687e6f383b742a3c6542dc673a84d9f82d02de/clang/lib/Interpreter/DeviceOffload.cpp#L32


https://github.com/llvm/llvm-project/blob/89687e6f383b742a3c6542dc673a84d9f82d02de/clang/lib/Interpreter/IncrementalParser.cpp#L31

Unlike the host-side `CompilerInstance` which runs `ExecuteAction`
inside the Interpreter constructor (thereby setting up Sema), the
device-side CI was passed into the parser uninitialized, leading to an
assertion or crash when accessing its internals.

To fix this, I refactored the `Interpreter::create` method to include an
optional `DeviceCI` parameter. If provided, we know we need to take care
of this instance too. Only then do we construct the
`IncrementalCUDADeviceParser`.
pvelesko pushed a commit that referenced this pull request May 5, 2025
llvm#138091)

Check this error for more context
(https://github.com/compiler-research/CppInterOp/actions/runs/14749797085/job/41407625681?pr=491#step:10:531)

This fails with 
```
* thread #1, name = 'CppInterOpTests', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x55500356d6d3)
  * frame #0: 0x00007fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99
    frame #1: 0x00007fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830
    frame #2: 0x00007fffee20917a libclangCppInterOp.so.21.0gitstd::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() + 58
    frame #3: 0x00007fffee224796 libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 838
    frame #4: 0x00007fffee22494d libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 13
    frame #5: 0x00007fffed95ec62 libclangCppInterOp.so.21.0gitclang::IncrementalCUDADeviceParser::~IncrementalCUDADeviceParser() + 98
    frame llvm#6: 0x00007fffed9551b6 libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 102
    frame llvm#7: 0x00007fffed95598d libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 13
    frame llvm#8: 0x00007fffed9181e7 libclangCppInterOp.so.21.0gitcompat::createClangInterpreter(std::vector<char const*, std::allocator<char const*>>&) + 2919
```

Problem : 

1) The destructor currently handles no clearance for the DeviceParser
and the DeviceAct. We currently only have this

https://github.com/llvm/llvm-project/blob/976493822443c52a71ed3c67aaca9a555b20c55d/clang/lib/Interpreter/Interpreter.cpp#L416-L419

2) The ownership for DeviceCI currently is present in
IncrementalCudaDeviceParser. But this should be similar to how the
combination for hostCI, hostAction and hostParser are managed by the
Interpreter. As on master the DeviceAct and DeviceParser are managed by
the Interpreter but not DeviceCI. This is problematic because :
IncrementalParser holds a Sema& which points into the DeviceCI. On
master, DeviceCI is destroyed before the base class ~IncrementalParser()
runs, causing Parser::reset() to access a dangling Sema (and as Sema
holds a reference to Preprocessor which owns PragmaNamespace) we see
this
```
  * frame #0: 0x00007fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99
    frame #1: 0x00007fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830
    
```
pvelesko pushed a commit that referenced this pull request Jul 2, 2025
When running the `openmp` testsuite on 32-bit SPARC, several tests
`FAIL` apparently randomly, but always with the same kind of error:
```
# error: command failed with exit status: -11
```
The tests die with `SIGBUS`, as can be seen in `truss` output:
```
26461/1:            Incurred fault #5, FLTACCESS  %pc = 0x00010EAC
26461/1:              siginfo: SIGBUS BUS_ADRALN addr=0x0013D12C
26461/1:            Received signal llvm#10, SIGBUS [default]
26461/1:              siginfo: SIGBUS BUS_ADRALN addr=0x0013D12C
```
i.e. the code is trying an unaligned access which cannot work on SPARC,
a strict-alignment target which enforces natural alignment on access.
This explains the apparent randomness of the failures: if the memory
happens to be aligned appropriately, the tests work, but fail if not.

A `Debug` build reveals much more:

- `__kmp_alloc` currently aligns to `sizeof(void *)`, which isn't enough
on strict-alignment targets when the data are accessed as types
requiring larger alignment. Therefore, this patch increases `alignment`
to `SizeQuant`.

- 32-bit Solaris/sparc `libc` guarantees 8-byte alignment from `malloc`,
so this patch adjusts `SizeQuant` to match.

- There's a `SIGBUS` in
  ```
  __kmpc_fork_teams (loc=0x112f8, argc=0, 
microtask=0x16cc8
<__omp_offloading_ffbc020a_4b1abe_main_l9_debug__.omp_outlined>)
      at openmp/runtime/src/kmp_csupport.cpp:573
  573	  *(kmp_int64 *)(&this_thr->th.th_teams_size) = 0L;
  ```
Casting to a pointer to a type requiring 64-bit alignment when that
isn't guaranteed is wrong. Instead, this patch uses `memset` instead.

- There's another `SIGBUS` in
  ```
0xfef8cb9c in __kmp_taskloop_recur (loc=0x10cb8, gtid=0, task=0x23cd00,
lb=0x23cd18, ub=0x23cd20, st=1, ub_glob=499, num_tasks=100, grainsize=5,
      extras=0, last_chunk=0, tc=500, num_t_min=20, 
codeptr_ra=0xfef8dbc8 <__kmpc_taskloop(ident_t*, int, kmp_task_t*, int,
kmp_uint64*, kmp_uint64*, kmp_int64, int, int, kmp_uint64, void*)+240>,
      task_dup=0x0)
      at openmp/runtime/src/kmp_tasking.cpp:5147
  5147	  p->st = st;
  ```
`p->st` doesn't currently guarantee the 8-byte alignment required by
`kmp_int64 st`. `p` is set in
  ```
   __taskloop_params_t *p = (__taskloop_params_t *)new_task->shareds;
  ```
but `shareds_offset` is currently aligned to `sizeof(void *)` only.
Increasing it to `sizeof(kmp_uint64)` to match its use fixes the
`SIGBUS`.

With these fixes I get clean `openmp` test results on 32-bit SPARC (both
Solaris and Linux), with one unrelated exception.

Tested on `sparc-sun-solaris2.11`, `sparcv9-sun-solaris2.11`,
`sparc-unknown-linux-gnu`, `sparc64-unknown-linux-gnu`,
`i386-pc-solaris2.11`, `amd64-pc-solaris2.11`, `i686-pc-linux-gnu`, and
`x86_64-pc-linux-gnu`.
pvelesko pushed a commit that referenced this pull request Jul 2, 2025
The function already exposes a work list to avoid deep recursion, this
commit starts utilizing it in a helper that could also lead to a deep
recursion.

We have observed this crash on `clang/test/C/C99/n590.c` with our
internal builds that enable aggressive optimizations and hit the limit
earlier than default release builds of Clang.

See the added test for an example with a deeper recursion that used to
crash in upstream Clang before this change with the following stack
trace:

```
  #0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /usr/local/google/home/ibiryukov/code/llvm-project/llvm/lib/Support/Unix/Signals.inc:804:13
  #1 llvm::sys::RunSignalHandlers() /usr/local/google/home/ibiryukov/code/llvm-project/llvm/lib/Support/Signals.cpp:106:18
  #2 SignalHandler(int, siginfo_t*, void*) /usr/local/google/home/ibiryukov/code/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3
  #3 (/lib/x86_64-linux-gnu/libc.so.6+0x3fdf0)
  #4 AnalyzeImplicitConversions(clang::Sema&, clang::Expr*, clang::SourceLocation, bool) /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12772:0
  #5 CheckCommaOperand /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:0:3
  llvm#6 AnalyzeImplicitConversions /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12644:7
  llvm#7 AnalyzeImplicitConversions(clang::Sema&, clang::Expr*, clang::SourceLocation, bool) /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12776:5
  llvm#8 CheckCommaOperand /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:0:3
  llvm#9 AnalyzeImplicitConversions /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12644:7
 llvm#10 AnalyzeImplicitConversions(clang::Sema&, clang::Expr*, clang::SourceLocation, bool) /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12776:5
 llvm#11 CheckCommaOperand /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:0:3
 llvm#12 AnalyzeImplicitConversions /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12644:7
 llvm#13 AnalyzeImplicitConversions(clang::Sema&, clang::Expr*, clang::SourceLocation, bool) /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12776:5
 llvm#14 CheckCommaOperand /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:0:3
 llvm#15 AnalyzeImplicitConversions /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12644:7
 llvm#16 AnalyzeImplicitConversions(clang::Sema&, clang::Expr*, clang::SourceLocation, bool) /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12776:5
 llvm#17 CheckCommaOperand /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:0:3
 llvm#18 AnalyzeImplicitConversions /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12644:7
 llvm#19 AnalyzeImplicitConversions(clang::Sema&, clang::Expr*, clang::SourceLocation, bool) /usr/local/google/home/ibiryukov/code/llvm-project/clang/lib/Sema/SemaChecking.cpp:12776:5
... 700+ more stack frames.
```
pvelesko pushed a commit that referenced this pull request Oct 31, 2025
`clang-repl --cuda` was previously crashing with a segmentation fault,
instead of reporting a clean error
```
(base) anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl --cuda
#0 0x0000000111da4fbc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x150fbc)
#1 0x0000000111da31dc llvm::sys::RunSignalHandlers() (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x14f1dc)
#2 0x0000000111da5628 SignalHandler(int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x151628)
#3 0x000000019b242de4 (/usr/lib/system/libsystem_platform.dylib+0x180482de4)
#4 0x0000000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem>, llvm::Error&, std::__1::list<clang::PartialTranslationUnit, std::__1::allocator<clang::PartialTranslationUnit>> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0)
#5 0x0000000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem>, llvm::Error&, std::__1::list<clang::PartialTranslationUnit, std::__1::allocator<clang::PartialTranslationUnit>> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0)
llvm#6 0x0000000107f6bac8 clang::Interpreter::createWithCUDA(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x2173ac8)
llvm#7 0x000000010206f8a8 main (/opt/local/libexec/llvm-20/bin/clang-repl+0x1000038a8)
llvm#8 0x000000019ae8c274
Segmentation fault: 11
```

The underlying issue was that the `DeviceCompilerInstance` (used for
device-side CUDA compilation) was never initialized with a `Sema`, which
is required before constructing the `IncrementalCUDADeviceParser`.

https://github.com/llvm/llvm-project/blob/89687e6f383b742a3c6542dc673a84d9f82d02de/clang/lib/Interpreter/DeviceOffload.cpp#L32

https://github.com/llvm/llvm-project/blob/89687e6f383b742a3c6542dc673a84d9f82d02de/clang/lib/Interpreter/IncrementalParser.cpp#L31

Unlike the host-side `CompilerInstance` which runs `ExecuteAction`
inside the Interpreter constructor (thereby setting up Sema), the
device-side CI was passed into the parser uninitialized, leading to an
assertion or crash when accessing its internals.

To fix this, I refactored the `Interpreter::create` method to include an
optional `DeviceCI` parameter. If provided, we know we need to take care
of this instance too. Only then do we construct the
`IncrementalCUDADeviceParser`.

(cherry picked from commit 21fb19f)
pvelesko pushed a commit that referenced this pull request Oct 31, 2025
llvm#138091)

Check this error for more context
(https://github.com/compiler-research/CppInterOp/actions/runs/14749797085/job/41407625681?pr=491#step:10:531)

This fails with
```
* thread #1, name = 'CppInterOpTests', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x55500356d6d3)
  * frame #0: 0x00007fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99
    frame #1: 0x00007fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830
    frame #2: 0x00007fffee20917a libclangCppInterOp.so.21.0gitstd::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() + 58
    frame #3: 0x00007fffee224796 libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 838
    frame #4: 0x00007fffee22494d libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 13
    frame #5: 0x00007fffed95ec62 libclangCppInterOp.so.21.0gitclang::IncrementalCUDADeviceParser::~IncrementalCUDADeviceParser() + 98
    frame llvm#6: 0x00007fffed9551b6 libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 102
    frame llvm#7: 0x00007fffed95598d libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 13
    frame llvm#8: 0x00007fffed9181e7 libclangCppInterOp.so.21.0gitcompat::createClangInterpreter(std::vector<char const*, std::allocator<char const*>>&) + 2919
```

Problem :

1) The destructor currently handles no clearance for the DeviceParser
and the DeviceAct. We currently only have this

https://github.com/llvm/llvm-project/blob/976493822443c52a71ed3c67aaca9a555b20c55d/clang/lib/Interpreter/Interpreter.cpp#L416-L419

2) The ownership for DeviceCI currently is present in
IncrementalCudaDeviceParser. But this should be similar to how the
combination for hostCI, hostAction and hostParser are managed by the
Interpreter. As on master the DeviceAct and DeviceParser are managed by
the Interpreter but not DeviceCI. This is problematic because :
IncrementalParser holds a Sema& which points into the DeviceCI. On
master, DeviceCI is destroyed before the base class ~IncrementalParser()
runs, causing Parser::reset() to access a dangling Sema (and as Sema
holds a reference to Preprocessor which owns PragmaNamespace) we see
this
```
  * frame #0: 0x00007fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99
    frame #1: 0x00007fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830

```

(cherry picked from commit 529b6fc)
pvelesko pushed a commit that referenced this pull request Mar 1, 2026
…er. (llvm#181941)

The progress event reporter has a thread that reports events every 250
millisecond. and is destroyed in its destructor.

When in event reporter desctructor, the event reporter may have pending
event but the call mutex is destroyed leading to the crash.

Relevant stack trace from CI.
```
[2026-02-13T17:46:13.577Z] libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument 
[2026-02-13T17:46:13.577Z] PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash report from ~/Library/Logs/DiagnosticReports/. 

[2026-02-13T17:46:13.577Z]  #0 0x0000000102b6943c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x10008943c) 
[2026-02-13T17:46:13.577Z]  #1 0x0000000102b67368 llvm::sys::RunSignalHandlers() (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x100087368) 
[2026-02-13T17:46:13.577Z]  #2 0x0000000102b69f20 SignalHandler(int, __siginfo*, void*) (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x100089f20)
[2026-02-13T17:46:13.577Z]  #3 0x000000018bbdb744 (/usr/lib/system/libsystem_platform.dylib+0x1804e3744) 
[2026-02-13T17:46:13.577Z]  #4 0x000000018bbd1888 (/usr/lib/system/libsystem_pthread.dylib+0x1804d9888) 
[2026-02-13T17:46:13.577Z]  #5 0x000000018bad6850 (/usr/lib/system/libsystem_c.dylib+0x1803de850) 
[2026-02-13T17:46:13.577Z]  llvm#6 0x000000018bb85858 (/usr/lib/libc++abi.dylib+0x18048d858) 
[2026-02-13T17:46:13.577Z]  llvm#7 0x000000018bb744bc (/usr/lib/libc++abi.dylib+0x18047c4bc) 
[2026-02-13T17:46:13.577Z]  llvm#8 0x000000018b7a0424 (/usr/lib/libobjc.A.dylib+0x1800a8424) 
[2026-02-13T17:46:13.577Z]  llvm#9 0x000000018bb84c2c (/usr/lib/libc++abi.dylib+0x18048cc2c) 
[2026-02-13T17:46:13.577Z] llvm#10 0x000000018bb88394 (/usr/lib/libc++abi.dylib+0x180490394) 
[2026-02-13T17:46:13.577Z] llvm#11 0x000000018bb8833c (/usr/lib/libc++abi.dylib+0x18049033c) 
[2026-02-13T17:46:13.577Z] llvm#12 0x000000018bb01b90 (/usr/lib/libc++.1.dylib+0x180409b90) 
[2026-02-13T17:46:13.577Z] llvm#13 0x000000018bb01b34 (/usr/lib/libc++.1.dylib+0x180409b34) 
[2026-02-13T17:46:13.577Z] llvm#14 0x000000018bb038a0 (/usr/lib/libc++.1.dylib+0x18040b8a0) 
[2026-02-13T17:46:13.577Z] llvm#15 0x0000000102b6fbac lldb_dap::DAP::Send(std::__1::variant<lldb_dap::protocol::Request, lldb_dap::protocol::Response, lldb_dap::protocol::Event> const&) (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x10008fbac) 
[2026-02-13T17:46:13.577Z] llvm#16 0x0000000102b6f890 lldb_dap::DAP::SendJSON(llvm::json::Value const&) (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x10008f890) 
[2026-02-13T17:46:13.577Z] llvm#17 0x0000000102b78788 std::__1::__function::__func<lldb_dap::DAP::DAP(lldb_dap::Log&, lldb_dap::ReplMode, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>, bool, llvm::StringRef, lldb_private::transport::JSONTransport<lldb_dap::ProtocolDescriptor>&, lldb_private::MainLoopPosix&)::$_0, std::__1::allocator<lldb_dap::DAP::DAP(lldb_dap::Log&, lldb_dap::ReplMode, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>, bool, llvm::StringRef, lldb_private::transport::JSONTransport<lldb_dap::ProtocolDescriptor>&, lldb_private::MainLoopPosix&)::$_0>, void (lldb_dap::ProgressEvent&)>::operator()(lldb_dap::ProgressEvent&) (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x100098788) 
[2026-02-13T17:46:13.577Z] llvm#18 0x0000000102b8939c lldb_dap::ProgressEventManager::ReportIfNeeded() (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x1000a939c) 
[2026-02-13T17:46:13.577Z] llvm#19 0x0000000102b8982c lldb_dap::ProgressEventReporter::ReportStartEvents() (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x1000a982c) 
[2026-02-13T17:46:13.577Z] llvm#20 0x0000000102b8a038 void* std::__1::__thread_proxy[abi:nn200100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, lldb_dap::ProgressEventReporter::ProgressEventReporter(std::__1::function<void (lldb_dap::ProgressEvent&)>)::$_0>>(void*) (/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/bin/lldb-dap+0x1000aa038) 
[2026-02-13T17:46:13.577Z] llvm#21 0x000000018bbd1c08 (/usr/lib/system/libsystem_pthread.dylib+0x1804d9c08) [2026-02-13T17:46:13.577Z] llvm#22 0x000000018bbccba8 (/usr/lib/system/libsystem_pthread.dylib+0x1804d4ba8)
```
rdar://170331108
pvelesko pushed a commit that referenced this pull request Mar 1, 2026
I created an issue about this in llvm#179976.

Clang's Address Sanitizer installs its own SEH filter which handles some
types of uncaught exceptions. Along with register values and some other
information, it also generates a stack trace. However, current logic is
incomplete. It relies on DbgHelp's SymFunctionTableAccess64 and
SymGetModuleBase64 which won't work with machine code that has its
RUNTIME_FUNCTION entry registered with Rtl* (e.g. RtlAddFunctionTable)
system calls. Most likely, this is because DbgHelp either relies on
information in PDB files or considers PDATA and XDATA only from loaded
EXE and DLL modules. Either way, consider the following example:

```
#include <windows.h>
#include <iostream>
#include <vector>

typedef union _UNWIND_CODE {
    struct {
        BYTE CodeOffset;
        BYTE UnwindOp : 4;
        BYTE OpInfo : 4;
    };
    USHORT FrameOffset;
} UNWIND_CODE, * PUNWIND_CODE;

typedef struct _UNWIND_INFO {
    BYTE Version : 3;
    BYTE Flags : 5;
    BYTE SizeOfProlog;
    BYTE CountOfCodes;
    BYTE FrameRegister : 4;
    BYTE FrameOffset : 4;
    UNWIND_CODE UnwindCode[1]; // Variable size
} UNWIND_INFO, * PUNWIND_INFO;

#define UWOP_PUSH_NONVOL      0
#define UWOP_ALLOC_LARGE      1
#define UWOP_ALLOC_SMALL      2
#define UWOP_SET_FPREG        3
#define UWOP_SAVE_NONVOL      4
#define UWOP_SAVE_NONVOL_FAR  5
#define UWOP_SAVE_XMM128      8
#define UWOP_SAVE_XMM128_FAR  9
#define UWOP_PUSH_MACHFRAME   10

int main() {
    // PUSH RBX         (0x53)                - Save non-volatile register
    // SUB RSP, 0x20    (0x48 0x83 0xEC 0x20) - Allocate 32 bytes (shadow space)
    // XOR RAX, RAX     (0x48 0x31 0xC0)      - Zero out RAX
    // MOV RAX, [RAX]   (0x48 0x8B 0x00)      - Dereference NULL
    
    std::vector<unsigned char> code = {
        0x53,
        0x48, 0x83, 0xEC, 0x20,
        0x48, 0x31, 0xC0,
        0x48, 0x8B, 0x00
    };

    size_t codeSize = code.size();
    size_t totalSize = 100;

    LPVOID pMemory = VirtualAlloc(NULL, totalSize, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);
    
    BYTE* pCodeBase = (BYTE*)pMemory;
    PUNWIND_INFO pUnwindInfo = (PUNWIND_INFO)(pCodeBase + codeSize);    

    size_t alignmentPadding = 0;
    if ((size_t)pUnwindInfo % 4 != 0) {
        alignmentPadding = 4 - ((size_t)pUnwindInfo % 4);
        pUnwindInfo = (PUNWIND_INFO)((BYTE*)pUnwindInfo + alignmentPadding);
    }

    memcpy(pCodeBase, code.data(), codeSize);

    pUnwindInfo->Version = 1;
    pUnwindInfo->Flags = UNW_FLAG_NHANDLER;
    pUnwindInfo->Flags = 0; 
    pUnwindInfo->SizeOfProlog = 5; 
    pUnwindInfo->CountOfCodes = 2; 
    pUnwindInfo->FrameRegister = 0;
    pUnwindInfo->FrameOffset = 0;

    pUnwindInfo->UnwindCode[0].CodeOffset = 5;
    pUnwindInfo->UnwindCode[0].UnwindOp = UWOP_ALLOC_SMALL;
    pUnwindInfo->UnwindCode[0].OpInfo = 3; 

    pUnwindInfo->UnwindCode[1].CodeOffset = 1;
    pUnwindInfo->UnwindCode[1].UnwindOp = UWOP_PUSH_NONVOL;
    pUnwindInfo->UnwindCode[1].OpInfo = 3; // RBX

    RUNTIME_FUNCTION tableEntry = {};
    tableEntry.BeginAddress = 0;
    tableEntry.EndAddress = (DWORD)codeSize;
    tableEntry.UnwindData = (DWORD)((BYTE*)pUnwindInfo - (BYTE*)pMemory);

    DWORD64 baseAddress = (DWORD64)pMemory;
    RtlAddFunctionTable(&tableEntry, 1, baseAddress);

    typedef void(*FuncType)();
    FuncType myFunc = (FuncType)pMemory;
    myFunc();

    return 0;
}
```

Windows' kernel can propagate hardware exception through that function,
so clearly these entries are at least partially correct. Right now,
ASan's stack walking produces this (compiled with latest release,
clang++):

```
PS D:\Local Projects\cpp-playground> ./a.exe
=================================================================
==14216==ERROR: AddressSanitizer: access-violation on unknown address 0x000000000000 (pc 0x0199561c0008 bp 0x004cf0cffb30 sp 0x004cf0cff970 T0)
==14216==The signal is caused by a READ memory access.
==14216==Hint: address points to the zero page.
    #0 0x0199561c0007  (<unknown module>)
    #1 0x000000000000  (<unknown module>)
    #2 0x000000000000  (<unknown module>)

==14216==Register values:
rax = 0  rbx = 4cf0cffaa0  rcx = 7ffcb97b4e28  rdx = 19955dc0000
rdi = 11bf564a0040  rsi = 0  rbp = 4cf0cffb30  rsp = 4cf0cff970
r8  = 7ffffffffffffffc  r9  = 1  r10 = 0  r11 = 246
r12 = 0  r13 = 0  r14 = 0  r15 = 0
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: access-violation (<unknown module>)
==14216==ABORTING
```

Frames one and two is just some stack space allocated by that dynamic
function. While patched version produces this:

```
PS D:\Local Projects\cpp-playground> ./a.exe
=================================================================
==13660==ERROR: AddressSanitizer: access-violation on unknown address 0x000000000000 (pc 0x01ed5ad70008 bp 0x00d76492f650 sp 0x00d76492f490 T0)
==13660==The signal is caused by a READ memory access.
==13660==Hint: address points to the zero page.
    #0 0x01ed5ad70007  (<unknown module>)
    #1 0x7ff732e518a1 in main (D:\Local Projects\cpp-playground\a.exe+0x1400018a1)
    #2 0x7ff732e56a9b in invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    #3 0x7ff732e56a9b in __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #4 0x7ffcb878e8d6  (C:\WINDOWS\System32\KERNEL32.DLL+0x18002e8d6)
    #5 0x7ffcb966c53b  (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18008c53b)

==13660==Register values:
rax = 0  rbx = d76492f5c0  rcx = 7ffcb97b4e28  rdx = 1ed5a870000
rdi = 12135afa0040  rsi = 0  rbp = d76492f650  rsp = d76492f490
r8  = 7ffffffffffffffc  r9  = 1  r10 = 0  r11 = 246
r12 = 0  r13 = 0  r14 = 0  r15 = 0
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: access-violation (<unknown module>)
==13660==ABORTING
```

Now we see that stack walking handled our dynamic function properly.
Interestingly enough, it appears that other overloaded version of
UnwindSlow procedure that works without CONTEXT structure already has
some logic to handle this. Theoretically, symbolizer should also be able
to provide some information about these functions, but I don't think
that this is necessary.

I added SANITIZER_WINDOWS64 check because I am pretty sure Microsoft
only mentions these functions for 64 bit version of their OS. I also
can't check how this works on ARM.
pvelesko pushed a commit that referenced this pull request Mar 1, 2026
Using code/ideas from the x86 backend to optimize a select on a bitcast
integer. The previous aarch64 approach was to individually extract the
bits from the mask, which is kind of terrible.

https://rust.godbolt.org/z/576sndT66

```llvm
define void @if_then_else8(ptr %out, i8 %mask, ptr %if_true, ptr %if_false) {
start:
  %t = load <8 x i32>, ptr %if_true, align 4
  %f = load <8 x i32>, ptr %if_false, align 4
  %m = bitcast i8 %mask to <8 x i1>
  %s = select <8 x i1> %m, <8 x i32> %t, <8 x i32> %f
  store <8 x i32> %s, ptr %out, align 4
  ret void
}
```

turned into 

```asm
if_then_else8:                          // @if_then_else8
        sub     sp, sp, llvm#16
        ubfx    w8, w1, #4, #1
        and     w11, w1, #0x1
        ubfx    w9, w1, #5, #1
        fmov    s1, w11
        ubfx    w10, w1, #1, #1
        fmov    s0, w8
        ubfx    w8, w1, llvm#6, #1
        ldp     q5, q2, [x3]
        mov     v1.h[1], w10
        ldp     q4, q3, [x2]
        mov     v0.h[1], w9
        ubfx    w9, w1, #2, #1
        mov     v1.h[2], w9
        ubfx    w9, w1, #3, #1
        mov     v0.h[2], w8
        ubfx    w8, w1, llvm#7, #1
        mov     v1.h[3], w9
        mov     v0.h[3], w8
        ushll   v1.4s, v1.4h, #0
        ushll   v0.4s, v0.4h, #0
        shl     v1.4s, v1.4s, llvm#31
        shl     v0.4s, v0.4s, llvm#31
        cmlt    v1.4s, v1.4s, #0
        cmlt    v0.4s, v0.4s, #0
        bsl     v1.16b, v4.16b, v5.16b
        bsl     v0.16b, v3.16b, v2.16b
        stp     q1, q0, [x0]
        add     sp, sp, llvm#16
        ret
```

With this PR that instead emits

```asm
if_then_else8:
   adrp x8, .LCPI0_1
   dup v0.4s, w1
   ldr q1, [x8, :lo12:.LCPI0_1]
   adrp x8, .LCPI0_0
   ldr q2, [x8, :lo12:.LCPI0_0]
   ldp q4, q3, [x2]
   and v1.16b, v0.16b, v1.16b
   and v0.16b, v0.16b, v2.16b
   ldp q5, q2, [x3]
   cmeq v1.4s, v1.4s, #0
   cmeq v0.4s, v0.4s, #0
   bsl v1.16b, v2.16b, v3.16b
   bsl v0.16b, v5.16b, v4.16b
   stp q0, q1, [x0]
   ret
```

So substantially shorter. Instead of building the mask
element-by-element, this approach (by virtue of not splitting) instead
splats the mask value into all vector lanes, performs a bitwise and with
powers of 2, and compares with zero to construct the mask vector.

cc rust-lang/rust#122376
cc llvm#175769
pvelesko pushed a commit that referenced this pull request Mar 12, 2026
`SE.getUMaxExpr` causes assertion failure due to type mismatch here:


https://github.com/llvm/llvm-project/blob/main/llvm/lib/Analysis/LoopAccessAnalysis.cpp#L253

Running `opt -S -p loop-vectorize -debug-only=loop-vectorize
llvm/test/Analysis/LoopAccessAnalysis/type-mismatch-in-scalar-evolution.ll
` without the changes made in LoopAccessAnalysis.cpp causes assertion
failure.
Attaching the stack dump for reference:
```
LV: Checking a loop in 'loop_contains_store_assumed_bounds' from input.ll
LV: Loop hints: force=? width=4 interleave=0
LV: Found a loop: for.body
LV: Found an induction variable.
opt: /home/kshitij/llvm-project/llvm/lib/Analysis/ScalarEvolution.cpp:3918: const llvm::SCEV* llvm::ScalarEvolution::getMinMaxExpr(llvm::SCEVTypes, llvm::SmallVectorImpl<const llvm::SCEV*>&): Assertion `getEffectiveSCEVType(Ops[i]->getType()) == ETy && "Operand types don't match!"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace and instructions to reproduce the bug.
Stack dump:
0.	Program arguments: opt -S -passes=loop-vectorize -debug-only=loop-vectorize -force-vector-width=4 -disable-output input.ll
1.	Running pass "function(loop-vectorize<no-interleave-forced-only;no-vectorize-forced-only;>)" on module "input.ll"
2.	Running pass "loop-vectorize<no-interleave-forced-only;no-vectorize-forced-only;>" on function "loop_contains_store_assumed_bounds"
 #0 0x000058ee97c5e652 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/local/bin/opt+0x4f44652)
 #1 0x000058ee97c5af0f llvm::sys::RunSignalHandlers() (/usr/local/bin/opt+0x4f40f0f)
 #2 0x000058ee97c5b05c SignalHandler(int, siginfo_t*, void*) Signals.cpp:0:0
 #3 0x00007c49d4c45330 (/lib/x86_64-linux-gnu/libc.so.6+0x45330)
 #4 0x00007c49d4c9eb2c __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007c49d4c9eb2c __pthread_kill_internal ./nptl/pthread_kill.c:78:10
 llvm#6 0x00007c49d4c9eb2c pthread_kill ./nptl/pthread_kill.c:89:10
 llvm#7 0x00007c49d4c4527e raise ./signal/../sysdeps/posix/raise.c:27:6
 llvm#8 0x00007c49d4c288ff abort ./stdlib/abort.c:81:7
 llvm#9 0x00007c49d4c2881b _nl_load_domain ./intl/loadmsgcat.c:1177:9
llvm#10 0x00007c49d4c3b517 (/lib/x86_64-linux-gnu/libc.so.6+0x3b517)
llvm#11 0x000058ee98003fdb llvm::ScalarEvolution::getMinMaxExpr(llvm::SCEVTypes, llvm::SmallVectorImpl<llvm::SCEV const*>&) (/usr/local/bin/opt+0x52e9fdb)
llvm#12 0x000058ee98004507 llvm::ScalarEvolution::getUMaxExpr(llvm::SCEV const*, llvm::SCEV const*) (/usr/local/bin/opt+0x52ea507)
llvm#13 0x000058ee980dc728 llvm::getStartAndEndForAccess(llvm::Loop const*, llvm::SCEV const*, llvm::Type*, llvm::SCEV const*, llvm::SCEV const*, llvm::ScalarEvolution*, llvm::DenseMap<std::pair<llvm::SCEV const*, llvm::Type*>, std::pair<llvm::SCEV const*, llvm::SCEV const*>, llvm::DenseMapInfo<std::pair<llvm::SCEV const*, llvm::Type*>, void>, llvm::detail::DenseMapPair<std::pair<llvm::SCEV const*, llvm::Type*>, std::pair<llvm::SCEV const*, llvm::SCEV const*>>>*, llvm::DominatorTree*, llvm::AssumptionCache*, std::optional<llvm::ScalarEvolution::LoopGuards>&) (/usr/local/bin/opt+0x53c2728)
llvm#14 0x000058ee9814008b llvm::isDereferenceableAndAlignedInLoop(llvm::LoadInst*, llvm::Loop*, llvm::ScalarEvolution&, llvm::DominatorTree&, llvm::AssumptionCache*, llvm::SmallVectorImpl<llvm::SCEVPredicate const*>*) (/usr/local/bin/opt+0x542608b)
llvm#15 0x000058ee9a0fa1ca llvm::LoopVectorizationLegality::canUncountableExitConditionLoadBeMoved(llvm::BasicBlock*) (/usr/local/bin/opt+0x73e01ca)
llvm#16 0x000058ee9a0faee0 llvm::LoopVectorizationLegality::isVectorizableEarlyExitLoop() (/usr/local/bin/opt+0x73e0ee0)
llvm#17 0x000058ee9a104678 llvm::LoopVectorizationLegality::canVectorize(bool) (/usr/local/bin/opt+0x73ea678)
llvm#18 0x000058ee9a08c953 llvm::LoopVectorizePass::processLoop(llvm::Loop*) (/usr/local/bin/opt+0x7372953)
llvm#19 0x000058ee9a090e21 llvm::LoopVectorizePass::runImpl(llvm::Function&) (/usr/local/bin/opt+0x7376e21)
llvm#20 0x000058ee9a0914e0 llvm::LoopVectorizePass::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) (/usr/local/bin/opt+0x73774e0)
llvm#21 0x000058ee99e419a5 llvm::detail::PassModel<llvm::Function, llvm::LoopVectorizePass, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) PassBuilderPipelines.cpp:0:0
llvm#22 0x000058ee97f18905 llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) (/usr/local/bin/opt+0x51fe905)
llvm#23 0x000058ee995d70d5 llvm::detail::PassModel<llvm::Function, llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) AMDGPUTargetMachine.cpp:0:0
llvm#24 0x000058ee97f17051 llvm::ModuleToFunctionPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/usr/local/bin/opt+0x51fd051)
llvm#25 0x000058ee995d7775 llvm::detail::PassModel<llvm::Module, llvm::ModuleToFunctionPassAdaptor, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) AMDGPUTargetMachine.cpp:0:0
llvm#26 0x000058ee97f1783d llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (/usr/local/bin/opt+0x51fd83d)
llvm#27 0x000058ee9c153909 llvm::runPassPipeline(llvm::StringRef, llvm::Module&, llvm::TargetMachine*, llvm::TargetLibraryInfoImpl*, llvm::ToolOutputFile*, llvm::ToolOutputFile*, llvm::ToolOutputFile*, llvm::StringRef, llvm::ArrayRef<llvm::PassPlugin>, llvm::ArrayRef<std::function<void (llvm::PassBuilder&)>>, llvm::opt_tool::OutputKind, llvm::opt_tool::VerifierKind, bool, bool, bool, bool, bool, bool, bool, bool) (/usr/local/bin/opt+0x9439909)
llvm#28 0x000058ee97c3f380 optMain (/usr/local/bin/opt+0x4f25380)
llvm#29 0x00007c49d4c2a1ca __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
llvm#30 0x00007c49d4c2a28b call_init ./csu/../csu/libc-start.c:128:20
llvm#31 0x00007c49d4c2a28b __libc_start_main ./csu/../csu/libc-start.c:347:5
llvm#32 0x000058ee97c309a5 _start (/usr/local/bin/opt+0x4f169a5)
```

This is caused by a type mismatch between
`SE.getSCEV(DerefRK.IRArgValue)` and `DerefBytesSCEV`.
Fixing this by extending them to the wider type.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants