xanadu-next: Game crashes with d3d8to9 and nine

[orbea]

OS: Slackware64-current
wine-nine-standalone: iXit/wine-nine-standalone@b41e162
mesa: https://github.com/mesa3d/mesa/commit/21c795ab0723cf0595768ea3f2165c10e89fd8c1
wine: 4.11
d3d8to9: 1.9.2
GPU: RX Vega 56

I am trying to run the game Xanadu Next.

https://appdb.winehq.org/objectManager.php?sClass=version&iId=26343

This game uses d3d8, but its very broken graphically with wined3d and has been for a long time.

https://bugs.winehq.org/show_bug.cgi?id=40643

When using d3d8to9 and native versions of both d3dx9_43 and d3dcompiler_43 it actually seems to work with limited testing. However when nine is enabled the game will immediately crash.

Wine crash log: xanadu_backtrace.txt

I made an apitrace of the game with d3d8to9 without nine and replaying it with wined3d works, but it will also crash when replayed with nine.

Wine apitrace crash log: apitrace_xanadu_backtrace.txt
trace: xanadu-next_d3d8to9.trace.xz.txt

Note that this trace only contains the title screen which has always worked with wined3d unlike the actual gameplay.

Additionally when nir is enabled, wine itself will crash with this very unhelpful backtrace.

Thread 7 "XANADU.exe:sh2" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 23709]
0x77cba5e4 in ?? ()
(gdb) bt
#0  0x77cba5e4 in ?? ()
Backtrace stopped: Cannot access memory at address 0x5

[orbea]

I have determined this issue occurs when mesa is compiled with clang-8.0.0, but does not occur with gcc-9.1.0. I have built a full debug build of mesa with no optimizations and still could reproduce this issue.

[orbea]

It appears the wine segfault with nir is not the same issue, it still occurs even when mesa is compiled with gcc.

[orbea]

Just to be clear games I have found to break with mesa + clang + nine so far includes Trails in the Sky, Xanadu Next and Grim Dawn. At least some games like A Hat in Time are not affected.

So far the only game to break with d3d8to9 + nir + nine is Xanadu Next.

[orbea]

I compiled the current mesa master (https://github.com/mesa3d/mesa/commit/451f030c068ccfa87aaab624d8e392875fd46c08) with clang-8.0.1 while explicitly adding -Wall -Wextra -Weverything and here is the very long log...

mesa.clang.log.xz.txt

[orbea]

This issue is caused by a combination of -O1 or higher and -march=native. It can also be reproduced with -march=bdver2 and -march=amdfam10, but doesn't occur with -march=athlon-xp or -march=i686.

I'll narrow this down more later and update this issue then.

[orbea]

Additionally when nir is enabled, wine itself will crash with this very unhelpful backtrace.

This issue with nir is now fixed in mesa 19.2 and wine 4.15, but I haven't further looked at the original issue with clang optimizations yet.