Provide Updated Dawn Prebuilt

[a-day-old-bagel]

I'm on the warpath trying to compile some new prebuilt Dawn binaries to link to so that zgpu can use the newer WebGPU features (like subgroups, in my case).

I'm making an issue here about it in order to make it a matter of record, and have a place to refer to when this kind of discussion comes up.

There has been an effort made before to include the building of Dawn from source as a zgpu standard feature

My attempts to merely upgrade the prebuilt binaries are only a shadow of that ambition, but maybe if I can figure out at least this much, I can use the scars from the experience to work on a goal such as that one afterward.

Currently, I'm trying to fork and fix the pipeline that Mach Engine used to have for doing all this automatically. If I can just get that pipeline to work again, even once, I can maybe provide the results as updated prebuilts. Then maybe it will be worth thinking about integrating the functional parts of that pipeline into zgpu so that it can do it all itself.

Anyway, that's my thinking.

[a-day-old-bagel]

As a progress note, I'll mention that what I'm currently working with (and all my failed iterations) can be found here:

Fork of Mach's mach-gpu-dawn: https://github.com/a-day-old-bagel/necromach-gpu-dawn
Fork of Dawn used by the above: https://github.com/a-day-old-bagel/necromach-dawn

And I may make some more random "necromach" repos as needed to reproduce (or iterate upon) the kinds of builds Mach used to do in their CI.

EDIT: If I get something actually robust working, it might be worth asking if we should be putting any of the relevant repos under the organization instead, but that's all hypothetical atm.

EDIT2: Context: My thinking is headed along these lines:
Given that the Mach maintainer abandoned the use of Dawn due to the huge maintenance burden of their build pipeline, I'm trying to avoid a lot of that by piggybacking the zig build off of the cmake build scripts, rather than having the one huge build.zig manually stay in sync with all the generated source files. It may work, or it may just force me back to the Mach solution eventually, we'll see. But if it works, it might prove a better route toward a low-maintenance automatic build pipeline that can be used from within the zgpu build script on demand (that would be the dream).

[a-day-old-bagel]

I've gotten my own project to compile, using zgpu, with a new version of Dawn. I consider this a success, but there's still a lot more work to be done. zgpu.zig and wgpu.zig need major refactors for all the changes made to Dawn over the past couple of years. Stuff like there being no explicit SwapChain object anymore, and adapter/device requests being all async/await with Futures now, apparently necessarily.

I've began some of this refactoring work, but I'm going to have to test it against all the existing code in zig-gamedev that uses zgpu. Mostly I think I can do a lot of that testing, but I don't have access to Apple hardware, and I also don't know exactly how people have been using zgpu through emscripten.

A lot of the updates that were made by #8 are things that I haven't fully grokked, and I'd like to know how those changes were being tested so that I can replicate those same tests to validate any refactor work.

I may need to call on the expertise of those of you who have built projects using emscripten or Apple platforms.

[a-day-old-bagel]

Previously there has been an include directory in the zgpu source tree. For my recent updated builds, I had actually completely deleted that and instead fetched https://github.com/webgpu-native/webgpu-headers in the build.zig.zon.

I had hoped that doing this would simplify maintenance going forward, but now I'm finding that webgpu-headers might not have enough coverage. It doesn't have all of the feature enums and stuff filled out, and doesn't expose some of the emscripten objects.

On the other hand, according to the webgpu/dawn docs, I don't think you're supposed to include and use webgpu.h if you're building for emscripten, and instead use the APIs provided by emscripten headers or something, which claim to include all the necessary stuff for webgpu. I'm not certain on that point. But I wanted to make a note here about my using webgpu-headers (and potentially needing to revert back to having includes in the zgpu source tree), so that I (or others) can remember to revisit the topic.

[a-day-old-bagel]

Previously, zgpu has relied on declaring extern fn ... everywhere in order for the zig code to link to the C calls that are assumed to be exposed by whatever prebuilt we have.

I see that some other zig bindings for webgpu implementations do this also, and I recognize it as a useful pattern, but I'd like to change it as part of these refactors. I'd like to properly import a header, like const c = @cImport(@cInclude("webgpu.h"));, more like what we do in zphysics, so that the calls made from wgpu.zig, for example, can actually be compile-time checked against whatever API version we're trying to use.

When using simply extern fn, the way I typically find out about a mismatch is by looking through a stack trace when the program segfaults at runtime. That might just be on me for not going about this the right way, but from my perspective, having wgpu.zig actually call into a proper header makes it more maintainable going forward. It actually gives you compiler output, etc. Hopefully that makes sense to anyone reading.

Anyway, I've been including that change in all the other changes I'm bundling into this snowballing monster of a changelist.

[a-day-old-bagel]

Given many long attempts to statically compile Dawn and DXC into a package for use on Windows builds targeting DX12, I might tentatively suggest that if we want to keep zgpu at all up-to-date with WebGPU, we might consider either requiring that windows builds targeting the DX12 backend must ship with a couple DLL's (the way microsoft intended), or requiring that the Vulkan backend is always the one Dawn uses on Windows and just not support DX12 at all. But I don't know how many people might be put off by either of those ideas, so I'm just throwing that out there.

If zig-gamedev does make those concessions, I think I'm able to get zgpu fully working to that point with current WebGPU/Dawn builds and do so in a rather more maintainable way than we had before, optimisitcally.

Creating static or shared libs which include current builds of Dawn, minus the DXC component, and keeping wgpu.zig and zgpu.zig in sync with those builds, is something I have figured out how to do, and may be enough, to put it another way.

[hazeycode]

Nice work. We already vendor DX12 Agilty SDK in zwindows. We can pull this in lazily when building for windows. Would that cover it or are there other dependencies?