OpenCL / Vulkan Interop

[leios]

Ok, let me summarize:

I am quite happy with OpenCL. It gets the performance I want and is straightforward to write; however, it cannot directly render to a screen. So what do I do? I could switch entirely to Vulkan compute shaders, but the performance is lacking in many cases and it is unclear why I would switch to a graphics API when I only need to do computation.

The simplest solution is that Vulkan allows for a zero-copy sharing of data between the APIs [1,2]. This should be the best case scenario for supporting platforms. The problem is that I don't know what platforms will actually support this. I think most desktop PCs will. I am pretty sure Android (maybe not pixel tho?) and Arm laptops will as well. Mac / iOS is unlikely and will probably require it's own solution. You can use OpenCL for now on modern macs, but it's deprecated.

Other ideas:

1. OpenCL without vulkan [3]. This involves a GPU -> CPU transfer for rendering with X (or whatever windowing API is necessary for specific platforms). It's a pain and the transfer times will probably be too slow for high FPS
2. CLSPV to compile OpenCL to vulkan. MoltenVK for Apple. I think the performance here is lacking.
3. Switch to Slang [5] or GLSL. MoltenVK for Apple. Again worried about performance / compilation performance. Slang can compile into Metal, but it treats it as a "textual target," meaning that it will generate a new text file to be read in to metal (for example). In that case, it makes more sense to just output metal directly since I am already string manipulating.
4. Metal for Apple, OCL / Vulkan for everything else. I think runtime compilation for metal shaders is straightforward? [4]

In practice, this means that I will try OCL / Vulkan interop and ride that as long as I can. It should work on all desktops. If I want to support mobile, then I will support Metal / Vulkan directly (via GLSL / Slang).

---

1. https://docs.vulkan.org/samples/latest/samples/extensions/open_cl_interop/README.html
2. https://docs.vulkan.org/samples/latest/samples/extensions/open_cl_interop_arm/README.html
3. https://github.com/ProjectPhysX/FluidX3D
4. https://stackoverflow.com/questions/32298719/manually-compile-metal-shaders
5. https://shader-slang.org/docs/first-slang-shader

[leios]

Another strat: I could "just use" vulkan compute shaders. Notes:

1. Guide to OpenCL -> Vulkan: https://www.neilhenning.dev/posts/opencl-vulkan-a-porting-guide-1/
2. glslangvalidator C api: https://github.com/KhronosGroup/glslang?tab=readme-ov-file#c-functional-interface-new

I think this should be doable with much less boilerplate than typical vulkan applications because I won't need to make use of any standard mesh machinery and can directly write the the swap chain images.

[leios]

I think (for simplicity), it is best to finish #4 with OpenCL and then come back to target Vulkan once everything is in place.

Again note that this is only if we care about realtime output. For anything that will be rendered as standalone images or videos, I don't need Vulkan

Another another option is to do what FlidX3D does and go straight OpenCL without Vulkan: https://github.com/ProjectPhysX/FluidX3D.

I think this involves mucking around with some X / Wayland code though.

[leios]

Note that it seems to be relatively easy to remain compliant with Vulkan swap chain buffers: https://vulkan-tutorial.com/Drawing_a_triangle/Presentation/Swap_chain

We can just specify that the VkSurfaceFormatKHR is something like VK_FORMAT_B8G8R8A8_SRGB, which would be an 8 bit R, G, B, A in that order.

[leios]

The other option is to support multiple backends (NVRTC -- NVidia RunTime Compulation / HIPRTC / similar for Metal).

In particular, finding a way to do metal runtime compilation is probably necessary for ios performance.