Stable Diffusion on AMD RDNA3 (opens in new tab)

OpenCL works pretty well. Can't say I notice large gaps of performance between CUDA and openCL for my hpc work.

OpenCL is hardly an alternative, plain old C, using compilation from source at runtime, with very basic tooling available.

Versus a polyglot compiler infrastructure, IDE tooling that includes shader debugging, and a rich ecosytem of GPU based libraries.

Even with SYSCL and SPIR-V, that has hardly improved, and while Intel bases oneAPI on top of SYSCL, that naturally also goes beyond the standard.

A big part of the reason is that Blender on Nvidia supports hardware accelerated ray tracing using OptiX. HIP-RT exists, but is not used in Blender yet. I think the Intel oneAPI backend for Arc GPUs also misses RT acceleration.

AMD claims to have HIP-RT working internally, but not yet suitable for posting publically. Intel is planning it, I think. Both should land around Blender 3.6, if I'm not mistaken.

If you take the raw FLOPS, CUDA (not OptiX) and HIP are actually nearly equivalent in performance last I remember. I think RDNA2 just does "more with less", at least in terms of gaming performance per FLOP (e.g. due to the huge cache).

Latest Blender release does not have the optimization work in yet.

AIUI, what's in current git master is very different.

It's because NVIDIA actually cared and AMD does not where it matters (customer HW).

Openness is totally secondary to functional. You know, the same kind of reasons as of why Linux on the desktop is not a mass market thing compared to Windows for a very long time.

CUDA is general purpose compute, but nvidia also releases cudnn which all the major libraries use because it is fast and good (if a little complex). There’s efforts underway to have a comparable library on open source general compute packages but none as mature or effective as cudnn so people just pay nvidia to use that in practice, which lets them invest even more in pulling ahead.

As an aside, I’ve been kinda surprised that this has existed for as long as it has, but I am probably biased and think Ml acceleration is more important than most large business do today.

Vulkan is designed for all GPU needs, from rendering to general purpose compute.

CUDA is only for general purpose compute.

CUDA is for GPGPU (general purpose GPU) which includes machine learning.

Vulkan is a primarily for graphics but does have options for GPGPU too. Vulkan is however not like OpenGL in that it's fairly close to the hardware in terms of abstraction.

For anyone on Arch, there is a third-party repository called arch4edu[0] that provides up to date builds of ROCm and its dependencies. On my iGPU, OpenCL sometimes works, sometimes crashes. Even finding a list of supported hardware is close to impossible. The whole situation is just ridiculous and makes AMD look bad.

[0] https://github.com/arch4edu/arch4edu

nod-ai/SHARK from the original submission is by far the fastest way I've found to run Stable Diffusion on a 5700 XT.

For 50 iterations:

* ONNX on Windows was 4-5 minutes

* ROCm on Arch Linux was ~2.5 minutes

* SHARK on Windows is ~30 seconds

Thanks for that - I have missed the block-sparse extension of the algorithm when I first read about it. And indeed this seems to be what the author means.