It was when you went outside said use case that things started getting worse, and you had to wait long time for fixes. Sometimes it was because the changes in XFree/X.Org were effectively fixated on how some other vendors did things (cough intel cough), or involved things that effectively nobody wanted to spend engineering to fix properly (like rebuilding rendering path to be able to handle hybrid graphics properly when hybrid graphics came into world years after critical set of X.Org devs decided to stop any real development into X.Org...).

Vulkan Compute also is nowhere close to feature parity with CUDA, so not sure it would be picked up instead.