It's gotten much easier with Nvidia cards thanks to VM gaming being officially supported. No more hiding the VM and that Error 47 nonsense.
Basically you bind the graphics card to the vfio driver by writing the PCI ID to /sys/bus/pci/drivers/vfio-pci/new_id and then you can use it in qemu with -device vfio-pci,host=01:00.0.
If you have sane IOMMU groups and have an extra GPU for the host it should work with just those two steps. You still have to decide about how you handle input and output, but that's very user specific. Virtual input over the qemu GUI works well enough, there's also an evdev thing that lets you switch by pressing both Ctrl keys, or pass an entire USB controller. For audio I recommend scream[1] for the lowest latency (I get 2ms VM->PA->Speaker with MuQSS scheduler) with full 7.1 audio and if you don't want a dedicated monitor or switch monitor inputs there is Looking Glass[2] which captures frames and shuffles them to the host over shared memory. It also handles input via spice.
It's still a bit of work, but for me that's mostly on the Windows side rather than the passthrough stuff itself.
[1]: https://github.com/duncanthrax/scream
[2]: https://github.com/gnif/LookingGlass