So... probably a 3090 or 4090? Or more exotic budget options like an A770 or a tenstorrent pcie card.
I would run Clear Linux or CachyOS (depending on if I need Nvidia driver support).
I also have a rather old desktop computer that runs Linux, has a 1050 and has a ZFS storage array on hard disks and serves files, media, controls WiFi access points and does other IoT controlling tasks.
Other than getting a 4090 I don’t want for much. It is possible to build a much more powerful machine but in terms of fan noise it is not easy to live with. Some people’s “ideal” would be about small size, low power, no fan, etc. See this site
For insight into hardware from hyper scale to micro desktops you can buy on ebay.
Right now I would run/train Stable Diffusion and maybe run Facebook's GPT-like model (that just apparently leaked out from 4chan), and then run clients to mess with them on computers, phones and stuff. But the generative AI space is moving so fast, all that could be obsolete in a month.
There are crazy AI projects for all sorts of stuff (text to speech, "vector" searches on data, meme animation generation from stick figure drawings), and the requirement to run them is generally "a GPU with lots of VRAM"
