Intellij is awesome, and I use it for Scala with Emacs keybindings as well but like you said, the performance is just atrocious. Memory leaks, stutters, hangs, bad latency, you name it. Every Scala dev team I've been on constantly curses at Intellij. And yet, no one switches, because the functionality it provides is so good.
I never could get ensime to work, but I should try metals.