When you choose to turn your establishment into a Nazi bar, people who don't want to hang out with Nazis will find someplace else to be. That isn't unfair, that's just the consequences of your actions.
Musk didn't want those interesting people on his platform to begin with, and he he did everything he could to drive them away.