Regardless, the real point is that property rights trump free speech rights. You can't come over to my house and say whatever you want, because it's private. And I'm pretty sure the UN DoHR says the same thing.
And further, given the leftist thing that it is, I'd bet dollars to donuts that the UN DoHR also says people have the right not to be on the receiving end of speech that incites violence towards them, a.k.a. hate speech. So... property rights > hate speech rights > free speech rights.