Exactly. If I was one of those people, no one but my 30-50 most trusted friends and family members would be allowed 100 yards near me.
I would be afraid for my safety 24/7, for my children's... For my private life (I would never talk about my personal opinions to anyone outside of my immediate family). Etc.
Think Taylor Swift in Walmart. She could probably get trampled over and die (no joke). Zuckerberg or Gates in a bar or a county fair? Probably beaten to death or shot.
It's easy for us to imagine they live isolated from regular people because they think they're better than us (although they probably think that)... But the reality is that regular people are dangerous for this kind of people.