The people who share ownership are entitled to the resource and can decide how someone can join.
The land your home is built on is part of a shared resource: the space available for the community.
If we apply your logic there s nothing wrong with immigrants showing up and colonizing natives - be it in America, Australia, etc - shared resource bro!