The only people who can be “driven out” are the renters, but if you let owners subdivide and build more, even they should be able to stay in the neighborhood indefinitely. It’s only once you break the chain by not building new homes that the existing community is forced to move. The original neighbors may become a minority compared to the newcomers, but the alternatives to building enough housing for everyone who wants to live in a neighborhood appears to me to be:
- make your neighborhood undesirable for even you to live in.
- make it impossible for new people to buy their way into the community.
- accept that non-owners and not-rich-enough owners will be priced out and forced to move from their homes.