We must confront that fact, or negate it, for example by taxing away the investment opportunity.

Investors buy housing to rent it out to others, that's the investment. People who own their own home implicitly pay themselves rent, by choosing to live in what they own versus renting instead.

Getting non-resident owners out of housing only means that renters have been banned from an area.