You're going to need some exceptions. What happens when someone dies and leaves their house to their kids? What if someone's home is temporarily unlivable (say due to a fire or flood which requires extensive renovation) -- do they have to live on the streets?

And that's not even addressing the obvious question of what happens to tenants if there are no landlords.

The problem is supply, not distribution.