Construction companies that build residences are clearly creating value for society, so they should be allowed to make some kind of profit. Real estate agencies that buy property from construction companies are doing a very bad thing by renting them as "luxury" housing to people who otherwise have no choice since their livelihood is tied to an urban area.
Why not put a cap on the profit that can be made from a residence? Something like 5x construction costs, plus ongoing costs of maintenance. Suddenly, the "luxury" housing market is no longer suffocating all the affordable housing out of town since the property owner can't expect to rake in the piles of money every month.