I would argue that is because you’re buying with the wrong intention. If you just want a place to live, rather than expecting insane home appreciation that we have seen in the last decade, then I doubt this would be as much of an issue.
Personally, I think it’s because Americans have an aversion to high density housing. “I don’t want anyone living on top of me” is a common refrain I’ve heard, along with wanting a backyard. This was feasible when we had population levels of the 1940s, but not today.