I agree that the local land use is a total disaster, but even if they made it better, SF has a few skinny tendrils of transit which creates a few skinny corridors of transit-accessible area.
The constant excuse I kept hearing when I was living there was that there isn't enough space!! But that's only true if you won't do anything to increase the viable space to live in (and optimize the space that is present, through better land use). Even densifying the transit along these corridors, San Franciscans will still whine and complain about how there's no space, ignoring the obvious solution.
The answer to me is to do it both, yesterday. But I'm not a San Franciscan anymore and I understand that my values differ from theirs.
Aside, it's also worth mentioning that in Tokyo as well, single family homes near train stations abound. The difference is that they're smaller and denser, and mixed in with apartments. There aren't that many towers here! But the roads are mini-sized and they pack units tightly to better use land. If San Franciscans hate towers that cast shade, they have alternatives.
[1] https://en.wikipedia.org/wiki/Transport_in_Greater_Tokyo