Or because it gives more freedom--you're not restricted to particular routes or particular times.
Mass transit can beat driving on all these metrics (speed, price, and freedom of travel) when the population density is high enough that it's cost-effective for mass transit to have dense coverage, so that you're never more than a short walk from a train station or a bus stop. But many people (including me) don't want to live in a place with that high a population density. That's ultimately why mass transit hasn't caught on in the US the way it has in other countries.
It's kind of a chicken and eggs sort of thing but I'm pretty sure if you've suddenly by some miracle have really awesome mass transit in LA or Houston like you do in NYC, Tokyo & lots of European cities (even medium sized ones like Prague or Copenhagen) I'll bet loads of those freedom and sparsity loving folk will start using it and over time population and living patterns will adapt to the new conditions.
Not everywhere. When I was in school in Boston, I went everywhere by public transit. I never missed having a car (I rented one the few times I wanted to travel somewhere like up to New Hampshire to see foliage). However, I also learned from that experience that I don't want to live in a place like that for the rest of my life.
not every city with great transit has to be a stressful megalopolis like NYC - Vienna or Zurich for example are really nice and quiet (and clean and have low crime) despite having great transit
I didn't find Boston to be a "stressful megalopolis" (I haven't lived in NYC so I can't speak to that). I just realized I didn't want to live permanently in a place that dense. I've also visited London and Paris, both of which have excellent mass transit (at least by US standards), and while I enjoyed visiting them, I don't think I would want to live permanently in those cities either. Maybe I'll get a chance sometime to visit one of the smaller European cities you mention to see what they're like.
As for why it hasn't caught on, it's because it's incomplete and inconvenient for most people. Even where it works, it's still not great. Ever take a slow bus across town in Manhattan? It's faster to walk.
The 9-lane roads in metropolitan areas aren't confined to the densely populated areas; in fact, many of them are there only because so many people commute from the less densely populated suburbs to the densely populated inner city to work, instead of living in the inner city where they could easily take mass transit.
