That's sort of happened. Witness the relatively little appetite the airlines have for the largest planes, e.g. A380.

But, yeah, I'm an hour to the airport even early morning, arriving at least 1.5 hours before flight time. For London, I usually have a change in Newark. TSA actually doesn't take me long and London's usually pretty fast at immigration these days with biometric US passports. But then it's usually about an hour to my hotel in London. So supersonic trans-Atlantic wouldn't do a lot for me in the scheme of things.