The huge trip-duration and arrival-time uncertainty due to the shared rail, and having to defer to cargo trains, is what sinks it, for me. They can vary by large double digit percentages of the nominal time, at which point they're usually much slower than just driving.
I love how relaxing and comfortable they are and not having the security circus and dealing with crowded, distant-from-the-city airports, but not at nearly-air-travel prices and a 25-75% time premium over driving.
