Old meaning not young. Pretty much all crimes or any other forms of messy behavior worldwide are committed by young men.
But the median age in NYC is 38 and Tokyo is 45. (source: two Google searches I just did). That means a lot!
It's true they don't jump the gates often and they don't have loud panhandlers. Instead the societal transit ills are passed out drunks, suicides and molesters. (Not meaning these actually happen all the time, it's just my impression of what people talk about.)
> it’s pretty much everyone who is not filthy rich isn’t it?
Hmm, it's more about what you're doing, I think? Rich people use transit all the time if it serves their purposes afaik. One thing that helps in Japan is the culture of wearing face masks means you won't be recognized in public. (Obviously this doesn't work if you're like a 7' NBA player.)
For going between cities the trains are actually the nice expensive option, and flying or taking a night bus is cheaper.
But trains are also basically only good at carrying yourself. If you're traveling in a group, or carrying equipment with you, or don't want to walk a lot then you'd still want to drive or take a taxi locally.