The real worry(though not too terrifying) is more about a rouge driver with a hacked transmitter mimicking a "professional driver" and sending them bunk data that could veer them into an accident. One way to mitigate this is by having some kind of sonar/radar/sensor on each car in the cardinal directions that would indicate if the command was safe. I am sure there are other ways to game the system that I am not seeing from this cursory reflection.
Plus the feeling of control is very important. Flying is safer then driving, but numbers be damned, in a plane you have no control so it feels a lot worse then driving.
Citation? I'm not saying it's not true. I'm just curious if there are actually surveys of this. I've personally never felt a subway/tram/plane was any less safe because of lack of control so found the claim hard to empathize with.