It still needs to work 99.99% of the time. I am more thinking extreme edge cases like: "Drove off the highway to avoid collision and stuck in a ditch. Bridge out now what? etc."

Basically, you don't need a perfect solution in every case, just a safe solution in every case. A self driving car that get's to a gas station is a good enough step 1 even if you have some human attendants there to fill up. Sure, it might add $2 bucks per fill up that's still cheaper than highway driving.

PS: Self driving car, only works in NYC is also useful.

And if you can remote in, I'm worried about hacks. Imagine the chaos if a terrorist group got in and turned the wheel of an entire fleet in a random direction. It seems to me anything can get hacked/any individual can be pressed, and it's alsways just a matter of if it's worth the trouble. Seems a hack like this is would be worth some, to some groups. Also: they only need to succeed Once. The rest of us need to have perfect security among all fleets of all corporations and brands for all time.