Regardless of the OP's position on this, I do think the reality is that in practice it's difficult to sell or legalize anything that isn't significantly better than 'the average driver'. Whether it makes sense or not, we seem to demand any 'AI-powered' system to be better than the average human. I personally wish this wasn't the case, but it appears to be so.

For example, I personally marvel at the abilities of Siri and Google Now, at the very least when it comes to understanding what I'm saying. It's quite possible, in fact, that both are often objectively better at understanding me than most people around me: more than once I said something to Siri where I immediately realized that it was mumbly enough for any real human in my vicinity to ask 'what?' (often only to right away act on what I said, indicating that they did manage to interpret what I'd said). And yet Siri or Google Now actually transcribed what I'd said.

Being as good as the people around me would not be good enough, because every time Siri fails it makes me shake my fist at the whole concept of 'personal assistants' on my iDevices, rather than sigh at the umpteenth person around me saying 'what?' and then obviously acting on what I'd said, which is much more common.

I've not studied this properly, so I might be wrong. But I get the impression that I hold my AI to a much higher standard than I would my human interactions.

Most of the safety of AI comes from it being implemented in vehicles that are already far safer than average.