I think a big part of the problem is how programmers developing AI routines deal with generating aspects of a problem space the AI doesn't understand. Which is all of it.
I use Google voice. When I receive voice calls, the Google Voice application attempts to transcribe the voice recording.
Most of the content is fine, but when the program encounters part of the message that doesn't meet it's expectations, IT JUST MAKES SOMETHING UP SOUNDS GOOD TO IT.
So if someone leaves me the voice message "Your code is 532-94AA" Google voice transcribes it as "Your code is 532-9488"
If my wise-cracking friend ends a a message with, "Talk to you never" Google will translate this as "Talk to you later".
If someone leaves me voice directions, "Turn <garbled> at the intersection", google will randomly pick a direction: "Turn right at the intersection".
It's like having a lazy, irresponsible assistant who really doesn't give a shit about communicating messages to you accurately. As long has he sounds good, that's all he cares about. "I don't know" isn't a concept Google AI even considers.
Instead of admitting it isn't confident about a sound, it just reaches up it's ass and makes a guess. And since AI's actual understanding of content is zero, everything sounds good to it.
When an AI Bias happens in your favor, or when it happens and your the victim of it, and it has real consequences, what recourse does anyone have agains the bias of the AI.
Obviously, AI is based on the idea that computers are far faster at a task than many human minds...
So when we put actual 'decision' making power (meaing it will result in an in-real-life consequence to the Human - and lets assume its a false account of what happened, etc - who is liable?
How can this be regulated?
Who is using AI against me?
How can I know?
How can I fight it?
How can I validate it (if the AI results in my innocence of a crime/situation/scenario)
Who and how many fall through the cracks?
---
The darkest future is one where whole societal behavior patterns are tacitly modified by AI layers upon all aspects of their life.
