> Your example of the Go AI being programmed to learn is not all that accurate for what has been achieved here.
What do you mean?
AlphaZero was trained entirely on self-play, and is a generic reinforcement learning algorithm. All it starts with are the rules (Chess, Go, Shogi) and a few million games later it beats — so far as I can see from a quick Google — all the humans, and most matches against AlphaGo Zero which learned the same way and which in turn beat AlphaGo Lee in every match, and that (unlike the aforementioned) was trained on examples of human matches in addition to self-play… but still learning from those examples as there's no known useful[0] set of rules that even says if a Go game is over let alone which moves are good.
There are AI which can find and exploit its weaknesses, but I've not seen anyone else suggest humans can defeat it.
> I didn't keep the link for the discussion on the confounding of the Go AI system. What the discussion covered though was that there were simple Go configurations that the GO AI failed abysmally on when playing a human - it didn't learn here.
Do you remember the name of the AI?
A bit of rummaging got me KataGo, but the humans had to use another AI to discover the weaknesses of KataGo rather than figuring it out for themselves.
And yes, KataGo absolutely does learn. The fact you can trivially stop the learning process is a feature not a bug for AI, precisely because it means any safety testing of the sort you're calling for is actually possible (albeit rather different than formal logic).
[0] pathological cases are easy — "board empty == not finished" — but not helpful.
