undefined | Better HN

0 pointsismail6y ago0 comments

Asimov 3 rules as the final policy when making decisions should sort this problem out. This assumes that the rules cannot be changed by the AI.

0 comments

4 comments · 2 top-level

Falling36y ago· 2 in thread

I've always been very incredulous that there would be any possibility of taking something sufficiently complex to be considered an AGI and hard-coding anything like the 3 rules into it.

smogcutter6y ago

By the same token, I’m extremely suspicious of the idea that such a sufficiently complex AGI could also be dumb enough to optimize for paper clip production at the expense of all life on earth (or w/e example).

ludwigschubert6y ago

...and many would say that’s because us humans are bad at imagining optimizing agents without anthropomorphizing them. This is a reasonable, even typical suspicion that many people share! The best explanation I know of why it’s unfortunately wrong is by Robert Miles in a video, but if you prefer a more thorough treatment, you could also read about “instrumental convergence” directly. If you find a flaw in this idea, I’d be interested to hear about it! :)

Robert Miles’ video: https://youtu.be/ZeecOKBus3Q

Instrumental Convergence: https://arbital.com/p/instrumental_convergence/

Now afaik nothing in this argument says that we can’t find a way to control this in a more complex formalism-but we clearly haven’t done so yet.

1 more reply

mithr6y ago

> This assumes that the rules cannot be changed by the AI.

And sidesteps the fact that many of Asimov's stories were precisely about robots finding ways around these rules :)

j / k navigate · click thread line to collapse