undefined | Better HN

0 pointsihumanable1y ago0 comments

Alignment in the realm of AGI is not about getting everyone to agree. It's about whether or not the AGI is aligned to the goal you've given it. The paperclip AGI example is often used, you tell the AGI "Optimize the production of paperclips" and the AGI started blending people to extract iron from their blood to produce more paperclips.

Humans are used to ordering around other humans who would bring common sense and laziness to the table and probably not grind up humans to produce a few more paperclips.

Alignment is about getting the AGI to be aligned with the owners, ignoring it means potentially putting more and more power into the hands of a box that you aren't quite sure is going to do the thing you want it to do. Alignment in the context of AGIs was always about ensuring the owners could control the AGIs not that the AGIs could solve philosophy and get all of humanity to agree.

0 comments

ndriscoll1y ago

Right and that's why it's a farce.

> Whoa whoa whoa, we can't let just anyone run these models. Only large corporations who will use them to addict children to their phones and give them eating disorders and suicidal ideation, while radicalizing adults and tearing apart society using the vast profiles they've collected on everyone through their global panopticon, all in the name of making people unhappy so that it's easier to sell them more crap they don't need (a goal which is itself a problem in the face of an impending climate crisis). After all, we wouldn't want it to end up harming humanity by using its superior capabilities to manipulate humans into doing things for it to optimize for goals that no one wants!

concordDance1y ago

A corporate dystopia is still better than extinction. (Assuming the latter is a reasonable fear)

portaouflop1y ago

I disagree. Not existing ain’t so bad, you barely notice it.

simianparrot1y ago

Neither is acceptable

tdeck1y ago

Don't worry, certain governments will be able to use these models to help them commit genocides too. But only the good countries!

vasco1y ago

It still think it makes little sense to work on because guess what, the guy next door to you (or another country), might indeed say "please blend those humans over there", and your superaligned AI will respect its owners wishes.

wruza1y ago

AGI started blending people to extract iron from their blood to produce more paperclips

That’s neither efficient nor optimized, just a bogeyman for “doesn’t work”.

FeepingCreature1y ago

You're imagining a baseline of reasonableness. Humans have competing preferences, we never just want "one thing", and as a social species we always at least _somewhat_ value the opinions of those around us. The point is to imagine a system that values humans at zero: not positive, not negative.

freehorse1y ago

Still there are much more efficient ways to extract iron than from human blood. If that was the case humans would have already used this technique to extract iron from the blood of other animals.

1 more reply

j / k navigate · click thread line to collapse