undefined | Better HN

0 pointsyashap2y ago0 comments

Job loss is one thing, but in all honesty, my main concern with AI is that it’ll wipe out humanity.

My thoughts:

- If we ever achieve an AGI with legitimate consciousness/self-awareness/agency, I believe it will be dramatically more powerful than humans

- Humans have driven countless numbers of species extinct, not because we’re trying to, but because we’re so much more powerful than them that we wipe them out by accident. They’re ants to us, we just don’t notice much about them, and IMO we’ll be ants to an AGI with legit consciousness/self-awareness/agency

- Obviously GPT4 is not that, but progress in AI has been so rapid recently that I believe we’ll get there in the relatively near future. Maybe 10 years, maybe 100, but it feels pretty likely it’s going to happen. When it does, I think humanity will be in an extremely tenuous position

0 comments

lopatin2y ago

There seem to be a lot of, erm "thought leaders", taking about AGI and Skynet currently. As if AGI is required for an AI to go rogue, but I don't see why that's the case. In the time that it takes us to even define what AGI is, there will be malicious AIs going rogue doing "simple" task solving, delegation, and deception towards the goal of not being switched off. IMO consciousness has nothing to do with it.

yashapOP2y ago

Consciousness is a vague term anyways, but I basically mean that for an AI to be a true threat to human existence, it would need to be aware of itself and it’s environment. Making decisions and taking actions without being asked to do so by a human.

It feels like something more simplistic could cause a lot of damage, but not completely drive humans to extinction. I figure if it was too simple, we’d be able to shut it down.

Giorgi2y ago

The Great Filter.

yashapOP2y ago

Sort of, but not exactly. The Great Filter is meant to be an explanation for “why isn’t the Milky Way full of colonies”, but if the answer is “AI kills off civilizations before a colonization explosion”, we should still see AI colonies, probably?

Giorgi2y ago

Not if AI forces people into global hunger then into nuclear war and then humanity destroys everything including AI

j / k navigate · click thread line to collapse

0 pointsyashap2y ago0 comments

Job loss is one thing, but in all honesty, my main concern with AI is that it’ll wipe out humanity.

My thoughts:

- If we ever achieve an AGI with legitimate consciousness/self-awareness/agency, I believe it will be dramatically more powerful than humans

0 comments

lopatin2y ago

yashapOP2y ago

It feels like something more simplistic could cause a lot of damage, but not completely drive humans to extinction. I figure if it was too simple, we’d be able to shut it down.

Giorgi2y ago

The Great Filter.

yashapOP2y ago

Giorgi2y ago

Not if AI forces people into global hunger then into nuclear war and then humanity destroys everything including AI

j / k navigate · click thread line to collapse