undefined | Better HN

0 pointsyeck2y ago0 comments

Until I can trust that when I send an AI agent off to do something that I will be successful without me babysitting and watching over it constantly AI won't truly be transformative (since the human bottleneck will remain).

This is one of the core promises of alignment. Without it how can there be trust? While there are probably short term slow downs with an alignment focus, ultimately it is necessary to avoid throwing darts in the dark.

0 comments

reissbaker2y ago

I wouldn't mind a focus on reliably following tasks with greater intelligence; what I think is negative utility is focusing more compute and research resources on hypothetical superintelligence alignment — the entire focus of Ilya's "Superalignment" project — when GPT-4 is still way, way sub-human-intelligence. For example, I don't think the GPT Store was in any way a dangerous idea, which seems to have been Ilya's claimed safety red line.

yeckOP2y ago

I wouldn't call GPT-4 sub-human intelligence. While it's intelligence is less robust aggregate human intelligence, I don't think there is any one person alive who can compete with the breadth of GPT-4 knowledge.

I also think that the potential of what currently is possible with existing models has not been fully realized. Good prompting strategies and reflection may already be able to produce a system that is effectively AGI. Might already exist in several labs.

reissbaker2y ago

Wikipedia has broader knowledge, and yet no one calls it intelligent. I'm talking about reasoning capability, and GPT-4 is well below human on complex tasks, especially multi-step tasks, which is why "autonomous agents" like AutoGPT, BabyAGI, etc are not yet very useful.

For example: https://arxiv.org/abs/2311.09247

j / k navigate · click thread line to collapse

0 pointsyeck2y ago0 comments

0 comments

reissbaker2y ago

yeckOP2y ago

reissbaker2y ago

For example: https://arxiv.org/abs/2311.09247

j / k navigate · click thread line to collapse