undefined | Better HN

0 pointsgodelski2mo ago0 comments

  > there's no reason we'd need to have humans working jobs that only involve typing stuff into a computer and going to meetings all day

I'm not sure I understand, and want to check. That really applies to a lot of jobs. That's all admins, accountants, programmers, probably includes lawyers, and probably includes all C-suite execs. It's harder for me to think of jobs that don't fit under this umbrella. I can think of some, of course[0], but this is a crazy amount of replacement with a wide set of skills.

But I also think that's a bad line to draw. Many of those jobs include a lot more than just typing into a computer. By your criteria we'd also be replacing most scientists, as so many are not doing physical experiments and using the computer to read the work of peers and develop new models. But also does get definition intended to exclude jobs where the computer just isn't the most convenient interface? We should be including more in that case since we can then make the connection for that interface.

I think we need a much more refined definition. I don't like the broad strokes "is computer". Nor do I like skills based definitions. They're much easier to measure but easily hackable. I think we should try to define more by our actual understanding of what intelligence is. While we don't have a precise definition we have some pretty good answers already. I know people act like the lack of an exact definition is the same as having no definition but that's a crazy framing. If we had that requirement we wouldn't have any definitions as we know nothing with infinite precision. Even physics is just an approximation, but it's about the convergence to the truth [1]

[side note] the conventional way to do references or notes here is with brackets like I did. So you don't have to escape your asterisks. *Also* if it lead a paragraph with two spaces you get verbatim text

[0] farmer, construction worker, plumber, machinist, welder, teacher, doctors, etc

[1] https://hermiene.net/essays-trans/relativity_of_wrong.html

0 comments

daxfohl2mo ago

Actually it occurs to me that even if we did have AGI, or even if ASI, heck if ASI even moreso, we'd still need desk jobs to maintain the guardrails.

Intelligence is one thing, being able to figure out how get a task done (say). But understanding that no, I don't want you to exploit a backdoor or blackmail my teammate or launch a warhead even though that might expedite the task. Or why some task is more important than another. Or that solving the P=NP problem is more fulfilling than computing the trillionth digit of pi. That's perhaps a different thing entirely, completely disjoint with intelligence.

And by that definition, maybe we are in the neighborhood of AGI already. The things can already accomplish many challenging tasks more reliably than most humans. But the lack of wisdom, emotion, human alignment, or whatever we want to call it, lead it to accomplish the wrong tasks, or accomplish them in the wrong way, or overlook obvious implicit requirements, may cause people to view it as unintelligent, even if intelligence is not the issue.

And that may be an unsolvable problem because AI simply isn't a living being, much less human. It doesn't have goals or ambitions or want a better future for its children. But it doesn't mean we can never achieve AGI.

Oh, and to your first question, yes it's a huge number of jobs, maybe half of jobs in developed nations. And why not? If you can get AI to do the work of the scientist for a tenth of the price, just give it a general role description and budget and let it rip, with the expectation that it'll identify the most promising experiments, process the results, decide what could use further investigation, look for market trends, grow the operation accordingly, that's all you need from a human scientist too. Plausibly the same for executives and other roles. Of course maybe sometimes the role needs a human face for press conferences or whatever, and I don't know how AI would be able to take that, but especially for jobs that are entirely internal-facing, it seems like there's no particular need for a human. Except that maybe, given the above, yes, you still need a human at the helm.

godelskiOP2mo ago

  > we'd still need desk jobs to maintain the guardrails.

Agreed. I don't get why people think it is a good idea not to. I'd wager even the AGI would agree. The reason is quite simple: different perspectives help. Really for mission critical things it makes sense to have multiple entities verifying one another. For nuclear launches there's a chain of responsibility and famously those launching have two distinct keys that must be activated simultaneously. Though what people don't realize is that there's a chain of people who act and act independently during this process. It isn't just the president deciding to nuke a location and everyone else carrying out the commands mindlessly. But in far lower stakes settings... we have code review. Or a common saying in physical engineering as well among many tradesmen "measure twice, cut once".

It would be absolutely bonkers to just hand over absolute control of any system to a machine before substantial verification. These vetting processes are in place for a reason. They can be annoying because they slow things down, but they're there because they speed things up in the long run. Because their existence tends to make things less sloppy, so they are less needed. But their existence also catches mistakes that were they made slow down processes far more than all the QA annoyances and slowdowns could ever cause combined.

  > And why not? If you can get AI to do the work of the scientist for a tenth of the price

And what are the assumptions being made here? Equal quality work? To my question, this is part of the implication. Price is an incredibly naive metric. We use it because we need something, but a grave mistake is to interpret some metric as more meaningful than it actually is. Goodhart's Law? Or just look at any bureaucracy. I think we need to be more refined than "price". It's going to be god awfully hard to even define what "equal quality" means. But it seems like you're recognizing that given your other statements.

daxfohl2mo ago

And "maintaining guardrails" may be far more grandiose than it sounds. It's like if we have this energy source that could destroy the planet, but the closer you get to it without going past some threshold, the energy you get from it is proportional to the inverse of how close you are to it. There's some wiggle room and you can poke and prod and recover if it starts to go ballistic, but your goal is to extract as much energy (or wealth or whatever) out of it as possible. Every company in the world, every engineer on the planet would be pushing to extract just a little bit more without going beyond the limit.

AI could go the same way. It's a creation engine like nothing that's ever been seen before, but it can also become a destruction engine in ways that we could never understand or hope to counter, and left unchecked, the odds of that soar to near certainty. So the first job is to place dummy guardrails around it. That's where we are now. But soon that becomes too restrictive. What can we loosen? How do we know? How can we recover if we're wrong? We're not quite there yet, but we're not not there either.

Of course eventually somebody is going to trigger it and it's going to go ballistic. Our only hope is that it happens at exactly the right time where AGI can cause enough damage for people to notice, but not enough to be irrecoverable. Maybe we should rename this whole AGI thing to Project Icarus.

nomel2mo ago

> we'd still need desk jobs to maintain the guardrails.

Simply put, an economy and society of managers. That's terrifying.

froggit2mo ago

> [0] farmer, construction worker, plumber, machinist, welder, teacher, doctors, etc

The reason AGI couldn't do these is the lack of a suitable interface to the physical world. It would take a trivial amount of effort for these to be designed and built by the AGI. Humans could be cut from the loop after an initial production run made up of just the subset of these physical interface devices needed to build more advanced ones.

j / k navigate · click thread line to collapse

0 comments

daxfohl2mo ago

Actually it occurs to me that even if we did have AGI, or even if ASI, heck if ASI even moreso, we'd still need desk jobs to maintain the guardrails.

godelskiOP2mo ago

  > we'd still need desk jobs to maintain the guardrails.

  > And why not? If you can get AI to do the work of the scientist for a tenth of the price

daxfohl2mo ago

nomel2mo ago

> we'd still need desk jobs to maintain the guardrails.

Simply put, an economy and society of managers. That's terrifying.

froggit2mo ago

> [0] farmer, construction worker, plumber, machinist, welder, teacher, doctors, etc

j / k navigate · click thread line to collapse