undefined | Better HN

0 pointsjimbo8089mo ago0 comments

Humans don't make mistakes nearly as much, the mistakes they do make are way more predictable (they're easier to spot in code review), and they don't tend to make the kinds of catastrophic mistakes that could sink a business. They also tend to cause codebases to rapidly deteriorate, since even very disciplined reviewers can miss the kinds of strange and unpredictable stuff an LLM will do. Redundant code isn't evident in a diff, and things like tautological tests, or useless tests where they're mocking everything and only actually testing the mocks. Or they'll write a bunch of redundant code because they really just aggressively avoid code re-use unless you are very specific.

The real problem is just that they don't have brains, and can't think. They generate text that is optimized to look the most right, but not to be the most right. That means they're deceptive right off the bat. When a human is wrong, it usually looks wrong. When an LLM is wrong, it's generating the most correct looking thing it possibly could while still being wrong, with no consideration for actual correctness. It has no idea what "correctness" even means, or any ideas at all, because it's a computer doing matmul.

They are text summarization/regurgitation, pattern matching machines. They regurgitate summaries of things seen in their training data, and that training data was written by humans who can think. We just let ourselves get duped into believing the machine is the where the thinking is coming from and not the (likely uncompensated) author(s) whose work was regurgitated for you.

0 comments

6 comments · 3 top-level

ACCount379mo ago· 3 in thread

>The real problem is just that they don't have brains, and can't think.

That would have had more weight if you haven't just described junior developer behavior beforehand.

"LLMs can't think" is anthropocentric cope. It's the old AI effect all over again - people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot.

gnulinux9969mo ago

> That would have had more weight if you haven't just described junior developer behavior beforehand.

Effectively telling that junior developers "don't have brains" is in very bad taste and offensively wrong.

> people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot.

Would you like to elaborate on this?

I was told that McDonalds employees would have been replaced by now, self-driving cars will be driving the streets and new medicines would have been discovered.

It's been a couple of years that "AI" is out, and no singularity yet.

ACCount379mo ago

LLMs use the same type of "abstract thinking" process as humans. Which is why they can struggle with 6-digit multiplication (unlike computer code, very much like humans), but not with parsing out metaphors or describing what love is (unlike computer code, very much like humans). The capability profile of an LLM is amusingly humanlike.

Setting the bar for "AI" at "singularity" is a bit like setting requirements for "fusion" at "creating a star more powerful than the Sun". Very good for dismissing all existing fusion research, but not any good for actually understanding fusion.

If we had two humans, one with IQ 80 and another with IQ 120, we wouldn't say that one of them isn't "thinking". It's just that one of them is much worse at "thinking" than the other. Which is where a lot of LLMs are currently at. They are, for all intents and purposes, thinking. Are they any good at it though? Depends on what you want from them. Sometimes they're good enough, and sometimes they aren't.

2 more replies

XenophileJKO9mo ago

Living in Silicon Valley, there are MANY self driving cars driving around right now. At the stop light the other day, I was between 3 of them without any humans in them.

It is so weird when people pull self driving cars out as some kind of counter example. Just because something doesn't happen on the most optimistic time scale, doesn't mean it isn't happening. They just happen slowly and then all at once.

1 more reply

alecco9mo ago

> Humans don't make mistakes nearly as much [...]

Yeah, I remember how in every large corporation the specs were perfectly interpreted and with no issues, at all. Humans are great at communication and understanding each other.

https://google.com/search?q=tree+swing+cartoon

ozim9mo ago

I don't really see this argument playing out.

As much as I was trying out copilot — if I ask it to make "todo app" it will make me a "todo app" it will not hallucinate making "calculator app".

Duplicated code or tautological tests are not going to sink the business.

From my experience I have seen much more problems and hours burned caused by wrong application of DRY than from duplication.

I can say that in regulated environments duplicated code that has no abstractions is even preferred.

j / k navigate · click thread line to collapse