undefined | Better HN

0 pointsfragmede1y ago0 comments

You can usually tell that a lot of people just go off rumors they read once off Twitter or reddit or somewhere about hallucinations or doing math, against a weaker model, without every validating what they read online or updating their model of how well latest models work.

Just have to learn to let it go, despite xkcd 386.

0 comments

codexon1y ago

I use gpt4o and claude 3.5 while coding every day. The rumors on twitter and reddit are accurate.

I constantly run into incorrect answers from the LLMs every day. Just recently I asked whether I needed to reverse the bit shift to mask the upper 24 bits in an IP address on a little endian platform and it incorrectly told me no probably because most of the answers on Google appeared to answer no to similarly phrased questions.

theaussiestew1y ago

That's probably the best we can do for now with LLMs. Surely it would be unreasonable for LLMs to provide a correct answer when the majority of people on various forums also provide the wrong answer. Some day LLMs will be able to objectively provide truthful statements but we're not there yet. Regardless, that LLMs are competing with longstanding programmers is already an impressive feat, even if they're not 100% correct.

codexon1y ago

I'm not going to pass any judgement on what is reasonable or unreasonable for LLMs, just pointing out that it makes serious mistakes.

Many people here argue that LLMs are able to reason, you can check my post history 3 months ago to see an example of this.

And if you have to double check everything, how much time are you really saving? And is this thing really on track to replace programmers any time soon? Absolutely not.

2 more replies

philipwhiuk1y ago

> Regardless, that LLMs are competing with longstanding programmers is already an impressive feat, even if they're not 100% correct.

But they're not. They're a little tool some programmers use to save a small amount of time. They're not replacing programmers.

And there's no evidence they will ever be as good as actual programmers.

1 more reply

sandspar1y ago

Maybe you're using a strawman.

fragmedeOP1y ago

you need to give a counterexample to claim it's a strawman.

sandspar1y ago

"The people who disagree with me have no good reason to" is prima facie a strawman.

j / k navigate · click thread line to collapse

0 comments

codexon1y ago

I use gpt4o and claude 3.5 while coding every day. The rumors on twitter and reddit are accurate.

theaussiestew1y ago

codexon1y ago

I'm not going to pass any judgement on what is reasonable or unreasonable for LLMs, just pointing out that it makes serious mistakes.

Many people here argue that LLMs are able to reason, you can check my post history 3 months ago to see an example of this.

And if you have to double check everything, how much time are you really saving? And is this thing really on track to replace programmers any time soon? Absolutely not.

2 more replies

philipwhiuk1y ago

> Regardless, that LLMs are competing with longstanding programmers is already an impressive feat, even if they're not 100% correct.

But they're not. They're a little tool some programmers use to save a small amount of time. They're not replacing programmers.

And there's no evidence they will ever be as good as actual programmers.

1 more reply

sandspar1y ago

Maybe you're using a strawman.

fragmedeOP1y ago

you need to give a counterexample to claim it's a strawman.

sandspar1y ago

"The people who disagree with me have no good reason to" is prima facie a strawman.

j / k navigate · click thread line to collapse