undefined | Better HN

0 pointskevinwang3y ago0 comments

Eh, it's still impressive that these systems can write such good code despite pretty much just predicting the next word. I guess it's a matter of perspective. You can either be astounded by how much it can do relative to your expectations from 2018 or you can be skeptical relative to the volume of excitement. I think some excitement is also people extrapolating to the future: if predicting the next word gets you this far, what happens when you actually try to make it good?

0 comments

pms3y ago

> what happens when you actually try to make it good?

1. Do you think that companies paying millions of dollars to ML researchers aren't already trying to make it good?

2. I think it will take a real revolution in AI/ML to do what people here are extrapolating into the future. That revolution will eventually come, but I doubt it'll be as quick as people think. Just think about the excitement people had about Siri 10+ years ago, or about Full Self Driving 5+ years ago. In my opinion, in 5-10 years from now GPT will be in the same place where Siri and Full Self Driving currently are. Eventually we will make the leap we're dreaming of, but that leap isn't happening yet.

copperx3y ago

> despite pretty much just predicting the next word

Is GPT really like the old school NLP trigram stuff?

hnfong3y ago

"predicting the next word" sounds trivial until you realize the value and complexity of "predicting the next word[s] by a world leading expert in a particular domain".

Jensson3y ago

Quoting famous people sounds smart until you realize they just memorized a ton of trivia. These models have demonstrated that they don't learn logical models, instead they learn to generate text that looks logical at first glance but is nonsense.

1 more reply

j / k navigate · click thread line to collapse