undefined | Better HN

0 pointsadvael1y ago0 comments

I swear no matter how many times people say it people will still conflate all ML with LLMs. No, chatGPT is not driving advances in self-driving or weather prediction

0 comments

xnx1y ago

For better or worse, "LLM" or "generative ai" has become roughly synonymous with the current wave of ML.

I know very little about ChatGPT, but Waymo is using an LLM: "Powered by Gemini, a multimodal large language model developed by Google, EMMA employs a unified, end-to-end trained model to generate future trajectories for autonomous vehicles directly from sensor data." (https://waymo.com/blog/2024/10/introducing-emma)

porridgeraisin1y ago

Waymo uses reinforcement learning (what it was before LLMs) (TD3+BC according to one of their blogs)

Emma is something they tried, but further down the article they explain why they don't use it as such yet.

xnx1y ago

Yep. It's an interesting experiment and really stretched my understanding of what an LLM is and can do.

advaelOP1y ago

Huh. I mean it makes sense to train end-to-end on all the interrelated tasks involved in driving but putting a whole-ass language model in the middle of that seems like a stunt. I wonder if it does better than like, any random transformer not trained on language first? Still, I hadn't heard that so I guess I was wrong about that one

magicalist1y ago

No, you were right, this appears to be just research on how applicable LLMs could be to the space. They talk about the improvements their LLM makes, especially in being multimodal vs training multiple independent models, but also the limitations that appear to prevent it from being useable as it is. Maybe some form if it will be used some day (it does seem like it would be useful to have semantic understanding of the world integrated into the system), but at least as of when this was published, it's not actually used.

famouswaffles1y ago

Vision Language Models are absolutely being trialed for self-driving

https://wayve.ai/thinking/lingo-2-driving-with-language/

advaelOP1y ago

Okay so because of the ambiguity of the other reply I'm just gonna say, I don't think we should be surprised that someone is trying to use LLMs to do basically anything. That's basically what prints funding money right now, so long as you're the kind of company or guy the VCs or whoever will believe in. The signal here is "does it do something to appreciably advance the state of the art over previous methods"?

famouswaffles1y ago

Seems to be the case to me, reading this and waymo's attempts. There's paper on EMMA here - https://arxiv.org/abs/2410.23262

And there are state of the art weather prediction transformers.

https://arxiv.org/abs/2312.03876

1 more reply

j / k navigate · click thread line to collapse

0 comments

xnx1y ago

For better or worse, "LLM" or "generative ai" has become roughly synonymous with the current wave of ML.

porridgeraisin1y ago

Waymo uses reinforcement learning (what it was before LLMs) (TD3+BC according to one of their blogs)

Emma is something they tried, but further down the article they explain why they don't use it as such yet.

xnx1y ago

Yep. It's an interesting experiment and really stretched my understanding of what an LLM is and can do.

advaelOP1y ago

magicalist1y ago

famouswaffles1y ago

Vision Language Models are absolutely being trialed for self-driving

https://wayve.ai/thinking/lingo-2-driving-with-language/

advaelOP1y ago

famouswaffles1y ago

Seems to be the case to me, reading this and waymo's attempts. There's paper on EMMA here - https://arxiv.org/abs/2410.23262

And there are state of the art weather prediction transformers.

https://arxiv.org/abs/2312.03876

1 more reply

j / k navigate · click thread line to collapse