undefined | Better HN

0 pointsdougabug3y ago0 comments

This characterization is not really accurate. OpenAI has had almost a 2 year lead with GPT-3 dominating the discussion of LLMs (large language models). Google didn’t release its paper on the powerful PaLM-540b model until recently. Similarly, CLiP, Glide, DALL-E, and DALL-E2 have been incredibly influential in visual-language models. Imagen, while highly impressive, definitely is a catch-up piece of work (as was PaLM-540b).

Google clearly demonstrates their unrivaled capability to leverage massive quantities of data and compute, but it’s premature to declare that they’ve secured victory in the AI Wars.

0 comments

benreesman3y ago

I agree that it’s still a jump ball in a rapidly moving field, I was saying Google is far ahead, not that they’ve won.

And I don’t think whatever iteration of PaLM was cooking at the time GPT-3 started getting press would have looked to shabby.

I think Google crushed OpenAI on both GPT and DALL-E in short order because OpenAI published twice and someone had had enough.

alphabetting3y ago

OpenAI and FAIR are definitely in the same league as Google but Google has been all-in on AI from the beginning. They've probably spent well over $100B on AI research. I really enjoyed the Genius Makers book which came out last year from an NYT reporter on history of ML race. Deepmind apparently turned down a FB offer of double what Google was offering.

benreesman3y ago

Cade Metz is that author and most of it I can only speculate on.

The bits and pieces I saw first hand tie out reasonably well with that account.

dougabugOP3y ago

That’s pretty speculative and dubious (the holding back part) given the heavy bias to publication culture at Google Research and DeepMind. OpenAI has hardly been “crushed” here; PaLM and Imagen are solid, incremental advances, but given what came before them, not Earth-shattering.

If I were going to cite evidence for Alphabet’s “supremacy” in AI, I would’ve picked something more novel and surprising such as AlphaFold, or perhaps even Gato.

It’s not clear to me that Google has anything which compares to Reality Labs, although this may simply be my own ignorance.

Nvidia surely scooped Google with Instant Neural Graphics Primitives, in spite of Google publishing dozens of (often very interesting) NeRF papers. It’s not a war, all these works build on one another.

benreesman3y ago

I want to be clear, all of this stuff is fascinating, expensive, and difficult. With the possible exception of a few trailer-park weirdos like me, it basically takes a PhD to even stay on top of the field, and you clearly know your stuff.

And to be equally clear, I have no inside baseball on how Brain/DM choose when to publish. I have some watercooler chat on the friendly but serious rivalry between those groups, but that’s about it.

I’m looking from the outside in at OpenAI getting all the press and attention, which sounds superficial but sooner or later turns into actual hires of actual star-bound post docs, and Google laying a little low for a few years.

Then we get Gato, Imagen, and PaLM in the space of like what, 2 months?

Clearly I’m speculating that someone pulled the trigger, but I don’t think it’s like, absurd.

1 more reply

j / k navigate · click thread line to collapse

0 comments

benreesman3y ago

I agree that it’s still a jump ball in a rapidly moving field, I was saying Google is far ahead, not that they’ve won.

And I don’t think whatever iteration of PaLM was cooking at the time GPT-3 started getting press would have looked to shabby.

I think Google crushed OpenAI on both GPT and DALL-E in short order because OpenAI published twice and someone had had enough.

alphabetting3y ago

benreesman3y ago

Cade Metz is that author and most of it I can only speculate on.

The bits and pieces I saw first hand tie out reasonably well with that account.

dougabugOP3y ago

If I were going to cite evidence for Alphabet’s “supremacy” in AI, I would’ve picked something more novel and surprising such as AlphaFold, or perhaps even Gato.

It’s not clear to me that Google has anything which compares to Reality Labs, although this may simply be my own ignorance.

benreesman3y ago

Then we get Gato, Imagen, and PaLM in the space of like what, 2 months?

Clearly I’m speculating that someone pulled the trigger, but I don’t think it’s like, absurd.

1 more reply

j / k navigate · click thread line to collapse