undefined | Better HN

0 pointsAndrewKemendo2y ago0 comments

A false dilemma, also referred to as false dichotomy or false binary, is an informal fallacy based on a premise that erroneously limits what options are available.[1]

[1]https://en.wikipedia.org/wiki/False_dilemma

0 comments

JumpCrisscross2y ago

These models cost millions to train. The only reason open-source LLMs have a heartbeat is they’re standing on Meta’s weights. The only third path is a public option.

LoganDark2y ago

> The only reason open-source LLMs have a heartbeat is they’re standing on Meta’s weights.

Not necessarily.

RWKV, for example, is a different architecture that wasn't based on Facebook's weights whatsoever. I don't know where BlinkDL (the author) got the training data, but they seem to have done everything mostly independently otherwise.

https://github.com/BlinkDL/RWKV-LM

disclaimer: I've been doing a lot of work lately on an implementation of CPU inference for this model, so I'm obviously somewhat biased since this is the model I have the most experience in.

JumpCrisscross2y ago

My personal bet is specialised models have a niche. Do you think one of these could compete with GPT if e.g. trained on a law firm’s correspondence and contracts?

1 more reply

DirkH2y ago

Didn't the whole "we have no moat" paper show how this is actually not the case and that the future is far brighter for open-source LLMs?

j / k navigate · click thread line to collapse

0 comments

JumpCrisscross2y ago

These models cost millions to train. The only reason open-source LLMs have a heartbeat is they’re standing on Meta’s weights. The only third path is a public option.

LoganDark2y ago

> The only reason open-source LLMs have a heartbeat is they’re standing on Meta’s weights.

Not necessarily.

https://github.com/BlinkDL/RWKV-LM

disclaimer: I've been doing a lot of work lately on an implementation of CPU inference for this model, so I'm obviously somewhat biased since this is the model I have the most experience in.

JumpCrisscross2y ago

My personal bet is specialised models have a niche. Do you think one of these could compete with GPT if e.g. trained on a law firm’s correspondence and contracts?

1 more reply

DirkH2y ago

Didn't the whole "we have no moat" paper show how this is actually not the case and that the future is far brighter for open-source LLMs?

j / k navigate · click thread line to collapse