undefined | Better HN

0 pointsenriquto1y ago0 comments

It's alarming that he refers to llama as if it was open source.

The definition of free software (and open source, for that mater), is well-established. The same definition applies to all programs, whether they are "AI" or not. In any case, if a program was built by training against a dataset, the whole dataset is part of the source code.

Llama is distributed in binary form, and it was built based on a secret dataset. Referring to it as "open source" is not ignorance, it's malice.

0 comments

Nesco1y ago

The training data contains most likely insane amounts of copyrighted material. That’s why virtually none of the “open models” come with their training data

enriqutoOP1y ago

> The training data contains most likely insane amounts of copyrighted material.

If that is the case then the weights must inherit all these copyrights. It has been shown (at least in image processing) that you can extract many training images from the weights, almost verbatim. Hiding the training data does not solve this issue.

But regardless of copyright issues, people here are complaining about the malicious use of the term "open source", to signify a completely different thing (more like "open api").

tempfile1y ago

> If that is the case then the weights must inherit all these copyrights.

Not if it's a fair use (which is obviously the defence they're hoping for)

1 more reply

jdminhbg1y ago

> In any case, if a program was built by training against a dataset, the whole dataset is part of the source code.

I'm not sure why I keep seeing this. What is the equivalent of the training data for something like the Linux kernel?

enriqutoOP1y ago

> What is the equivalent of the training data for something like the Linux kernel?

It's the source code.

For the linux kernel:

          compile(sourcecode) = binary

For llama:

          train(data) = weights

jdminhbg1y ago

That analogy doesn't work. `train` is not a deterministic process. Meta has all of the training data and all of the supporting source code and they still won't get the same `weights` if they re-run the process.

The weights are the result of the development process, like the source code of a program is the result of a development process.

j / k navigate · click thread line to collapse

0 comments

Nesco1y ago

The training data contains most likely insane amounts of copyrighted material. That’s why virtually none of the “open models” come with their training data

enriqutoOP1y ago

> The training data contains most likely insane amounts of copyrighted material.

But regardless of copyright issues, people here are complaining about the malicious use of the term "open source", to signify a completely different thing (more like "open api").

tempfile1y ago

> If that is the case then the weights must inherit all these copyrights.

Not if it's a fair use (which is obviously the defence they're hoping for)

1 more reply

jdminhbg1y ago

> In any case, if a program was built by training against a dataset, the whole dataset is part of the source code.

I'm not sure why I keep seeing this. What is the equivalent of the training data for something like the Linux kernel?

enriqutoOP1y ago

> What is the equivalent of the training data for something like the Linux kernel?

It's the source code.

For the linux kernel:

          compile(sourcecode) = binary

For llama:

          train(data) = weights

jdminhbg1y ago

The weights are the result of the development process, like the source code of a program is the result of a development process.

j / k navigate · click thread line to collapse