A type of wishful thinking fallacy.
In law scale matters. It's legal for you to possess a single joint. It's not legal to possess 400 tons of weed in a warehouse.
It sounds then like you're saying that scale does indeed matter in this context, as using every single piece of writing in existence isn't being slurped up purely to learn, it's being slurped up to make a profit.
Do you think they'd be able to offer a usefull LLM if the model was trained only what what an average person could read in a lifetime?
That is intent of scale. To trigger LLMs to reach this point of "emergence". Whether or not it's AGI is a debate I'm not willing to entertain but everyone pretty much agrees that there's a point where the scale flips from a transformer being an autocomplete machine to something more than that.
That is legal basis for why companies would go for scale with LLMs. It's the same reason why people are allowed to own knives even though knives are known to be useful for murder (as a side effect).
So technically speaking these companies have legal runway in terms of intent. Making an emergent and helpful AI assistant is not illegal, but also making a profit isn't illegal either.
You could say the same in LLM training, that doing so at scale implies the intent to commit copyright infringement, whereas reading a single book does not. (I don't believe our current law would see it this way, but it wouldn't be inconsistent if it did, or if new law would be written to make it so.)
Scale is only used for emergence, openAI found that training transformers on the entire internet would make is more then just a next token predictor and that is the intent everyone is going for when building these things.
I think this is even more common and more brazen when it comes to "disruptive" businesses and technologies.
No wishful thinking here.
I'm not sure you understood what you said, but superficially it appears that you are agreeing with me?
Just because it's legal to read 100s of books does not make it legal to slurp up every single piece of produced content ever recorded.
We're talking man many orders of magnitude in scale there, and you're the one who pointed out that scale :-/
>Just because it's legal to read 100s of books does not make it legal to slurp up every single piece of produced content ever recorded.
The law says you're perfectly in your legal right to slurp up every piece of content ever produced.
>We're talking man many orders of magnitude in scale there, and you're the one who pointed out that scale :-/
I'm aware, and the law doesn't talk about scale.
But tens (hundreds?) of thousands of books over the span of a few weeks? That's definitely "scale".
