If a set of OpenAI model weights ever leak, it would be interesting to see if OpenAI tries to claim they are subject to copyright. Surely it would be a double standard if the outcome is distributing model weights is a copyright violation, but the outputs of model inference are not subject to copyright. If they can only have one of the two, the latter point might be more important to OpenAI than protecting leaked model weights.
At least in the US, a derivative work is a creative (i.e. copyrightable) work in its own right. Neither AI models nor their output meet that bar, so it's not clear what the infringing derivative work could be.
They are however, considered to be infringing.
My point is that with LLM outputs that's not true - according to the copyright office they are not themselves expressive content, so it's not obvious how they could infringe on (i.e. contain the expressive content of) other works.
For example, suppose if I photograph a copyrighted painting, and then started selling copies of the slightly-cropped photo. The output wouldn't have enough originality to qualify as a derivative work (let alone an original work) but it would still be infringement against the painter.
