> You can't reproduce watermarks that aren't in the training set.

You most certainly can reproduce watermarks that aren't in the training set. It can synthesize novel watermarks the same way it can synthesize novel images.

The vast majority of output is non-infringing.

> It's outside of the rights granted by the copyright holders in most cases (MIT and CC are probably the few licenses which allows this, and even CC/NC wouldn't allow it and MIT falls afoul of the attribution clause).

Except

a) This assumes that training isn't covered under fair use

b) attribution is only required for output that infringes - not the model itself.

Most of your arguments seem to apply to the output not to the model itself or the act of training it.

> I disagree, because it could be prevented by not training on copyrighted material in the first place.

This.

> The legal system hasn't yet reviewed such widespread infringement yet to create precedent. But eventually it will, and I don't believe it will be kind.

There you go, we'll see how this is tested against the AI bros once the legal system catches up.