> I don't see why you couldn't do the same thing to e.g. the binary of the Windows kernel
Forget the binary; there have been Windows code leaks every now and again over the years. Feed one of those into a model, start generating code for ReactOS, and see how long until MS decides that actually AI is infringing...