This ban doesn't punish the machine; it doesn't care whether or not it's banned from Twitch. It does two things:

* Makes sure the people controlling the machine are incentivized to correct it (not that I think they needed it, but what I think is irrelevant)

* Ensures that content that Twitch management doesn't want on their platform, and which is in violation of their TOS, is removed immediately, regardless of the thoughts/actions of the people controlling the machine.

Yeah, but you also don't necessarily allow it to continue to broadcast while exhibiting errant behaviour. Your interpretation of this as a "punishment" of the machine isn't necessarily correct, nor the only perspective that's ultimately important here.

As far as Twitch is concerned, there's a channel that's breaking TOS, repeatedly. They're not "punishing" the owner of that channel, they're just enforcing the TOS, something the channel owner agreed to when creating their channel.

OpenAI (which powers this Seinfeld generator) is putting significant resources into preventing it from creating unsafe and antisocial material. Until that is prefected, outside entities like Twitch can use traditional moderation tools, such as the 14 day ban issued in this case.

The point is that whether it's an RNG or language model or just a program printing a list of numbers like I posted, it's just a state machine spitting stuff out, it has no intention. So it's not really productive to get mad at it just because it produced something you don't like. The language model is spitting out random stuff based on the representation it has learned. It doesn't know what it means, nobody is making any kind of value judgement that results in it saying stuff, so why get offended by it?

Edit:

  Because if our adversaries are the products of inanimate and unthinking evolution, we cannot regard the problem in terms of revenge or payback... that would be no different than whipping the ocean for having sunk a ship and drowned its sailors 

  - Stanislaw Lem, The Invincible

The output has an element of randomness however. Regenerating the output will give you a different, albeit similar, answer.

In that case, instead of Seinfeld, AI should make a show about something.