I see what you're saying in some cases, but in the cases where the user is explicitly attempting to create images of copyrighted characters (e.g. the Mario example), they would definitely know. I honestly don't see this as a practical issue - as far as I'm aware (and like most on HN I follow these things more than the average person), there aren't a lot of concerns about inadvertent generation of copyrighted material. It's certainly not at issue in the NYT lawsuit.
2. Before you, the AI consumer, uses the generated result, a company (in this case OpenAI) is already charging for it. I'm currently paying OpenAI. That AI is currently able and willing to sell me copyrighted images as part of my subscription. Frankly that should be illegal, full stop.
Totally fair, but I feel like it's a bit more of a gray area. If I use Photoshop to create or modify an image of Mario for personal use, we'd call that fine. I grant you that here OpenAI is doing more of the "creating" than in the Photoshop example, but we still do generally allow people to use paid tools to create copyrighted images for personal use.
I'd also pose a question to you - what if OpenAI weren't charging? Is it acceptable to train an open source model on copyrighted images and have it produce those for personal use?
I guess I just understand the law to revolve more around what the end product is used for, as opposed to whether a paid tool can be used to create that end product.