The IRS’s actual power in this regard is extremely limited. There’s so much wiggle room to define the “mission” that, at best, the only thing they can do is make sure you’re not secretly using your nonprofit as a tax shelter for a regular for-profit business, and even then some companies find ways around it (see IKEA).

Ordinary fraud (as opposed to tax fraud, though maybe that too) is even easier to pull off with a nonprofit than with an for-profit. Any nonprofit has a fundraising mechanism (which is usually an obnoxious set of dark patterns and outright spam reached through decades of optimization; the techniques are well understood), some overhead for managing the organization and money itself, and then the actual work involved with the mission. It’s quite simple for the overhead and fundraising parts of the organization to become most of the organization. But it gets worse: when your “mission” is vague enough, you can also turn that side of the business into a bunch of sinecures for your friends. There are well known “awareness” nonprofits that do literally nothing to actually solve the problems they’re nominally about because their job is to “spread awareness” (i.e. the top of the funnel for the fundraising part of the organization).

This is ironically also the origin story of “effective altruism”. The IRS doesn’t stop you from running a non-profit scam, so you have to do your own research to find out which non-profits are legit, and that’s what the effective altruists started off with, along with some cost-benefit modeling once you get past the outright scams.

The problem with OpenAI seems to be related to a vague mission as well. From the “an organization is what it does” perspective, OpenAI develops AI. If you have a formal mission that emphasizes safety over capability, you could certainly argue that OpenAI was closer to achieving that mission by sabotaging its own work, but if your interpretation of “safety” is to sabotage the development of AI, there are certainly better ways to do it than to develop AI and then sabotage yourself.

One reason open source software projects struggle to get donations is because the IRS is skeptical of OSS as a true non-profit activity, so often doesn’t allow 503c tax deductible donations. OpenAI seems like a data point in favor of the IRS skepticism.