"I think he behaved poorly in relation to two topics I care 
   about: leading open source projects, and ending sexism."

I would think that the best way to do that is to lead by example by leading and open source project and combatting sexism in the project(s) you lead instead of knocking down my strawmen.

If you lead an open source project and fight to end sexism, then great. If not, you are in no position to criticize Ben.

   "I also think people [who have contributed to] the project have 
   a right to criticize him. [Everyone else can GTFO]. He does owe 
   them an explanation; any action taken in the project's name is 
   subject to review by the project community."

FTFY. This entire ordeal would have been a non-issue if you could mute non-contributors.

My true agenda? Establishing a culture where non-contributors don't waste the time of contributors with their bikeshedding.

What excludes someone from the category of non-contributor? They submit test cases. They clarify documentation to make it clearer how to use the code or understand the algorithms in the code. They implement features. They participate in design discussions. They contribute use cases and elaborate on why those use cases are important. They fix bugs in the open source code instead of simply hacking around it in their own code. They write blog posts on how to use the open source code. They publish examples of the open source code in use. They give talks on how to use the open-source code.