There has to be a give and take, otherwise this logic can justify many things we've decided is wrong. Things for example like firing someone trying to unionize or for being black or a woman. There is a difference between freedom of association for an individual and freedom of association between a company/organization and its employees or members and that is part of why.
We have a give and take, it's called a "protected class". It lists reasons you can't fire people and includes reasons like being black or being a woman. We can update the laws to protect people in other classes as well, but I don't see anyone clamoring to make "defenders of pedophiles" or "creeps who alienate their coworkers" a protected class.
Yeah, because that would solve problems. Honestly, the same people out for blood would be church types in the 18th century. And by the mail in question I sincerely question the competence of the MIT in general, because Stallman was actually correct.