Quite the opposite. I'm arguing in principle, and if you would like to redefine these words, I have no problem with that.
> try taking one more step and imagine how this "racism" would manifest itself
I fear that people continue to only associate with those in their small bubble of what makes them comfortable instead of reaching out and forging relationships that span sex/color/religion. Maybe they feel that they need a "black" version of education, and feel less confident in their ability to thrive in a multi-cultural society. Instead of opening up to new cultures we cement ourselves in one that we can identify with based on superficial factors like skin color and clothes.
I'm actually arguing from a position deeper than "white men should be able to join too!". Please, don't take it that way. I think humans tend to be afraid of people who are different than themselves, and that's the core issue. When we turn our skin color into an identity, we segregate ourselves from so many people. I think something like sports/teambuilding would be more effective at breaking down the barriers we're witnessing in our society.