You are demonstrating unfamiliarity with the difference between
de jure and
de facto. When people of other demographics feel uncomfortable being there because the audience is predominantly white and predominantly male, it is effectively exclusionary even if there isn't a single word to the contrary.
That doesn't make it your fault. It does mean it's something you have to make a real effort to understand and accept in light of the privilege you have.