I may be missing your point. If you mean that two men shouldn't have to stop meeting at a bar when they're both comfortable with it, simply because there's a woman on the team who wouldn't be comfortable in the same situation, well... let's say your team has a habit of meeting at a Brazilian steakhouse to make important decisions. Then you hire Dave's brother Jerry, who is a strict ethical vegetarian. Don't you think it might be a decent idea to pick a different restaurant? Jerry swears he's willing to come along to the steakhouse and bring his own food, though you notice he looks pretty queasy and never touches his food the whole time you're there.

Or maybe you think it's fine if he quit if he's so uncomfortable, your team just isn't right for women oops I mean vegetarians and they'd be better off working somewhere else.