If I live in Virginia until adulthood, then move to American Samoa, my gender expression is going to radically change. I'll start wearing skirts. If I then move to Qatar, my gender expression will change again. I might still think of myself as a man through those changes, but whats expected of me and how I view myself with those cultures will shift. "What does it mean to be a man?" Is very different, globally.

So even if I consider myself a man, that definition regularly changes for different contexts. Clothing, conversational style, physical affection (it's common for men to hold hands in parts of the Middle East, and considered uncomfortable in the states.)

Given gender expression and identity can change as you transit cultures, surely you can see that some people might belong to cultures whose definition of "what does it mean to be a man" might be "whatever the fuck you want". Punk and queer subcultures, for instance, have different gender expectations than say, the Vatican.

For some cultures, genitals have little to no bearing on one's social expectations. Fit into the role that feels right, who cares about what is in your DNA.