Because we have easily observed societal norms that have to account for some percentage of the difference, without measuring differences absent those norms we have no conceivable way to know that hereditary traits contribute to any percentage of the differences.
Because many historical cases of what were thought to be fundamental differences between the genders turned out to be strictly socially informed. I think it's more likely that the current suite of stereotypes is also incorrect than that we just happened to finally get it right.
