They did control for obvious appearance differences (e.g. color & type of clothing, hair length) and morphology (e.g. height/body size), even people's approach. But not more subtle traits such as gait, waist-hip ratio, odor...
One hypothesis suggested that in early history, women may have more commonly caught smaller prey (birds) than men did, and this fear could be evolutionarily ingrained.