Ah. You have a very limited background. Right.
We have been the same species for thousands of years. And for thousands of years there have been male and female (sex) groups that have presented as men and women (gender), that is not the same as the one you might assign to them. (The galli priests of Rome, the Hijras of India, the khanith of Arabia, etc.)
And no... No, a child is not always able to tell them apart. Nor does a child, necessarily, instinctively call them either man or woman. If you were to ask a child of the Tiwi tribe what one of their ladyboys are, today, man or woman, they would answer "ladyboy". If you press the point - the child becomes confused. The ladyboys have been present in the Tiwi for roughly the last eight to ten thousand years or so ("kirrijamiyagirraga" in Tiwi). They are as much a part of the social fabric as man or woman. Tiwi is a gendered language, and has three genders (kirri - male, makirri - female, rraga - ladyboy), as well as a fourth tense that is used for gender-neutral terms (ramiya). And, well, if you assume that they're all genetic males, you'll get that wrong as well.