Men are stronger and faster and not just a little bit. If you allow men in women's sports, (basketball, soccer, boxing etc) then women will not be competitive in those sports.
Male puberty changes body composition in non-reversible ways. Muscle distribution, composition, quantity and bone density, all favor men that have gone through puberty.
