Also I think you will find that returning your skin to a normal temperature after chilling in in any way (water, aircon, etc) temporarily decreases core temperature after which you expend energy to return it to normal.
We may have had thousands (if not millions) of years to evolve solutions to heat conservation but they aren't perfect, or people would never die in freezing water or conditions.