not always true. Minor phase mismatch will create higher frequency sound. You wouldn't hear ultrasound, though how bad it is to have an ultrasound source in you ears i don't know.

The phase mismatch can be temporal - circuitry reacting either a bit too fast or too slow - as well as spatial - the sound wave front most probably aren't parallel to the speakers frontal plane.