I struggle with this a lot, too. Maybe "complex numbers" is a misnomer, and we should call them "complex operations", or "complex methods".
I have a hunch that for math people complex numbers are tied really close to the operations you perform with them. But to me, the lay man, the numbers are just data.
From reading around on HN I infer that there are certain things that are grouped together because they share certain actions. Groups, sets, monads.
However, if you were to encounter the number (1, 5) in the wild, you wouldnt know if it was a complex number or just a 2 dimensional vector until it is operated upon.