Also if I had said 'partner' or 'friend' instead of 'wife' the comment would probably have been less humorous, as it plays on a cliche situation. I'd rather not live in a world where I'd have to neuter such a statement in the name of absolute correctness over common sense.
The cliche draws upon a stereotype. It’s not a great stereotype to perpetuate in my opinion. Better to just not make the joke than to perpetuate a stereotype.