I think you are mistaken to think that being a man, in itself, is what causes flirting behavior to be hit with a sexual harassment complaint. The greater factor is that the man in the equation is likely to be the one in power, and thus the woman has a reason to feel threatened if she rebuffs him.
To be fair, that's quite subtle. Power relationships are often invisible and masquerading as something "light." It's easier to understand if you've been on the short end of the stick a bunch of times.
