Right, this is the premise of the siege mentality, but you haven't actually established it. I have not observed any "structural power imbalance" in favor of sexual harassment. I've observed that sexual harassment happens, but that's no more than you can say about any other socially unacceptable behavior.
Since this is the central premise of your argument, it is curious that you assume it as a given without even attempting to establish it. I wonder if you've even consciously questioned it yourself.
Next, 'Sexual harassment' isn't a side, nor is it an agent. The 'sides' are 'the sexists' vs 'the non-sexists', or if you want a more 'fighting words' formulation, 'the misogynists' vs 'the decent human beings' (with, in both cases, 'the neutrals' and 'the not-actually-involved's' as well).
The structural power imbalance is really really easy to observe. Look at the reactions to posts like the OP's, both here on HN and in terms of the comments (if their blog has comments enabled) or email they receive.
(As an aside, for this kind of thing Ashe Dryden's blog posts (for example) are better than the OP's at cataloguing responses and providing links.)
There are replies after incident reports like this kind of thing saying that the woman deserved it, or "can't you guys take a joke", or that it doesn't actually happen, or "it doesn't count as sexism, the real sexism is that sexist acts get called sexism" etc etc etc. The people who respond that way seem to honestly feel like they are in the majority - and, as incident reports seem to indicate, many conference organisers act like they agree with that assessment. If there is a mechanism that protects people who perform sexist actions, and protects the protectors, and so on, and no counterbalancing mechanism, then that there is evidence of a structural imbalance.
As for "it happening more than other socially unacceptable behaviour" - we actually have papers that measure the effects of this kind of thing, and their conclusions were uniformly that sexism and misogyny were serious problems with large negative effects in computer science and programming fields. The problem with racism is probably comparable, but fortunately many of the things that have proven effective at reducing sexism related problems also reduce racism related problems, and the two things do not work at cross purposes, so it isn't an either-or problem. The problem with magical thinking and a lack of education might be comparable, but efforts to fix general rationality deficits are often hampered by having entrenched specific rationality deficits, and basic exercises in testing reality and accepting conclusions if pointed at dissolving the various edifices that protect racism and sexism in the industry would advance both causes, so this also isn't an either-or. If you were a marxist or anarchist I could see you thinking that the wage-labour situation was more pressing than any of the above, but again modern marxist and anarchist thought seems to be going in the direction that microfascisms that are allowed to fester will result in macrofascism, plus the whole ideas of perpetual revolution and solidarity, so dealing with racism and sexism are pretty important in order for the revolution to actually be able to win at all.
Does this answer your points?
You made more or less the deflection that I was expecting--that the issue isn't just sexual harassment, but rather that society, in aggregate, empowers men at the expense of women. But see, even there you are committing a fallacy of division, because it doesn't follow that the status quo on every individual issue empowers men at the expense of women. So you can't pull in that overarching narrative of heroic feminists vs. evil misogynists that easily.
The other rhetorical trick feminists use to bully people into not disagreeing so loudly is one you've pulled here--the notion that disagreeing with feminism is equivalent to misogyny. From my perspective, all you're doing is making a fallacious argument that the misogynist power structure has made sexual harassment acceptable in our society, waiting for people to say "no it hasn't", and then turning around and saying "yes it has--just by disagreeing with my argument you are part of the misogynist power structure." This argument isn't in good faith. It's isomorphic to what I call the Ayn Rand fallacy. Ayn Rand's fallacy was: all my beliefs are governed by a philosophical system built on reason, therefore if you disagree with me you are irrational. In Ayn Rand's inner circle, wearing beards or enjoying Mozart were considered irrational because they conflicted with Ayn Rand's tastes. You are making an isomorphic argument: my beliefs are governed by a philosophical system predicated on not hating women, therefore if you disagree with me you hate women. In making this argument, feminists don't reveal their interlocutors as misogynists; they reveal themselves as Ayn Rand--arrogant ideologues with no consideration for the possibility they might be fundamentally mistaken.
Furthermore, you also haven't sufficiently demonstrated your premise that society, in aggregate, empowers men at the expense of women. You say it's "really easy to observe", and it may be through your lens. Through mine, it's clear that in some things, men are indeed empowered at the expense of women, while in other things, women are empowered at the expense of men. I wouldn't be confident in any aggregation of either side's various privileges and oppressions. Neither of us has a perfectly clean lens to start with, and it isn't helpful to exclusively look through the biased lens of a tendentious ideology, either.
Fundamentally, any such aggregation wouldn't be of much use anyway because individual issues should be addressed one at a time, empirically, not through some overarching ideological narrative. It's funny that you mention Marxism, because you seem to share with them the same central error: you have not yet outgrown the naive notion that one overarching philosophical system can solve all problems. Life is more than just some manichean narrative where you can cast yourself on the side of the heroes. It is more complicated than that.
(P.S. If you're interested in continuing this, please feel free to email me. This thread is rapidly growing stale, but this discussion has already helped me to clarify a lot of thoughts I've been having recently, so I do appreciate and value it.)