Not sure if that's really solvable with rules, though.

My experience with downvotes is that people mostly use it as a "I don't like this" button, which is proxy for "I couldn't think of a counterargument so I don't want to look at it."

(I noted recently that downvotes and counterarguments appear to be mutually exclusive, which I found somewhat amusing.)

Whereas I will often upvote things I personally disagree with, if they are interesting or well reasoned. (This seems objectively better to me, of course, but maybe it's personality thing.)