Speaking as a regular HN downvotee, I'd say "don't like your facts" accounts for at most 25% of downvoting.
Sure, you can always make a moderation system more complex. At best, that will somewhat reduce the complaints. At the cost of both the added complexity, and all the new-and-undesirable ways in which humans users can interact with that shiny new complexity.
