When you come to a new site, it's usually considered civil to learn the rules of that site, although of course I cannot force you to do so. For example, assuming that reddit rules apply on HN is kinda silly. Relatedly, you might be interested to note that there are dozens of HN old-timers that will automatically downvote (nearly) any comment that contains a "downvote me if you want" or a "don't just downvote me if you disagree" or "these downvotes are BS" or something.
You're right, though, that not all users agree. And hopefully you learn from this comment that it's okay, on HN, to downvote me if you disagree, even if you think I contributed to the conversation.
PS: welcome to HN.