If you see a comment or article you approve of, you should upvote it, regardless of how popular (or not) it is among other people. If there are 529 people who approve of a certain comment, then it comes by those 529 upvotes honestly.
Everyone thinks they're just giving them a little bit. But all those little bits add up to a lot. Without any visibility into how much other people have already given them, there is no way for people decide if they really need the money or not.
If you saw a beggar on the street, and knew that they'd already been given $1000 so far that day, would you still give them a dollar?
In short, as-is, the OP really has no way of knowing just how aggravating his site is.
I'll concede that 127 visible upvotes are much preferable to 127 "me too" replies, though!
1. Those who aren't familiar with the topic. With a single-digit score, there's potentially not much value there. With a three-digit score, hey, maybe people shouldn't use large images in the background. Then this person can either follow it or look up or ask why this is. It's marking significance to an otherwise unknown topic.
2. Similarly, if the author of the site sees this, they can see just how many people agree with this. This is important because again, high score means people agree with it. This person may still not take action, but should at least consider it if it's a high enough score. Maybe even chime in and respond with a rationale.
3. The rest of the community. This will either acknowledge your own views or raise a red flag if the score is high enough. Did you miss something? Are enough people just plain wrong according to what you believe to be true? Should you post a counter-point or add/ask for clarification?
I guess what I didn't take into account in my previous comment is that the guy responsible for the animated GIF might drop by HN and might take the GGP's comment more seriously if he sees a large number of votes attached to it. This makes sense and is another good reason to get vote counts back here, especially if it discourages others from posting me-too comments.