To be completely honest I find down-vote button way to close to up-vote button, so sometimes I am not sure if I clicked right one.
So some of them might be accidental.
It's also annoying that the arrows disappear the instant that you click on them. If the up/downvote was just highlighted after a click, you could then let users click again to correct their choice.