I find that downvotes generally account for well below 10% of my voting, and I use them mostly to denote needed correction in tone or presentation, or repeated endorsement of a "fact" without reference after it's been asked for. Even then, I'll only do so after leaving a comment or confirming someone else has a comment that covers my reasoning for downvoting. Without some explanation, it's far to easy to assume downvotes are due to disagreement on what is all to often a subjective item, which doesn't really help the discussion at all.

A Poll on people's downvoting behavior would be interesting, but as it would be self-reporting and subjective, the results would be next to meaningless if mundane.