>By not voting, I've surrendered to what the entire crowd wants by average (which might be entirely counter to the direction I want).
By having a democracy you've surrendered to what the entire crowd wants by average, by not voting you've surrendered to what the rest of the crowd wants by average, and as the size of the crowd grows the distinction between these two states of surrender approaches zero.