Maybe upvotes/downvotes could be weighted by the karma of the person giving them, thus putting more emphasis on those with high karma (which might sound elitist, but the only reason those people have high karma is because people think they talk sense).

This would concentrate power slightly in the hands of those long time dedicated commentators ('the community'), and slightly lower the strength of newcomers who are still getting used to the ettiquette ('the mob').

Hopefully this would go someway to lowering the effect of newcomers, until they rack up enough karma themselves to 'graduate' into the higher karma ratings.