Then there's the source. First, Most Vimeo content is produced by creative professionals so the content itself is meant to look amazing. In contrast, YouTube does bulk video, most of which is homemade. Vimeo looks better on content alone.
Second, the Vimeo video is directly encoded from the creator. YouTube's creative content tends to be commercial work posted via distributors and agencies. It's likely that the content shown on YouTube has been encoded twice - first by the creator for the distributor, then again by YouTube's uploader.
There's other things. It's probable that Vimeo is using 2-pass encoding, allowing for more accurate VBR. It takes longer, but looks much nicer. YouTube users want speed, so one-pass may be standard.
Generally, the first pass is to determine which sections are static vs. which are busy. The second pass then takes advantage and can adjust the rate. So for long talking head scenes, it may drop the data rate to 1.2 Mbps, and action scenes can be ramped up to 3Mbps+ so the action doesn't look blocky.
YouTube has been so slow to load on my iOS devices lately that I'm loath to watch any video content from YouTube at all.
