Regardless, the point is that QoS exists to keep non-video traffic from getting trampled out by video traffic. Video traffic is such a disproportionately large amount of total Internet traffic that virtually the only services that would be significantly impacted by being in the "slow lane" are video services. The links wouldn't even be saturated in the first place without video traffic.
EDIT: Also wanted to add that congestion is almost guaranteed with adaptive bit rate streaming. Netflix will use as much bandwidth as is available up to like 9 mbit/s.